[Feature]: Support single-run semantics for side-effecting workflow task steps

Feature request

PR: https://github.com/chaos-mesh/chaos-mesh/pull/4879

Introduce a workflow step type, or a dedicated execution mode for templateType: Task, that provides single-run semantics for side-effecting task steps.

templateType: Task is useful for shell-based workflow steps, but it is hard to rely on it for non-idempotent operational actions when pod recreation remains part of the step lifecycle.

This is especially relevant for scheduled workflows with historyLimit: 1.

After a workflow run completes, the last finished workflow and its related objects remain in the cluster until the next scheduled run. In practice, this creates a vulnerable interval between two scheduled executions.

While the old workflow still exists, its task pods may disappear for ordinary Kubernetes reasons, including:

• eviction;
• node removal;
• garbage collection;
• manual cleanup.

When this happens, Chaos Mesh may recreate the pod and the corresponding Task step may be executed again, even though the previous workflow run had already completed.

For workflows where Task steps include side effects such as notifications, alerting control, service operations, or environment selection, this makes the current execution model hard to use safely.

What seems to be missing here is a single-run lifecycle for side-effecting task steps, while still preserving the Task authoring model and stdout / exitCode support for branching.

Proposed solution

Consider introducing a new workflow step type, or an explicit execution mode for Task, with semantics like these:

• preserve the current Task-style authoring model;
• preserve stdout and exitCode for conditionalBranches;
• create the task pod only once;
• remove the pod after completion;
• if the pod disappears before result collection is finalized, mark the step as failed instead of re-executing user logic;
• store the step result in workflow state rather than treating pod recreation as the normal recovery path.

A name like EphemeralTask could fit this behavior, but the main request is the execution model rather than a specific name.

Additional context

In practice, current workarounds have noticeable trade-offs:

• setting historyLimit: 0 removes the vulnerable window, but also removes useful observability of the last run;
• making task output deterministic can reduce some downstream risks, but it does not prevent step re-execution itself;
• adding custom guards inside shell commands through the Kubernetes API requires extra RBAC permissions, creates extra API load, and pushes workflow safety into user code instead of the controller.

A simplified example is a scheduled workflow where one step performs an operational side effect, and another step returns stdout used by conditionalBranches:

apiVersion: chaos-mesh.org/v1alpha1
kind: Schedule
metadata:
  name: test-chaos-schedule
spec:
  concurrencyPolicy: Forbid
  historyLimit: 1
  schedule: "*/10 * * * *"
  type: Workflow
  workflow:
    entry: workflow
    templates:
      - templateType: Serial
        name: workflow
        children:
          - notify-test-started
          - disable-alerting
          - select-target
          - notify-test-stopped

      - templateType: Task
        name: notify-test-started
        task:
          container:
            command: ["/bin/sh", "-c"]
            args:
              - echo chaos-test started

      - templateType: Task
        name: disable-alerting
        task:
          container:
            command: ["/bin/sh", "-c"]
            args:
              - echo disabling alerting for chaos window

      - templateType: Task
        name: select-target
        conditionalBranches:
          - expression: stdout == "zone-a"
            target: pod-chaos-zone-a
          - expression: stdout == "zone-b"
            target: pod-chaos-zone-b
        task:
          container:
            command: ["/bin/sh", "-c"]
            args:
              - echo zone-a

      - templateType: PodChaos
        name: pod-chaos-zone-a
        ...

      - templateType: PodChaos
        name: pod-chaos-zone-b
        ...

      - templateType: Task
        name: notify-test-stopped
        task:
          container:
            command: ["/bin/sh", "-c"]
            args:
              - echo chaos-test stopped
    ```