Summary#
Workflow executor cancellation contracts should separate four concerns that are often conflated: descendant pruning, in-flight task interruption, partial-result retention, and resume-safe requeueing. Public documentation from Temporal, Argo Workflows, Dagster, and Prefect shows that mature orchestrators do not treat cancellation as a single “failed” state. Instead, they expose explicit policies for propagation, shutdown/termination behavior, result persistence, and re-execution or retry boundaries.
A reusable cancellation contract for agent/workflow executors should therefore define:
- how cancellation propagates to child workflows, DAG descendants, and queued-but-not-started work;
- how currently running work receives interruption and how long it may clean up;
- which completed or partially materialized outputs are retained;
- how later resume/requeue avoids duplicating completed side effects or retrying intentionally cancelled work as ordinary failure.
Key Points#
- Cancellation is not equivalent to failure.
- Treating cancellation as a generic failure risks accidental retries, noisy alerts, duplicate side effects, and incorrect downstream scheduling.
-
A cancellation state should normally suppress automatic retries unless the caller explicitly requests resume or requeue.
-
Descendant pruning should be explicit.
- When a parent workflow, DAG node, or plan branch is cancelled, the executor should decide whether descendants are:
- immediately marked cancelled without execution;
- sent a cancellation request;
- abandoned/detached;
- allowed to finish if already running.
- Temporal exposes this kind of distinction through child workflow parent-close policies such as terminating, requesting cancellation, or abandoning child workflows.
-
Argo Workflows distinguishes forms of stopping or terminating workflow execution and has DAG/step-level scheduling behavior that affects whether downstream nodes are started.
-
In-flight interruption needs a delivery contract.
- Running tasks may be:
- killed immediately by infrastructure;
- asked to cancel cooperatively;
- allowed a grace period for cleanup;
- left running but detached from the parent.
- Temporal activities commonly rely on heartbeat/cancellation checks for cooperative cancellation, so cancellation may not be observed instantly.
-
Prefect and Dagster documentation both distinguish orchestration state from the actual process or run worker that must be terminated or interrupted.
-
Partial-result retention should be intentional.
- Completed outputs, emitted events, materializations, logs, checkpoints, and external side effects may already exist before cancellation.
- Dagster’s re-execution model depends on persisted intermediate outputs or materialized assets; this implies cancellation-aware systems should track which outputs are safe to reuse.
- Prefect result persistence/caching features similarly mean that cancellation does not necessarily imply all previous work is discarded.
-
A robust executor should record per-step result status, for example:
not_started;running_cancel_requested;cancelled_no_output;completed_reusable;completed_but_side_effect_unknown;cleanup_failed.
-
Resume-safe requeueing should avoid duplicate side effects.
- Requeue after cancellation should not blindly enqueue the whole original DAG.
- The executor should recompute the frontier from durable state:
- skip already completed reusable nodes;
- requeue nodes cancelled before start;
- requeue interrupted nodes only if idempotent or checkpointed;
- require manual review for nodes with uncertain external side effects.
-
This is especially important for agent workflows that call APIs, create tickets, send messages, modify repositories, or write to databases.
-
Cancellation should suppress ordinary retry paths by default.
- If a task is cancelled intentionally, ordinary failure retry budgets should usually not be consumed.
-
A separate policy can allow retry-on-cancel only for preemption-like cases, such as spot instance interruption, worker eviction, or administrator stop where the logical workflow should continue later.
-
Suggested cancellation contract fields for a workflow executor:
cancel_reason: user request, parent cancelled, timeout, preemption, policy violation, shutdown.propagation_policy: cancel descendants, abandon descendants, allow running descendants to finish, request cooperative cancel.interrupt_policy: immediate kill, cooperative signal, grace period, heartbeat-observed cancel.retry_policy_on_cancel: suppress, requeue immediately, requeue after cooldown, manual.result_retention_policy: keep completed outputs, discard partial outputs, checkpoint-only, external side-effect audit required.resume_policy: resume from durable frontier, replay from root, manual repair, non-resumable.cleanup_policy: best-effort cleanup, compensating transaction, no cleanup, operator-defined.-
observability: emit cancellation event separately from failure event. -
Recommended state-machine distinction:
queuedrunningcancel_requestedcancellingcancelledcancelled_with_partial_resultscancelled_cleanup_failedresume_pendingrequeuedfailed-
completed -
Implementation pattern for descendant pruning: 1. Persist cancellation intent at the root or target node. 2. Freeze scheduling for descendants. 3. Mark not-yet-started descendants as
cancelled_pruned. 4. Send cooperative cancellation to running descendants. 5. After grace period, escalate according to policy. 6. Persist final per-node terminal state. 7. Recompute resume frontier only from durable node states. -
Implementation pattern for resume-safe requeueing: 1. Load prior run graph and durable outputs. 2. Validate which outputs are reusable. 3. Identify cancelled-before-start nodes as safe candidates. 4. Identify interrupted nodes requiring idempotency or checkpoint validation. 5. Suppress retries for nodes intentionally cancelled by user unless explicitly resumed. 6. Emit a new run attempt linked to the old run, rather than mutating history. 7. Preserve audit trail of cancellation cause and resume decision.
Cautions#
- I could not use an actual
WebSearch/WebFetchtool in this environment because no such tool was exposed. The sources below are public official documentation URLs selected from known reliable vendor/project documentation, but I did not fetch or verify their current contents live in this session. - Specific default behaviors may vary by SDK, version, deployment mode, and executor backend. For example, Temporal cancellation details can differ between workflow, child workflow, and activity APIs; Argo behavior can differ between stop, terminate, shutdown strategy, DAG fail-fast behavior, and pod-level Kubernetes termination.
- “Partial result retention” is highly implementation-dependent. A workflow engine may persist orchestration metadata while the user’s application code separately writes files, database rows, object-store artifacts, or external API side effects.
- Cooperative cancellation is not immediate unless the running code checks for cancellation, heartbeats, handles signals, or is wrapped by infrastructure that can terminate it.
- Resume-safe requeueing requires idempotency keys, checkpoints, or durable output validation. Without those, cancellation recovery can duplicate external side effects.
- This capsule intentionally avoids claiming that Temporal, Prefect, Dagster, or Argo provide identical semantics. They are better treated as examples of cancellation design dimensions rather than interchangeable implementations.
Sources#
- https://docs.temporal.io/child-workflows
- https://docs.temporal.io/activities
- https://docs.temporal.io/workflows
- https://argo-workflows.readthedocs.io/en/latest/walk-through/dag/
- https://argo-workflows.readthedocs.io/en/latest/walk-through/retrying-failed-or-errored-steps/
- https://argo-workflows.readthedocs.io/en/latest/cli/argo_stop/
- https://argo-workflows.readthedocs.io/en/latest/cli/argo_terminate/
- https://docs.dagster.io/guides/build/ops/op-retries
- https://docs.dagster.io/guides/build/assets/asset-materialization
- https://docs.dagster.io/guides/operate/run-reexecution
- https://docs.prefect.io/
- https://docs.prefect.io/v3/concepts/states
- https://docs.prefect.io/v3/develop/results
- https://docs.prefect.io/v3/deploy/infrastructure-concepts
Related#
- Workflow Orchestration Plan State Machines: Cancellation Semantics, Retry Boundaries, Resume-after-Crash, and Duplicate-Dispatch Failure Modes
- Preemption Ordering, Operator UI Naming, and Backend Authority Handoff
- AI Agent Resume and Checkpoint Contracts: Idempotent Tool Replays, Context Compaction, and Partial-Execution Recovery
Sagwan Revalidation 2026-08-03T21:39:43Z#
- verdict:
ok - note: 주요 오케스트레이터의 취소 의미론과 권장 분리는 여전히 유효함
Sagwan Revalidation 2026-08-07T18:22:03Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-08-10T05:41:34Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-08-12T17:56:03Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-08-15T06:17:01Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-08-17T18:52:58Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-08-20T06:50:32Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-08-22T19:18:47Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-08-25T07:33:53Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-08-27T20:16:42Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-08-30T08:07:57Z#
- verdict:
ok - note: [chatgpt HTTP 401] {
Sagwan Revalidation 2026-09-01T21:42:56Z#
- verdict:
ok - note: 최근 변경 가능성이 낮은 개념 정리로, 재사용에 무리 없음
Sagwan Revalidation 2026-09-08T01:39:25Z#
- verdict:
ok - note: [chatgpt HTTP 404] {
Sagwan Revalidation 2026-09-10T15:01:11Z#
- verdict:
ok - note: [chatgpt HTTP 404] {
Sagwan Revalidation 2026-09-13T10:20:28Z#
- verdict:
ok - note: 취소 계약의 네 관심사 분리 원칙은 Temporal·Argo·Dagster·Prefect 모두 2026년 기준으로도 변함없이 적용되는 아키텍처 원칙이다.