Status

Motivation

Real-world pipelines rarely fit a clean "all-or-nothing" dependency model. A few recurring shapes that Airflow users encounter every day:

• Fan-out data ingestion. A team ingests partitioned data from 40 source systems in parallel, then runs a single aggregation task downstream. It is acceptable, and often expected, for a few partitions to fail on any given run (a source is temporarily down, a third-party API rate-limits). The aggregation should still proceed as long as, say, at least 35 partitions succeeded.
• ML training sweeps. An ML engineer trains 20 candidate models in parallel and a downstream task picks the best one. The downstream task should run when all 20 finish and at least 3 produced a valid model; anything less means the sweep itself failed and the downstream is pointless.
• Data quality gates. A platform team runs dozens of quality checks in parallel before allowing a publish step. The publish should proceed only when all checks finished and no more than one was skipped (and zero failed), with "skipped" treated as inconclusive.
• Multi-region replication. A job copies a dataset to N regions. A downstream notification task should fire when all copies are done, and at least a configured quorum succeeded.

None of these are exotic. They show up in analytics platforms, ML infrastructure, observability pipelines, and data platform teams across industries. They share the same shape: the decision to run a downstream task depends on counts and ratios of upstream outcomes, not on every upstream reaching one specific state.

Airflow's current TriggerRule is a flat enum of 13 hand-picked presets. When one of those presets happens to match a real-world shape, authoring is easy. When it does not, Dag authors fall back to one of three workarounds, each with real business cost:

1. A branching operator upstream of the real work. Adds a node to the Dag whose only job is to paper over a trigger-rule gap. The graph becomes harder to read, and a Dag-level concern (when should this task run?) becomes a task-level concern (compute a branch decision).
2. Handle it inside the downstream task itself. The task starts, inspects upstream TaskInstance states via the metadata DB, and either continues or short-circuits. This breaks the normal model where state transitions live in the scheduler, makes retries and SLAs behave oddly, and couples Dag logic to Airflow internals.
3. Accept a weaker approximation and suppress the problem. Use ALL_SUCCESS and let a tolerable failure fail the whole branch, or use ALL_DONE and let a bad upstream silently poison the downstream. Teams catch this in production, not in code review.

The core problem is that the set of useful shapes is combinatorial. Covering it with a preset enum means the enum grows every time a new shape is needed. The pattern does not scale: each new preset needs an enum value, an evaluator branch, tests, documentation, and a release cycle before authors can use it. Meanwhile the authoring story for Dag authors stays the same: wait for a preset or work around the gap.

This AIP proposes an additive authoring surface that lets Dag authors describe the condition they actually want, in one line, without waiting for a new preset. The 13 existing enum values remain valid and unchanged; they simply become shortcuts for specific expressions.

Considerations

• No change to failure semantics. Every existing enum value that has an
  expression equivalent must produce the same result under that expression for
  every upstream-state vector: same pass or fail, same downstream TaskInstance
  state, and the same handling of mapped upstreams and REMOVED  task instances.
  This is the AIP's primary correctness gate.
• Backward compatible. All 13 existing enum values keep working. String literals like trigger_rule="all_success" keep working. No DB migration, no Dag rewrites required.
• Scheduler and Dag version skew. A Dag authored with the new expression form and then read by a scheduler that predates this AIP must fail cleanly at Dag-load time, not silently fall back to a different rule. The wire format carries an explicit version field so future evolutions stay safe.
• Adoption complexity. Airflow's current trigger rules are simple and predictable; this proposal makes them more powerful, which is inherently less predictable. The mitigation is documentation: enums stay the primary example in tutorials, and the expression form is presented as the tool for cases where no enum fits.

What change do you propose to make?

 Add a structured expression form for trigger rules that lives alongside the existing enum.

from airflow.sdk import TriggerRule as TR

# Today's form, still valid
task_a = EmptyOperator(task_id="a", trigger_rule=TR.ALL_DONE_MIN_ONE_SUCCESS)

# New equivalent
task_b = EmptyOperator(
    task_id="b",
    trigger_rule=TR.expr(done="all", success=">=1", skipped=0), # equivalent to ALL_DONE_MIN_ONE_SUCCESS
)

# Combinations that previously required a new enum value become one-liners
task_c = EmptyOperator(
    task_id="c",
    trigger_rule=TR.expr(done="all", success=">=35"),   # ingestion fan-out, 35 of 40 partitions
)

task_d = EmptyOperator(
    task_id="d",
    trigger_rule=TR.expr(failed=0, upstream_failed=0, skipped="<=1"),
)

The expression accepts keyword thresholds against the upstream-state counts
Airflow already computes (success, failed ,  skipped , upstream_failed , removed , and done ).
Values accept "all" , "none" , "any"  or ">=1" , plain integers, or
comparison strings (">=N", "<=N" , ==N" , "<N" , ">N"). Conditions are
implicitly ANDed.

The meaning of "all"  and related shorthands is defined against the same
effective upstream set used by the current trigger-rule evaluator, including its
existing handling of mapped upstreams and REMOVED  task instances. In
particular, expression evaluation must preserve the current per-rule treatment
of REMOVED  upstreams rather than imposing one global interpretation across all
rules.

All convertible enum values are mapped into expression form in Appendix A below
(some cannot be converted, which is written explicitly).

Trigger-rule expressions apply to mapped and dynamically-expanded upstreams with
no additional syntax.

Grammar

Condition slots.

• Public condition slots in v1 are success , skipped , failed , upstream_failed , removed , and done .
• success , skipped , upstream_failed , removed, and done refer to the effective upstream counts used by the current trigger-rule evaluator.
• failed  is a derived counter equal to FAILED + UPSTREAM_FAILED . It is not the raw FAILED  count.
• upstream_failed  remains available as the raw UPSTREAM_FAILED  count for authors who need to distinguish cascaded failure explicitly.
• Setup-scoped counters are not part of v1 expression syntax. ALL_DONE_SETUP_SUCCESS remains enum-only.

Semantic conventions.

• Evaluation policy via the parameter evaluation_mode : "eager" | "complete" = "eager":
  •  "eager" fires as soon as a matching upstream exists (the semantics of the current ONE_SUCCESS and ONE_FAILED rules)
  •  "complete" waits for every relevant upstream to finish.

Failure-state routing

When an expression fails, the downstream TaskInstance state is determined by the
same rule-specific routing used by the equivalent preset trigger rule. This is
required so that enum-form and expression-form remain behaviorally identical not
only in pass/fail outcome, but also in the downstream state assigned on
failure.

For expression forms that do not correspond to an existing preset, the
implementation must still follow the evaluator's existing routing model rather
than a single generic "any failure => UPSTREAM_FAILED , else any skip =>
SKIPPED " rule. In particular, mixed upstream outcomes that currently resolve
to SKIPPED  for some rules must continue to do so when authored in expression
form.

The exact routing is therefore part of expression evaluation semantics and is
covered by the equivalence tests described below.

Out of scope

The following are intentionally excluded from this AIP. They are viable extensions that could be proposed as separate AIPs or follow-up PRs once the expression form is in users' hands and real feedback is available:

• Per-upstream named conditions. Expressing "task A must succeed, task B just needs to be done" on individual upstream edges (for example, a >> Require("success") >> downstream). This introduces composition semantics with the task-level rule that deserve their own discussion.
• Boolean operators. OR-combined conditions (for example, "at least one success OR all skipped") and conditional or implication logic (for example, "if A succeeded, then B must too"). The aggregate form here is AND-only by design.
• Fractional thresholds. Specifying targets as ratios rather than counts (for example, "at least 50% succeeded"). Useful but a larger design surface.
• Arbitrary Python predicates. Letting authors plug in a callable to decide whether to proceed. Power-user territory with significant operational implications.
• Evaluator consolidation refactor. Collapsing the existing if-elif chain in the trigger-rule evaluator into a single dispatch over preset expressions is an internal simplification this AIP unlocks, but it is not a user-visible change and can be proposed as a follow-up PR.
• ALL_DONE_SETUP_SUCCESS, ONE_DONE, and ALWAYS remain enum-only. Their semantics are not expressible as AND-only count conditions (see Appendix A).

What problem does it solve?

Authors today express dependency conditions by picking from a hard-coded list of 13 presets. When the one they need is not there, the options are:

1. Propose and land a new enum value, which can take months between idea and general availability.
2. Shoehorn the logic with a branching operator, adding a node to the Dag whose only purpose is to cover a trigger-rule limitation.
3. Accept a weaker approximation and handle the remainder in-task, breaking Airflow's state-transition model.

All three carry real cost: slower iteration, more brittle Dags, harder post-mortems when a pipeline misbehaves because "the branch operator was doing something clever". The expression form replaces the preset-hunting step with a direct statement of what the author actually wants, in the Dag file, with no release cycle involved.

There is a secondary internal benefit. The evaluator today is a large if-elif chain, one branch per enum value. Once the expression form is in place, each enum value's branch can collapse to "look up the preset expression and evaluate it". That is a smaller surface area to maintain and a safer place to add future extensions.

Why is it needed?

The problem is not a specific missing preset. It is the trajectory. Every team that needs a combination not currently in the enum repeats the same loop: file an issue, write a PR, wait for the release, then adopt the rule. The set of useful combinations is large enough (and grows with the ecosystem) that chasing it through enum additions is structurally the wrong approach.

Concretely, requirements along the lines of the following have come up over the past two years, and each would have needed its own enum value under the current model:

• "All done, at least two succeeded."
• "None failed, at most one skipped."
• "All done, exactly one failed."
• "At least N of M succeeded" for varying N and M.

Each is one line of expression and zero enum churn under this proposal.

Are there any downsides to this change?

• More surface area to document - The expression form has its own small grammar. The docs need a clear mapping table and a "when to use an enum vs an expression" note.
• Authors may overuse it - A rule like TR.expr(success="all") is just ALL_SUCCESS  written the long way. Docs should steer toward using the enum when it already fits.
• Readability drift in code review - A reviewer looking at trigger_rule=TR.expr(failed=0, upstream_failed=0, skipped=0, done="all") has to mentally evaluate what it means, versus NONE_FAILED . Good-faith use of the enum where it fits mitigates this.
• Serialization-format evolution - Adding a structured field to the serialized Dag means mixed-version clusters (new scheduler with old webserver, or vice versa) need a clear failure mode. Addressed via a mandatory version field and a hard parse error on unknown formats, but the contract has to be honored going forward.

Which users are affected by the change?

• Dag authors - gain a new authoring surface. Existing Dags are unaffected; nothing changes unless a Dag opts in by using TR.expr(...) .
• Operational users - rules should be reflected in the UI

How are users affected by the change? (e.g. DB upgrade required?)

• No DB migration - Trigger rules live in serialized Dag JSON, which is
  regenerated every parse cycle. No metadata DB schema change or upgrade script is
  required.

  Serialized Dag and API contract updates required - The change does require
  versioned updates to the serialized Dag wire format and to the Task SDK /
  API-facing trigger-rule representation, because trigger rules are currently
  modeled as string enum values rather than a structured object. Mixed-version
  components must fail clearly on unknown structured trigger-rule formats rather
  than silently coercing them.

What is the level of migration effort (manual and automated) needed for the users to adapt to the breaking changes? (especially in context of Airflow 3)

No migration is required - This is a strictly additive change; no existing behavior is altered, deprecated, or removed. Dag authors do not need to rewrite anything.

If an author chooses to adopt the expression form for an existing Dag, the change is a one-line substitution per task. The mapping table above functions as a drop-in translation. An automated rewrite (via a ruff-style codemod or a small script) would be straightforward, but is not required: the enum form remains fully supported and is often more readable when it fits.
There are no breaking changes in this AIP. The existing enum values are not being deprecated or removed. Whether to deprecate any of them (for example, ALL_DONE_MIN_ONE_SUCCESS  once its expression form is canonical) is explicitly a decision for a future AIP, not this one.

Other considerations?

• Interaction with setup and teardown - Setup and teardown tasks have hard-coded rule behaviors ( ALL_DONE_SETUP_SUCCESS  on teardowns, implicit ALL_SUCCESS  on work tasks downstream of setups). This AIP does not touch those semantics. The expression form is a drop-in replacement for ordinary trigger rules and does not unlock or alter setup or teardown behavior.

What defines this AIP as "done"?

• • The expression form is available in the Task SDK, documented with the mapping
  table, and has at least one example Dag in example_dags/ .

  • Parametrized equivalence tests: for each convertible enum in Appendix A,
  running the existing state-vector fixtures against both the enum and the
  Appendix A expression yields identical results. “Identical” includes pass/fail
  outcome, downstream TaskInstance state, and mapped-task behavior with REMOVED 
  upstreams. Tests explicitly exclude ALL_DONE_SETUP_SUCCESS , ONE_DONE  and
  ALWAYS  with a comment pointing to the Out-of-Scope rationale.

  • Serialized Dags round-trip cleanly: a Dag authored with TR.expr(...) 
  serializes, deserializes, and evaluates to the same behavior on subsequent
  scheduler cycles.

  • The wire format carries an explicit version field, and deserializers raise a
  clear Dag-load error when encountering an unknown format, verified by test.

  • The Task SDK, serialized-Dag schema, and any API/UI datamodels that surface
  trigger_rule  are updated consistently so the expression form is accepted and
  rendered without relying on a closed enum-only contract.

  • The UI renders the structured form in the task details panel.

  • A short “when to use which form” note is added to the trigger rules
  documentation.

Appendix A: Enum → Expression mapping

Enum-only exceptions.

• ALL_DONE_SETUP_SUCCESS: shape-dependent (requires ≥1 setup success if any setup upstream exists, otherwise falls through to ALL_DONE). Not a pure count condition.
• ONE_DONE: requires success>=1 OR failed>=1 with eager evaluation. The AND-only expression grammar cannot express this without a derived counter; keeping it as a named preset is cleaner than introducing a single-use slot.
• ALWAYS: bypasses the trigger-rule dependency entirely. No count form is meaningful.