Page History

...

Aside: I think making dag partitioning an explicit thing in airflow would be an overall good thing, beyond the whole uniqueness issue.  It clarifies what exactly is going on, i.e. why execution X does not run until the next partition – cus it would not be complete otherwise.  It also makes it possible for the user to say, every day rerun the last N partitions.  This is essentially a partition-driven workflow with a configurable lookback.

Notes on conversion to and from partition-driven

One thing to consider is, what happens when user changes the partition scheme or schedule.  People will obviously do this, either intentionally or on accident. 

One option is to only track partition fulfillment when the dag is in partition mode.  Then, if dag is later changed to non-partition mode, then we would just leave the old partition records there, and ignore them.  Any runs created while dag not in partition mode would not update the partition records.  If the dag is switched back to partition-driven, then the partition store would not have knowledge of runs that happened when the dag ran in non-partition mode.  If user wants to migrate the history they would have to do so manually or via the api.

Another option would be that when we convert modes, we could try to infer the current partition state.

Another option would be to disallow conversion.

4. Leave it sorta fuzzy and ambiguous

...