...

A more limited version of this one, nullable logical date, is that we don't actually add timetables that don't use logical date, but we we allow users to trigger runs with no logical date.  These would be unaffected by the uniqueness constraint on logical date.

Technical note

Null values are allowed with uniqueness constraint.  So the unique constraint on logical date in dag run is not a problem here.problem here.

Other note

I think with option 2, it might make sense to introduce another date field, run_date  which would be populated for all runs.  It would be the date after which the dag run should be free to be scheduled.  This word is generic enough to work for both schedule-driven dags and event-driven dags.  It would always mean the earliest time that the dag should run.  For manually triggered e.g. it would be "right now".  When the scheduler is not overloaded, it should always be close to start_time . 

3. Add a partition concept, or a partition-esque concept

...

• there's no logical date uniqueness constraint on dag_run
• users could end up with multiple runs for a given logical date if user triggers manually, via api, or through trigger dag run operator
• in part because of this, backfill currently will always add new dag runs and never reuse existing dag runs
• as a result, use of backfill will always result in multiple runs per logical date
• when there are multiple runs for a given logical date, they will all be shown on the UI.  e.g. a red run might falsely indicate partition non-fulfillment
• there is nothing to prevent concurrent runs of a given logical date.  e.g. if there's a backfill running, and then a user clears an old failed dag run in the same range, or if to users or API calls trigger a dag run at around the same time, nothing will prevent that.
• what backfill does now is look at the state of the latest dag run (by start date) to ascertain state of the partition.  and for any date for which there's currently a running dag run, backfill will not create a dag run for that date.

5. Provide configurability at backfill run time or more "depends on past" options

...

Here are my concerns with it:

• these options would not maintain these aspects of old behavior
  • concurrent runs of a one logical date a.k.a. partition are disallowed
  • grid view is a partition-fulfillment view
• "choose when doing a backfill"
  • what you're talking about i think is, let user choose, at the time of backfill, whether to rerun all runs or, say, only the latest.  but "the latest" is a fraught concept as a partition inference strategy.  because (1) there could be a tie and (2) even if there isn't a tie, if say user created 3 runs at same time, it would be subject to randomness which one is the lucky one that we use. 

Proposal

I would be in favor of 2 or 3.

...