Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

To me this is a very interesting approach .  We could largely that on one hand, let's us keep the dag run semantics unchanged.  But allow users who want to trigger runs manually with no logical date to optionally set it null thereby avoiding any uniqueness constraint.But there are multiple paths here.  The simplest one is, keep everything the same but allow manual triggering of runs with null logical date and then these runs have no data interval.  Another path is to add timetables for which logical_date is always null and there are never data intervals

But it also makes it possible to just fully dispense with logical dates and data intervals for those dags which don't use them.

With this approach we sort of have our cake and eat it too.

2.a. timetables that don't use logical date

If you don't care about logical date or data intervals, then maybe indicate that with the timetable you use:

...

Code Block
languagepy
themeFadeToGrey
DAG(dag_id="blah", schedule="0 0 * * *", logical_dates=False)

2.b. don't add timetables with no logical date, but let user manually trigger with no such date

A more limited version of this one, nullable logical date, is that we don't actually add timetables that don't use logical date, but we 

3. Add a partition concept, or a partition-esque concept

...