DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
| Page properties | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
Video of current problem: https://www.youtube.com/watch?v=sNrBruPS3r4
...
Webserver Gunicorn processes read state from the DB instead of each webserver process managing its own state by reading the DAGs itself. This currently results in the weird disappearing and reappearing behaviour for several minutes after you make a change (e.g. add a new DAG). The DB will be the single source of truth.
Once the scheduler processed the DAGit DAG it should be visible for all Gunicorn workers. The webserver will not scan the DAG files anymore, only use it for specific actions:
Trigger; This requires the DAG to be processed once. This does not need to live in memory, and can be done on the fly.
Code; Just to view the code, execution of the DAG is not required here.
...