Page History

...

•  Executor storage + DAG transportation
  •  Executor should be able to transport large-scale DAG folders to slaves with minimal start-up 
    •  Git mode for development (init container on each worker uses git-sync pulls to pull dags from a git repo)
    •  Volume claim mode for production (use any `ReadOnlyMany` persistent volume claim to store and pull dags)
•  Airflow configuration support
  •  Changes made to airflow configuration on the scheduler should be reflected in every worker pod that is spun up. 
    •  There are a few ways to support this, see the "Up for debate" section
•  Task level configuration: 
  •  Should allow customizations of the kubernetes pods on a per-task basis. The minimum proposed options are:
    •  Resource usage customization
    •  Docker image customization
    •  Kubernetes secret customization + keychain (what does keychain refer to here)
•  Crash safety of scheduler
  •  The scheduler must tolerate unexpected crashes and start back up in a healthy state when restarting
    •  Use watcher to re-hydrate state from pods using labels instead of keeping data structures in memory
    •  Persist the watch resourceVersion and recover from crashes by starting the watch from the last applied resourceVersion 
    •  Clear all queued tasks on restart of the scheduler
•  Kubernetes health checker
  •  While the executor watches the pods that are spun up on kubernetes, we need to ensure the pods don't get "stuck" forever (for example, because a kubernetes secret you expected to exist does not exist)
    •   There are a few ways to support this, see the "Up for debate" section
•  CI/testing environments
  •  minikube cluster with `driver=None` mode on travis
  •  separate travis build executing kubernetes integration tests on minikube

...