Purpose:

To get out a useable KubernetesOperator and KubernetesExecutor we want to define what tasks should be included in the initial PRs of these products and what tasks should be reserved for later PRs

Initial PR:

These are the features that we would consider "crucial infrastructure." While these features will not allow the full flexibility desired by kubernetes proficient users, it will allow small and large-scale airflow instances to run natively on kubernetes.

Kubernetes Operator:

• Create a Pod Operator
  • Pod Operator should be able to "return" a value for XCOM purposes
  •  

Kubernetes Executor:

• Executor storage + DAG transportation
  • Executor should be able to transport large-scale DAG folders to slaves with minimal start-up cost.
  • git-server for development
  • volume plugin for production
• Task level configuration: 
• Using kubernetes secrets + keychain
• Throttling + assurance of cluster health.
• Crash safety for scheduler
  • Use Watcher to re-hydrate state using labels, resourceVersion
• CI/testing environments

Future Tasks:

Kubernetes Operator:

• Create a python custom resource so people can launch python jobs with arbitrary pip installs
  
• Possibly create a Job Operator
  

• • We can create a pending queue that the watcher can remove from when it receives a running event