Versions Compared

Old Version 5

changes.mady.by.user Unknown User (dimberman)

Saved on

compared with

New Version 6

changes.mady.by.user Unknown User (foxish)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Changed one statement about preserving state in K8s.

...

If an airflow worker fails it might be useful to keep the kubernetes worker reserved and preserved in it's same state for debugging purposes

  • Kubernetes is primarily meant to run stateless applications. To our current knowledge there is no way to preserve state for a kubernetes jobpods can retain state and logs, and we can use etcd to preserve some additional state as well (through a CRD in future).  

Other interesting points:
The Airflow Kubernetes executor should try to respect the resources that are set in tasks for
  scheduling when hitting the kubernetes API

...