Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Date: June 12th 2024

Authors: Elad Kalif, Shahar Epstein

...

  1. Operational User - Interacts with Airflow’s UI on a DAG/DAG Run specific basis. Normally triggers a run manually via pre-defined form in Airflow's UI.
    This actor may have no technical knowledge and even not be from the data domain realm (for example, product managers).


  2. DAGAuthor - Creates, modifies, or removes DAGs from the system. The may interact with various Airflow components (Connections, Variables, Xcoms, etc..). Within this type, we could recognize two subtypes:
    1. Technical Dag Author - interacts with .py files directly.
    2. Non-technicalDAG Author - interacts with an interface on top of .py files (for example, building DAGs via .yaml files).

  3. Deployment Manager - owns the deployment. They have the ability to deploy plugins, install Python packages/providers, and handle the scale of workers, webserver, and schedulers.
    They also decides on which executor to use, configuring the Airflow’s settings, and handling zombie or undead tasks. This actor may also enforce Cluster Policy, and possibly owns the CI/CD process of DAGs’s deployment.


  4. Infrastructure Manager - Owns the serving of the compute/storage/systems to be served for the deployment of Airflow. For example, owns the Kubernetes cluster, DBs available in the company, etc.
    This actor is sometimes external to the Airflow app and isn’t familiar with Airflow.



Actor

Required Skills

Required permissions

Responsibility

Notes

Operational user

Non-technical

UI

run/view existing DAGs


Dag Author

Python Knowledge

UI, API, DAG folder

Deploy DAGs


Deployment Manager

Python, Infrastructure

All components of Airflow

Health and stability of the application


Infrastructure Manager

Provides DB/K8s cluster to run the Airflow application


Sometimes, they’re not even aware of Airflow.

Airflow is just one of the many applications that run on the infrastructure.



It’s important to note that with multi-tenancy, there might be additional sub-actors.

...