|
Providing the same functionality from the API interface is available via airflowctl. This makes most of the airflow CLI commands redundant, causing duplicate efforts as well as exposing the database directly to the user, making the system insecure, and allowing interaction with database entries without any Role-Based Access Control (RBAC).
Actions were previously defined in AIP-81: Enhanced Security in CLI via Integration of API.
This proposal reduces maintenance effort by reallocating development focus from the airflow CLI to airflowctl for Remote Commands. This will result in faster delivery of features, reduce code duplication, and strengthen the Airflow Public (Core) API as the single source of truth. Providing functionalities from a secure channel using authentication methods and RBAC.
At a high level:
# Summary Migrate airflow <command> to airflowctl <command> while ensuring feature parity. Checklist ## 1. API Availability Confirm equivalent functionality is available in the Airflow Public API and airflowctl If Yes: proceed with airflowctl implementation and CLI deprecation warning If No: * Implement/extend the necessary API endpoint(s) * Implement airflowctl operation/command ## 2. CLI Deprecation * Add DeprecationWarning in airflow CLI for <command> * Update CLI help text to reference airflowctl alternative * Add migration note in documentation (CLI → airflowctl mapping) ## 3. Documentation * Update CLI reference docs (airflow <command>) to show deprecation * Update airflowctl documentation to include the new command * Add entry to migration guide (airflow CLI ↔ airflowctl) ## 4. Tests * Confirm parity tests between airflow CLI and airflowctl * Add regression tests for API + airflowctl and airflow CLI **Notes** **Reference:** AIP-94 Decouple Remote Commands from airflow CLI This PR should not remove CLI command immediately — only add deprecation warnings. Removal will happen after at least one major release cycle with warnings. |
We propose to limit the development of the airflow CLI for Remote commands and instead migrate these commands to a new dedicated tool. airflowctl, which interfaces exclusively with the Airflow Public (Core) API.
All Remote commands in the airflow CLI will be deprecated and emit warnings directing users to airflowctl.
Equivalent functionality will be implemented in airflowctl to ensure no loss of capability.
New Remote features will only be added to airflowctl (and the API), not to the airflow CLI.
Local administrative commands (e.g., database shell, process management) will continue to exist in the airflow CLI.
Duplicate development and maintenance: Currently, Remote functionality exists both in the airflow CLI and through the API, creating redundant implementations.
Security risk: airflow CLI can directly manipulate the database, bypassing RBAC and audit trails provided by the API.
Inconsistent user experience: Some commands differ between the airflow CLI and API-backed tools, leading to confusion.
Slower feature delivery: Contributors must implement features in multiple places (CLI + API), increasing development overhead.
To enforce the API as the single source of truth for all Remote commands.
To strengthen security by ensuring all Remote operations go through RBAC and audit mechanisms in the API.
To simplify maintenance by removing duplicate code paths.
To enable faster community contributions by focusing on one implementation (airflowctl + API).
To future-proof Airflow: airflowctl can evolve independently with richer functionality, without being constrained by legacy CLI structure.
Migration effort: Users and automation relying on airflow CLI Remote commands will need to update scripts and tooling to use airflowctl.
Learning curve: Users familiar with the airflow CLI must adapt to a new tool airflowctl.
Transition complexity: Ensuring full parity between airflow CLI and airflowctl during deprecation will require careful tracking (hence the PR template and mapping table).
Short-term overhead: Contributors must first ensure API parity before the CLI deprecation warning, which may initially slow down some changes for a small portion.
End Users: Anyone using the airflow CLI for Remote commands (e.g., airflow dags list -o table to airflowctl dag list -o table) will need to migrate to airflowctl.
Platform Administrators: Will benefit from a clearer separation — airflow CLI for local administrative commands, airflowctl for API-driven (Remote) commands.
Contributors/Developers: Will focus efforts on one Remote command implementation, reducing maintenance burden.
Security-Sensitive Deployments: Will gain from stronger enforcement of RBAC and prevention of direct DB access.
No database upgrade is required: This AIP only impacts the CLI tooling layer. The Airflow metadatabase schema is unaffected.
No scheduler/webserver/executor upgrade impact beyond normal release upgrades: Users can upgrade Airflow as usual; the only difference is CLI behaviour for Remote commands.
CLI usage changes:
Remote commands (airflow dags …, airflow connections …, airflow pools …, etc.) will emit deprecation warnings and direct users to use airflowctl instead.
Local administrative commands (e.g., airflow db shell, airflow info) remain available in the airflow CLI without changes.
Script & automation updates required: Any custom tooling, automation, or CI/CD scripts that call Remote airflow CLI commands will need to migrate to airflowctl.
Migration window: Users will have at least three minor (example: from 3. x.x to 3.x+3.x ) release cycles with warnings before Remote CLI commands are removed entirely.
Security improvements: Remote operations will go through the API layer, enforcing RBAC and audit logging — this may change access patterns in organisations that currently rely on direct DB access through airflow CLI.x
3. x.x to 3.x+3.x ) Airflow release has gone through the deprecation warning cycle before full removal.