AIP-79: Remove Flask AppBuilder as Core dependency

Status

This AIP is part of AIP-38, which aims to modernize the Airflow UI.

Motivation

Flask AppBuilder, or FAB, has been a useful tool in developing the auth and UI of Airflow 1 and 2, but it has started to become more burdensome than helpful.

We’ve relied on FAB for authentication (authn) and authorization (authz), but as of AIP-56 Extensible User Management FAB no longer has to be part of core Airflow, as we can rely on other systems for those functions.

However, today FAB does provide the most commonly used authn/authz for Airflow, and there is certainly value in maintaining backwards compatibility for that purpose and for custom plugins. So let’s set the stage for fully removing FAB later, while maintaining a smooth upgrade path for users today.

Goals

• Have a smooth migration path from Airflow 2 to Airflow 3, in particular for authn/authz
• Set ourselves up for a true “FAB-free” future (while also making it possible today)
• If possible, maintain backwards compatibility for plugins

Proposal

The UI components we use from FAB (e.g. CRUD for TIs) will be handled by AIP-38. The rest of this proposal will address the remaining uses.

Authentication / Authorization

The default FAB auth manager and its adjacent entities will be fully moved into the FAB provider. This is mostly done already, but parts of core still expect FAB, so we can do a fresh pass on this (e.g. remove the need for the ApplessAirflowSecurityManager).

This means an Airflow instance could be fully functional without FAB even being installed in the environment, and instead rely on a custom auth manager or other community manager auth manager.

In order to power local environments, which typically do not require authn or authz, core Airflow will contain a minimal auth backend that does not require any authentication and allows the user to perform all actions.

The FAB auth manager can be installed and used, if desired, for more complex environments.

Views from plugins

While AIP-68 will add support for Airflow 3 native plugins, backwards compatibility for plugins using FAB views (`appbuilder_views`) will be maintained, if possible.

Ideally, they continue working as-is. However, we reserve the right to remove support completely if we hit insurmountable hurdles during implementation - this is a best effort goal ultimately.

Menu items from plugins

A new interface for menu items will be added to plugins, as the current interface (`appbuilder_menu_items`) is specific to FAB. The FAB specific menu items interface will be deprecated in favor of a React menu item plugin.

Menu items from FAB (either from views in plugins or default menu items like the security tab) will be available in the Airflow 3 UI. We will likely have an API endpoint to expose the FAB menu items, which will allow the react UI to render them appropriately. This maintains backcompat for the plugins, and also allows the security pages to continue functioning.

How?

If the FAB provider is installed and the FAB auth manager is configured or plugins with FAB views are present, during webserver initialization the provider will create an appbuilder app to be used for that purpose.  We will do our best to avoid extra work if possible, but we will iron out the specifics during implementation. For example, the loading order of plugins and providers may make it difficult to detect if a plugin defines FAB views when the FAB provider is being loaded.

When the FAB auth manager is used, there are some db tables that need to be created or migrated. Instead of being tightly integrated into the sqlalchemy models in core, the FAB provider will take complete ownership of its db tables and maintain its own db cli commands (like migrate/reset/etc).

Why?

You may be wondering why this is worth it - can’t we instead just remove FAB altogether? While we could, today nearly all auth uses FAB. And we do not have a well defined replacement (something like Keycloak) scoped out, even if we do have a good interface to do it technically (auth manager). This means that, practically, to get adoption of Airflow 3 we will need backwards compatibility with the existing auth configuration in Airflow 2. A migration plan for migrating from the FAB auth manager to something else can be developed and made available to the Airflow community at a later date, and we can also end support for the FAB provider once we are comfortable that we have a solid replacement and migration path in place. Compatibility for plugins is just a bonus to make the Airflow 3 migration easier - the authn/authz aspect is the main driver for keeping FAB around.

How are users affected?

• Less complex dependencies - FAB is no longer required for core
• Minimal migration pain for users from Airflow 2 to 3
• Using the FAB provider requires more deployment steps (e.g. separate db commands)

What defines this AIP as done?

• Flask AppBuilder is no longer required to run the webserver and UI
• The existing FAB auth manager is fully functional from the FAB provider, but not required by core Airflow
• If possible, FAB plugin compatibility is maintained