AIP-79 Replace Flask App Builder

Status

Motivation

Flask App Builder, or FAB, has been a useful tool in developing the auth, webserver, and UI of Airflow 1 and 2, but it has started to become more burdensome than helpful. 

The FAB templates were useful to build out the first UI application, but have gotten in the way of implementing AIP-38 UI Modernization and developing new features and user experiences. Our React pages are each independent apps that are mounted inside of a FAB template html page. As of Airflow 2.9, we have to mount three separate React apps which have to be remounted on every page load because FAB controls the overall webpages. This hurts performance and prevents us from easily handling state across views. Supporting legacy FAB pages and their custom javascript also affects developer experience by forcing us to keep a complex webpack configuration for two very different javascript patterns. This hurts our ability to keep a modern standards like robust type-checking, linting, auto-formatting and hot-reloads.

Our webserver is also a mess. Almost all of it is powered by a single views.py file which is nearly 6,000 lines long. This file is also a weird hybrid of FAB models and pages but also haphazardly built custom endpoints for the UI to access. This is hard to maintain best practices for whether that is write tests, handle errors, add types or even how the code should be written. While the FAB parts are removed, this entire file needs to be refactored to improve code quality and maintainability.

We rely on FAB as our base auth manager, but as of AIP-56 Extensible User Management auth no longer has to be part of core Airflow. That AIP also set the groundwork to swap the FAB auth manager to a KeyCloak auth manager.

The maintainers of FAB call it a "simple and rapid application development framework". Put most simply, our needs have grown beyond it's scope. Therefore, when Airflow 3.0 is released we should remove and replace Flask App Builder.

Proposal

We should take the advantage of the ability to make breaking changes in Airflow 3 and remove FAB.

Full React App

The UI becomes a single independent client-side rendered React application. It will mount once and be fast and easy to navigate around between pages. The React app can handle all of its own asynchronous requests and router management. The UI project can have a modern build process with all the industry-standard developer guide rails. We can remove outdated dependencies like momentjs and bootstrap. This is necessary to truly mark AIP-38 as done.

UI REST API

We will refactor our webserver into a Flask REST API with its own OpenAPI spec and generated Swagger docs. This should look very similar to our public REST API with generated types, standard error handling, permission checks, etc. The only difference will be that we don't need to expose this internal API publicly or maintain backward compatibility. The UI should even generate types automatically based on the OpenAPI spec. Full queries and mutations too can be auto-generated anytime the API is changed. A developer can then immediately consume the API in React instead of editing about 4+ files of boilerplate.

New Auth Manager

The default FAB Auth Manager could be replaced with our own simple auth manager for out-of-the-box development or with a KeyCloak Auth Manager like discussed in AIP-56. KeyCloak brings much more functionality than FAB too Perhaps the FAB Auth Manager can be kept as a provider plugin. That still needs to be explored.

Many plugins, especially UI-facing ones, use FAB to generate views or create custom models. Those will have to be rewritten to interface with a modern React UI and a new webserver REST API. AIP-68 New Plugin Interface goes into more detail on how this can be implemented and provide new ways for plugins to connect into the Airflow UI.