Airflow 3.0 Development Milestones

The goal of this page is to document all the development milestones for the Airflow 3.0 release. This is intended to be a "living" document and reflect plans and progress during the development and test phase of the project. This is supplemental to the high level timelines in the main Airflow 3.0 page.

This was initially documented in preparation for the dev call on the 10th of October and reviewed as part of that dev call.

Date	Scope
Week of 24 Oct 2024	Completion AIP-74 Introducing Data Assets AIP-79: Remove Flask AppBuilder as Core dependency AIP-65: Improve DAG history in UI Calculate / Store observed DAG version in DB Keep history of serialized DAG and code AIP-66: DAG Bundles & Parsing Initial DAG Bundle interface and git/local merged AIP-72 Task Execution Interface aka Task SDK POC of Task Execution with Core / Task SDK split POC of Remote Python Task Execution POC of non-Python language support (GO) AIP-69 Edge Executor (Initial Name: Remote Executor) (MVP on main / Rework on AIP-72 open) AIP-38 Modern Web Application Updated designs for Dashboard / Home page, DagsList, DAG, DagRun, Task, and TaskInstance pages WIP AIP-75 New Asset-Centric Syntax (Backend changes) AIP-78 Scheduler-managed backfill (Backend changes) AIP-72 Task Execution Interface aka Task SDK Split Core & Task SDK Articulate distribution strategy for Task SDK AIP-68 Extended Plugin Interface for React Views (Working POC) AIP-38 Modern Web Application DagsList, Dashboard (started, but will iterate) AIP-84 Internal API for UI Operations (some endpoints migrated) AIP-83 Remove Execution Date Unique Constraint from DAG Run Unknowns (none) Won't be started AIP-76 Asset Partitions AIP-80 Explicit Template Fields in Operator Arguments AIP-81 Enhanced Security in CLI via Integration of API AIP-82 External event driven scheduling in Airflow
Week of 18 Nov 2024	Completion AIP-78 Scheduler-managed backfill (Backend changes) AIP-65: Improve DAG history in UI (Backend changes complete) API endpoints support retrieving versioned history Finalized detection of observed DAG version AIP-83 Remove Execution Date Unique Constraint from DAG Run AIP-66: DAG Bundles & Parsing Proof of concept of parsing with DAG Bundles AIP-72 Task Execution Interface aka Task SDK Split of Core and Task SDK Initial incarnation of Python execution based on split AIP-38 Modern Web Application DAG, DagRun and Task pages including DAG History (based on AIP-65) (Draft, but not final) UX Design changes for AIP-78 Scheduler-managed backfill WIP AIP-81 Enhanced Security in CLI via Integration of API Dynamic generation of API requests Central API communication mechanism Missing API Endpoints AIP-84 Internal API for UI Operations (most endpoints migrated) AIP-72 Task Execution Interface aka Task SDK Logging support for Tasks XCom and Variable support for Task Execution AIP-69 Edge Executor (Initial Name: Remote Executor) (Migrate to AIP-72) AIP-75 New Asset-Centric Syntax (Backend changes) AIP-68 Extended Plugin Interface for React Views (Working POC with most core functions) Performance benchmark scenarios to be shared by end of Nov (loosely based on AIP-59 Performance tests framework) AIP-81 Enhanced Security in CLI via Integration of API Unknowns ... Won't be started AIP-76 Asset Partitions AIP-80 Explicit Template Fields in Operator Arguments
Week of 16 Dec 2024	Completion AIP-84 Internal API for UI Operations API fully migrated to FastAPI Functional and improved compared to legacy AIP-66: DAG Bundles & Parsing Ability to configure DAG bundles via CLI/API Complete support for DAG bundles in task execution AIP-72 Task Execution Interface aka Task SDK Task Execution in python based on split Core / Task SDK Enabling XComs and Variables for task execution support in API Server AIP-75 New Asset-Centric Syntax (Backend changes) AIP-68 Extended Plugin Interface for React Views (Code Freeze if possible) AIP-38 Modern Web Application DAG dependencies (based on AIP-74 changes) Completed DAG, DagRun and Task pages including DAG History UX Design changes for Assets based on AIP-75 New Asset-Centric Syntax WIP AIP-72 Task Execution Interface aka Task SDK Integration with AIP-66 for DAG fetching AIP-81 Enhanced Security in CLI via Integration of API AIP-82 External event driven scheduling in Airflow Depends on AIP-75 New Asset-Centric Syntax (above) AIP-69 Edge Executor (Initial Name: Remote Executor) (Code Freeze, stabilization on AIP-72) Initial data collected to be done and shared for performance benchmark - by end of Dec (loosely based on AIP-59 Performance tests framework)
Week of 13 Jan 2025	Completion AIP-66: DAG Bundles & Parsing Complete development for DAG bundles and DAG fetching in task parsing AIP-72 Task Execution Interface aka Task SDK Support Dynamic Tasks for task execution support Task Execution support for most use cases in Local Executor AIP-38 Modern Web Application Backfill related changes (based on AIP-78 changes) WIP Packaging changes including Provider repackaging Upgrade checks AIP-66: DAG Bundles & Parsing Add support for task execution based on AIP-72 AIP-38 Modern Web Application Backfill related changes (based on AIP-78 changes) AIP-83 Amendment Backfill related changes (based on AIP-78 changes) AIP-82 External event driven scheduling in Airflow Depends on AIP-75 New Asset-Centric Syntax (above) AIP-81 Enhanced Security in CLI via Integration of API
Week of 27 Jan 2025	Completion: AIP-72 Task Execution Interface aka Task SDK DAG Param support with TaskSDK Celery integration with TaskSDK AIP-66: DAG Bundles & Parsing Complete support for DAG bundles and DAG Fetching in TaskSDK with AIP-72 AIP-38 Modern Web Application Backfill related changes (based on AIP-78 changes) Asset changes finalized (based on AIPs 74 and 75) AIP-82 External event driven scheduling in Airflow Depends on AIP-75 New Asset-Centric Syntax (above) WIP AIP-66: DAG Bundles & Parsing Scaling for DAG bundle parsing TTL for DAG bundles in Workers and DAG Parsers Publish first set of Alpha builds
Week of 03 Feb 2025	Completion: AIP-72 Task Execution Interface aka Task SDK Triggerer / Deferrable Operator integration with TaskSDK AIP-81 Enhanced Security in CLI via Integration of API AIP-66: DAG Bundles & Parsing Publish second set of Alpha builds
Week of 10 Feb 2025	Publish third set of Alpha builds
Week of 24 Feb 2025	Completion: AIP-72 Task Execution Interface aka Task SDK Callback integration Kubernetes Executor integration with TaskSDK Listener integration with TaskSDK AIP-38 Modern Web Application Backfill related changes (based on AIP-78 changes) Asset changes (based on AIPs 74 and 75) First incarnation of Airflow 3 based packaging complete UI still open to changes Publish first set of Beta builds
Week of 31 Mar 2025	Completion: AIP-72 Task Execution Interface aka Task SDK ObjectStorage integration with TaskSDK Notifier Integration AIP-38 Modern Web Application Asset changes (based on AIPs 74 and 75) Upgrade tooling Packaging / Distribution changes final Publish first set of Release Candidate builds

Space shortcuts

Page tree