Airflow Dev Call

The dev calls are open for anyone in the community to join and help in the Airflow 3 efforts. 

Schedule: Fortnightly (once in 2 weeks) on Thursday at 4pm GMT/UTC

Table of Contents

• 15 January 2026
  • Proposed Agenda
  • Attendees
  • Summary
• 29 January 2026
  • Proposed Agenda
  • Attendees
  • Summary
• 12 February 2026
  • Proposed Agenda
  • Attendees
  • Summary
• 26 February 2026
  • Proposed Agenda
  • Attendees
  • Summary
• 12 March 2026
  • Proposed Agenda
  • Attendees
  • Summary
• 26 March 2026
  • Proposed Agenda
  • Attendees
  • Summary
• 9 April 2026
  • Proposed Agenda
  • Attendees
  • Summary
• 23 April 2026
  • Proposed Agenda
  • Attendees
  • Summary
• 7 May 2026

15 January 2026

Proposed Agenda

Attendees

Summary

• Development and swim lane updates
  • Macro update on swim lane numbers and focus areas (Vikram)
    • Vikram walked through the updated swim lane numbers as posted on the Airflow 3.x wiki page. 
    • Vikram shared that with agreement from Jens and Jed, he had combined the Kubernetes provider area with the Helm chart swim lane, since the expertise was common to resolve issues and approve PRs across both those areas. 
    • Vikram also thanked all the tech leads for their focus on the swim lanes in the early part of the year, saying that he had a significant drop in the "open work items" in the first two weeks of the new year.
    • Vikram said that the "Logging" swim lane needed a volunteer to help lead efforts there, since it was limited by Ash's bandwidth at this time. 
    • Feedback from the team was to:
      • Jarek asked to exclude draft PRs from all areas
      • Pierre said that the API issues were really two-fold: UI and Execution. Pierre proposed to break those apart for easier tracking by marking execution API issues as "Task SDK" and the team immediately agreed.
      • Jens had asked about looking at the details with each issue or PR. Vikram responded that it was possible, but he had run into GitHub rate limits and had therefore stayed at looking at the issue / PR labels. 
      • Action item: Vikram to add the DAG (summarization script) to the repo for collaborative improvements on the scripts, especially for those areas which had multiple labels. 
  • UI / API update (Brent / Pierre)
    • The UI / API area continues to be the largest backlog area with significant volume. Pierre and Brent both reported significant progress since the last dev call in closing issues and merging PRs
    • They also shared that the Grid view performance was significantly improved for large DAGs (tested with 1000+ tasks). 
    • However, the Graph view still has issues with very large DAGs and more work is needed here. 
    • API Server is now stable, but UX challenges remain for very large DAGs and DAG runs. 
  • UI test framework update (Rahul Vats)
    • Rahul shared that we are about 30-35% coverage with over 40 test cases completed and 9 PRs merged. There are still 27 open issues still to be worked on.
    • A big focus is on pagination testing using the limit/offset approach for UI. 
    • Also, he was spending a significant amount of time on manual validation of the tests locally before merging, by intentionally introducing bugs in the local brach to ensure that the tests are catching those bugs. 
    • Action item: Rahul to continue end-to-end UI testing also with a focus on large DAG performance scenarios
• Airflow 3.2 scope discussion
  • AIP-76 Asset Partitions updated scope (Daniel Standish)
    • Daniel walked through a presentation detailing the updated work on Asset Partitions highlighting the focus on date based partitions. 
    • Daniel walked through the key components including: Partition-aware asset events, partition-driven DAG scheduling, and enabling DAGs to listen/respond to specific partition updates. 
    • There was a significant discussion around the similarity / differences between date-based partitions and logical date based data-intervals. 
    • Daniel emphasized the flexibility in keeping the above concepts separate going forward. Daniel shared the current status as having completed the listener functionality, and now working on the scheduler component. 
  • AIP-67 Multi-team (experimental) updated scope (Niko / Vincent)
    • Niko shared a quick update about the multi-team work saying that the Core scheduling is working with multiple executors, of which the Local executor work is complete and the Celery Executor work is in progress. 
    • This is expected to be in 3.2 as "experimental", without having any security isolation guarantees. 
    • The Security isolation work will be continuing with an expected delivery in Airflow 3.3. Vincent said that David Blain offered to help with the Keycloak auth manager support for multi-team.
    • Action item: Niko to share a detailed scope of 3.2 vs. 3.3 via slack and follow up in the next dev call.
  • AIP-86 Deadline Alerts Sync callbacks (Dennis / Ramit)
    • Dennis shared that time-based deadlines with all callback would be ready for Airflow 3.2
    • Dennis also said that a large DB migration PR is pending review after the serialized DAG refactor work
    • Dennis also said that the asset-based deadline trigger work would be deferred to Airflow 3.3
    • Action item: David to send deadline alerts completion / scope status via Slack.

29 January 2026

Proposed Agenda

Attendees

Summary

• Swim lane updates:
  • UI Test framework (Rahul Vats):
    • Rahul shared that we now have 65 test cases with good input from the community. 
    • He also shared that as a result CI now takes 15 minutes for these tests and optimization is needed, with one approach to move some of these into unit tests. Rahul said he is working with Brent on this. 
    • Rahul also mentioned that this has already been valuable in catching a bug in main before the 3.1.7 release. 
    • Still working on an important issue with Grid performance with large DAGs. Kaxil highlighted a great PR from Christos which fixed a performance issue i.e. #54103
  • UI / API  (Pierre Jeambrun): 
    • Pierre shared an overall update including several feature changes that will show up in Airflow 3.2 such as permissions changes for API-level access controls, UI customization changes, and React compiler optimizations. 
    • Kartik had added a significant change in PR #60274 which resulted in noticeable improvements in API server performance, by eliminating duplicate fetching.
• Airflow 3.2 development updates:
  • Task SDK Update (Amogh)
    • Amogh said that the Task SDK work is nearing completion excluding small core imports and that he had to make changes to the Deadline Alerts for SDK separation compliance which needs review from the AWS team. 
    • He also added that he was working on a declarative connections loading scheme via YAML spec to decouple providers.
  • Deadline Alerts (Dennis Ferruzzi):
    • Dennis shared that sync callback support PR is ready for base / local executors. Niko has reviewed this and will have changes. 
    • Dennis said that the behavior of the sync callbacks will be different from the old SLA callbacks because these will be run by the workers rather than by the DAG file processor. 
    • Kaxil said this could have concurrency implications and asked for the documentation changes detailing this behavior.
  • Multi-team (Niko / Vincent):
    • Niko said the executor updates were in flight with the AWS Batch executor work completed and others assigned to community members. Niko said that the celery executor work is complex and still in progress. 
    • The second stream of work was on the UI, with the pool selector UI updates by Vincent nearing completion. 
    • Niko said that the dag_id uniqueness work was difficult and probably would not be in the 3.2 release. 
  • Asset Partitions (Daniel Standish):
    • Daniel shared that the scheduler changes for DAG scheduling based on partitions was in flight, with several PRs having been added by Wei. 
    • The next big item was backfill support for asset partitions. 
    • Daniel also shared that the UI work was being planned with Brent.
  • AIP-98 Async support for Python Operator (David Blain):
    • David shared that this had already been merged and would be part of Airflow 3.2. 
    • There was an open item for documentation around this, specifically regarding the use cases supported. 
    • David shared that this would eventually lead to the Iterable Operator work in AIP-98.
• Discussion topics:
  
  • AIP-95 Providers lifecycle governance (Vikram)
    • Vikram shared that the first PR for the updated governance process was merged and that the new process enables the sponsoring of additional providers. 
    • Initial candidates identified include Vespa and Informatica, which are both in flight.
  • AIP-94 CLI Decoupling (Bugra)
    • Bugra noted that the voting period for this AIP was currently active through the week and invited feedback. 
    • Bugra elaborated that the current plan was to have Airflow CLI commands proxy to Airflow CTL for remote operations, without any user-facing changes. 
    • Local admin commands such as DB clean would remain unchanged. 
    • Kaxil suggested an alternative i.e. single Airflow CTL set of commands for all operations. 

12 February 2026

Proposed Agenda

Attendees

Summary

• Swim lane updates:
  • UI Test framework (Rahul Vats):
    • Rahul shared that we now have good coverage on the E2E UI tests with more tests having been added over the last two weeks bringing the total now to 83. 
    • He said that some of the end to end tests have been moved to being unit tests, so that the CI times could be reduced, and that this work was ongoing, since the CI was now around 14-15 minutes despite the move because of the additional tests.
  • UI / API update (Pierre):
    • Pierre confirmed that API issues were only for those supporting the UIs. The API issues supporting task execution were now flagged under TaskSDK. 
    • Pierre shared that the team was leveraging the test deployment created by Rahul containing millions of task instance records for performance issue identification, and that there was great community participation in this process, including endpoint improvements with caching.
• Airflow 3.2 development updates:
  • AIP-76 Asset Partitions (Wei Lee):
    •  Wei Lee shared a recorded demo showcasing the progress made to date on Asset Partitions, which showed support for date based partitions, leveraging timetables. 
    • The concepts of flexible date based partitions such as Hourly, Daily, Weekly, etc., were clearly demonstrated in the demo and the overall demo was very well received by the team. 
    • There were some questions around the partition keys and the visibility of those partition keys in the UI, which the team agreed to take offline with Wei. Vikram also requested community feedback on potential issues with mismatched partition keys and conditions for subsequent triggering. 
  • AIP-86 Deadline alerts (Dennis):
    • Dennis said that the last two PRs were ready for review, with synchronous callbacks working in the local executor and were abstracted in the base executor. 
    • Dennis shared that Celery implementation was nearly complete and that would be asking for community help on the other executors.
    • Dennis said that they were well positioned to hit the code freeze target date of the week of Feb 26th. 
    • Vikram to follow up with Dennis async regarding the configuration tradeoffs around sync callback execution, concurrency controls, and timeliness of alerts.
  • AIP-67 Multi-team (Niko / Vincent):
    • Vincent said that they were working on adding minimal multi-team functionality to simple auth manager for testing. 
    • Rajesh said that Niko had asked for community help on the Kubernetes executor for multi-team, but that there wasn't much progress here yet.
  • AIP-98 Async Python Operator (David Blain):
    • Vikram enquired if there was any progress on the documentation around the Async Python Operator, specifically the usage guidance as compared to Deferrable operators. 
    • There didn't seem to be, so Vikram to follow-up async with David on this. 
• Discussion topics:
  • AIP-99 Common data access patterns (Pavan)
    • Pavan presented a comprehensive overview of the work planned for this AIP, specifically including: SQL Query generation using DB schemas, Human-in-the-loop review before execution, Data transfer operators via DataFusion, and based on support for all existing Airflow database hooks
    • Pavan also showed a quick demo which covered: Automatic schema fetching from Postgres, SQL generation with validation, and XCom integration for query results. 
    • Pavan said that the implementation approach would be to start with basic SQL operators and would then expand to multiple databases.
    • The overview and demo was very well received by the team. 
    • Vikram asked for broader interfaces to be defined first, before going broad with database support and Pavan agreed with that guidance.
  • AIP-100 Task Priorities (Natanel and Theo S):
    • Natanel shared the analysis and draft design approaches written up by the two of them (Theo could not make the call) regarding priority-based scheduling within Airflow. 
    • Natanel said that the current priority-based scheduling causes starvation at significant scale, when concurrency limits are hit with worker saturation. Based on their research, the proposed solutions built on top of a combination of priority + aging. 
    • Jens shared his feedback (through Vikram) that he very much appreciated the analysis, but was unconvinced about any of the currently proposed solutions. 
    • Vikram also commended the team on their research into the problem, and added that he was surprised that task priorities still existed in Airflow, saying that he had though we had deprecated them a long time ago (since Airflow 2)
    • The general consensus was that the topic needed greater in-depth offline thought before proceeding to a conclusion towards an algorithm and a migration strategy.

26 February 2026

Proposed Agenda

Attendees

Summary

• Swim lane updates:
  • UI Test framework (Rahul Vats):
    • Rahul shared that the UI test framework has reached a significant milestone, now at 121 test cases after removing pagination and sorting cases from the meta issue, which has now been closed with all scenarios marked as complete.
    • Current work is focused on addressing test flakiness and reducing execution time by increasing the number of workers from the current default. 
    • Rahul plans to create a new meta issue next week for contributions to unit tests as a follow-up to this work.
    • On the performance front, Rahul and Pierre have created a meta issue for large DAG and task performance improvements, with 5 issues already closed and around 4 still open. The target is to close the remaining items before the 3.2 release. 
  • AIP-72 Task SDK update (Amogh)
    • Amogh shared that the AIP-72 project board in Github is officially closed today, having completed over 300 issues and PRs, which was significantly larger in scope than initially anticipated. Phase 2 planning for expanding to DAG processing and Triggerer are underway with initial thinking of focusing on the 3.3 timeframe. 
    • Analysis of the remaining imports shows minimal dependencies, with the main blocker being that Dag.test still requires Airflow Core, and some dependencies based on the Deadline Alerts executor work. 
  • UI / API swim lane update (Pierre / Brent)
    • Brent shared that UI plugins dependency work has been completed and that the Data Partitions UI work is in progress for 3.2. 
    • Graph view performance optimization for large Dags is in flight and is a stretch goal for the 3.2 release.
• Airflow 3.2 development updates:
  • AIP-76 Asset Partitions (Wei Lee / TP):
    • Wei shared a recorded video of the Asset Partitions work which demonstrated completed work for Date based partitioned DAGs. 
    • The outstanding element of work seemed to be documentation.
  • Deadline Alerts (Dennis Ferruzzi): 
    • Dennis noted that a PR has been pending review for almost a month. Amogh provided detailed feedback requiring significant changes, and while Jens had approved the PR, additional reviews were still needed. The team agreed that Dennis could proceed with follow-up PRs for incremental improvements in parallel.
  • Multi-team (Niko / Vincent):
    
    • Niko shared that the multi-team work had entered UAT phase with ongoing, active community testing. All features scoped for 3.2 were implemented and marked as "experimental" as previously discussed. With respect to executor support, Niko shared that Celery was completed, Kubernetes was in progress, and Edge discussions were ongoing. 
    • Niko also clarified that multi-team configuration required deployment manager involvement and was not intended to be dynamically configurable.
  • Release Management (Rahul / Kaxil):
    • Rahul shared that the 3.2 timeline was on track, with the beta branch having been cut. 
    • Vikram shared that approximately 1,500 meaningful commits were included between 3.1 and 3.2, excluding CI and dev tooling, which made this significantly larger than a normal, minor release. 
• Discussion topics:
  
  • Helm chart release (Bugra)
    • Bugra presented the current state of the Helm chart, with version 1.19 recently released and proposed moving to a major version i.e. 2.0 to drop deprecated features. 
    • Jed recommended at least one more 1.x release with proper deprecation warnings in place before making such a break. Jens supported a 1.20 release followed by a 2.0 clean up. 
    • The decision was to have the core Helm chart contributors schedule a dedicated call to align on the release strategy. Bugra took this action item.
  • Security / Isolation level for 3.2 (Jarek)
    • Jarek raised the current gap in security isolation, specifically that there is no isolation between tasks running on the same machine. 
    • Ash that said he had a PR implementing a feature for worker process memory protection, with additional work needed for DAG processor and triggerer isolation. 
    • After significant discussion, the team decided to defer the follow-on work to 3.3 given the complexity and the required integration with the DAG processor and Triggerer changes already targeted for that release.

12 March 2026

Proposed Agenda

Attendees

Summary

• Catch-up on action items from last call
  • Helm chart discussions update (Bugra Ozturk)
    • Bugra shared that Jarek, Jed, Jens, and others had met to discuss the Helm chart roadmap.
    • The high-level agreement was to continue with minor releases until all deprecations are complete and breaking changes are fully documented, before proceeding to a 2.0 release.
    • A follow-up meeting is planned for next week, with notes and action items to be shared on the dev list, including versioning details and security measures.
    • A concrete plan is expected in about three weeks, with the overall project timeline measured in months rather than weeks
• Airflow 3.2 Development updates:
  • Testing / Release Manager Update (Rahul Vats)
    • Rahul gave a 3.2 test plan update. Beta 1 was cut last week and testing is actively underway.
    • Regression testing is looking good overall, including the Task SDK changes.
    • There are close to 20 migrations in 3.2, making this both important and risky — a few edge cases have already been identified and issues raised.
    • One notable concern is that the Deadline Alerts migration tasks appear likely to break with large datasets due to complex migration logic.
    • There are also some migration challenges with SQLite currently being worked through. Open beta issues are being tracked here.
    • The current plan is Beta 2 for Monday and RC1 for the week of March 20th. Rahul asked for community help with testing
  • UI / API swim lane update (Pierre Jeambrun / Brent Bovenzi)
    • Pierre reported great activity from the community with lots of fixes merged.
    • Progress is being made on the optimization front and things are on a good track, with more work still to do.
    • Asset Partitions UI work is also making good progress.
  • Asset Partitions (Wei Lee / Tzu-ping Chung)
    • The only remaining open items are documentation and a couple of issues to fix. The work is otherwise nearly complete.
  • Deadline Alerts (Dennis Ferruzzi)
    • Dennis has a PR open for review and asked Ash and Amogh specifically for a look.
  • Multi-team (Niko Oliveira / Vincent Beck)
    • The team is in a bug-finding and bug-fixing phase and things are looking good for RC.
  • Python Async Operator docs (David Blain)
    • Documentation is in flight.
  • OTel update (Daniel Standish)
    
    • Daniel flagged that there are OTel interface changes in 3.2.
    • The existing OTel packages were never formally documented nor intended as stable public interfaces. Christos chimed in to support this.
    • A lazy consensus email has been sent by Daniel to the dev list to deprecate the existing interfaces from 2.10 onwards
• Discussion topic: Feedback from Airflow 2.11→3.1.7 Migration at Bosch (Marco Küttelwesch / Unknown User (jscheffl))

  • Jens and Marco gave a detailed and very valuable presentation on the challenges their team at Bosch faced upgrading a large, complex Airflow environment.

  • The team expressed gratitude for this real-world feedback

  • Issues they saw prior migration

    • Previous integration we had with direct DB access - workaround is to have a parallel DB connection until better solution(s)

      • Context does not provide status of other tasks in the Dag for Error handling/summary

      • Triggering and monitoring other dags not possible w/o DB connection (needed for scaling)

      • Setting of Dag and Task Notes not possible  --> Contribution to Task SDK was not (yet) accepted

      • Automation which adjust Pools and Queues for dedicated machine routing w/o DB access

      • Custom Dag level Archiving w/o DB Access

    • Suprises by breaking APIs that required some migration helpers:

      • XCom for SkipMixIn different semantic and key name changed in BranchOperator

      • XCom does not accept pathlib.Path objects anymore (==PosixPath())

      • Some other dataclass serialization problem in XCom

    • Python code in VenvOperator does not see code from Dag folder anymore --> Fix by adding PYTHONPATH manually

    • Triggerer did not see custom code from deferred sensor anymore, needed to tweak via adding to PYTHONPATH (Kafka Message Filter)

    • Airflow Python API breaking changes, needed to create a tool that can support AF2+3 backends to allow changing backend w/o integrated app change in parallel

    • Issue with XCom retrieval of Mapped task with mapping==1, not returning a list (Fix PR is open in parallel)

    • Our patched and beloved "custom footer" which showed Git sync status was not possible to be ported, alternative still in work via Plugin

    • Markdown in Trigger form UI was interpreted differently compared to AF2, UI glitches needed correction

    • Needed to change a lot of Dags interpreting "execution date" -> "logical date"

    • Trigger form failed with None as default in an enum with optional field (fix in AF3.1.8 coming)

    • Fab provider Oauth integration issues, mixing user contexts (Fixed in current provider)

    • Cookie of session across application, mixing different user contexts on multi site deployments (Fixed in 3.1.8)

    • Needed to re-write a MS EventGrid plugin receiving push events (Flask->FastAPI App)

    • Task log URL generation needed to change

    • Link to listing of "All failed tasks of all runs of a Dag with a Run-ID prefix" not existing anymore

  • Issues faced during or post migration

    • Dag processor failed because of "unstable rendered Dags" --> Version increase --> Full table scan in TI --> Flaky Dags --> Fixed by DB index on dag_version_id (Upstream contribution pending)

    • Scheduler were not healthy because of unstable Dag (after fixing Dag parser) required consistency reconciliation when running in large queue and rotating with every Dag parse, all Schedulers locked DB for re-conciliation and workers and API server were locked out of access, heartbeat failed (Upstream contribution pending)

      • Hard to see which background task and including which Dag caused the problem until debug code injected, no metric on scheduler details!

    • We see some Postgres DB Locks on task_instances (~5min,  root unclear until now) - tasks fail as heartbeat every 5s, retry after 5s, API server running out of DB connections, kills Pods in liveness as DB connection pool exhausted by DDOS from workers

    • JWT token timeout of 10min too short if tasks are staying in Celery queue, way too short, such failures produce empty logs, hard to find

    • Previously limited Dags via "max_active_tasks" were running almost uncontrolled, lot of complaints by affected groups suffering from capacity

    • Happily running tasks were killed by re-assignment of Tasks in Celery after 1h (w/o any config change in Celery!), need to disable LATE_ACK

    • Bad Unicode chars killed jobs on DockerOperator --> Fix contributed

    • Bad Unicode chars kill KPO jobs on triggerer --> PR in review in parallel (was closed because of AI Slop 2 times…)

    • Celery worker needing 50% more RAM for same concurrency (8GB->12GB for 16 tasks), generated a couple of OOMs

    • Locally patch to count deferred task instances into running (PR in Airflow open in discussion in parallel)

    • Redirect after login broken if user not authenticated (Fix in 3.1.8)

    • (new) Validation that run id prefix matches run-type

    •  We still see DB Locks lasting for 5+ minutes in production and seek for root cause

  • Open items we see as gaps post migration

    • UI: Ability to batch clear/mark as failed Dag runs (Target: Workaround via script) → #63854 + #63855

    • UI: Presentation of Task Notes is hard, proposal was rejected... planning to pitch an AIP soon as a new feature for "Dag/Task Summaries for Humans"

    • UI: Scrolling in large logs is as bad as in Github - initial loading faster but scrolling through in total is worse than in AF2. Also as partially loaded no browser search possible --> Search option in panel?

    • UI (requested but coming in 3.2.0): : Grid filter left panel by run status (running, failed, success at least)

    • UI (requested but coming in 3.2.0): Better filtering in Dag runs / Task Instances

    • UI (requested but coming in 3.2.0): Re-run Dag with previous config

    • UI nit: Can not click mapped task that is scheduled as link is missing

    • UI nit bug: Failure to load right panel when clicking on a task group left in grid

    • UI nit: Missing to color code test/development instances - hope this is coming with 3.2.0 theming...

    • UI nit: Admin / Pool view is bad if 20+ Pools listed, bad overview

    • UI nit: complaints that failed and skipped task color tooo similar --> will adjust locally with custom theming in 3.2.0?

    • UI nit: a lot of user bookmarks broken/404 (e.g. http://host/airflow/dags/<my_dag>/grid)

  • Compensated by very positive feedback in regards to

    • Now it is directly possible to see who triggered a Dag

    • URLs to logs finally working (was a pain with Grid in AF2)

    • UI: Much more modern than AF2!

    • UI: Translations are welcome!

    • HITL opens new options for approving runs which has a actual demand ATM

    • All users enjoy dark mode!

26 March 2026

Proposed Agenda

Attendees

Summary

• Note: Vikram was traveling and asked Ash Berlin-Taylor to lead the call on his behalf.

• Catch-up on action items from last call
  • None
• Airflow 3.2 Development Updates
  • Testing / Release Manager Update (Rahul Vats)
    • Rahul gave a 3.2 release and testing update. The current target is to cut RC1 on Monday, March 30th.
    • Beta 2 has already been cut and testing is actively in progress. From this point onwards, Rahul will be cherry-picking bug fixes and other items tagged with the 3.2.0 milestone, rather than fast-forwarding with main. The cherry-pick strategy was shared on the dev list.
    • Migration testing is now in good shape. The blockers raised on the last call around migration performance have been resolved with community help, and the migration scripts have been tested across all three supported databases with no remaining issues.
    • Regression testing for the SDK and other areas is looking clean, and there are currently no critical or high priority issues. UI sanity testing is still in progress with a few medium priority bugs identified, but none are considered RC blockers.
    • Rahul specifically called out AIP-76 Asset Partitions as being in good shape, with solid testing done by Atul. Deadline Alerts owners were asked to update the test plan status, as the current one appeared to be a reuse from the 3.1 async work. Multi-team testing looks mostly complete with one open issue remaining.
    • Ash noted that 3.2 is now effectively in feature freeze — no new features will be added, and anything not already in will need to wait for 3.2.1 or 3.3.
    • Community was asked to test Beta 2 and report any high or critical priority issues on the dev list or Slack, using the beta label for tracking.
  • UI / API swim lane update (Pierre Jeambrun)
    • Pierre reported that the team has been focused on bug fixes and improvements targeted for 3.2.
    • Good progress has been made on the optimization front, with improvements to the dashboard page, DAG run listings, DAG listings, and log streaming. These will ship in 3.2, though more optimization work remains.
  • Deadline Alerts (Dennis Ferruzzi)
    • Dennis shared a detailed update in Slack the night before the call.
    • There is a PR currently open and under active review. Ash, Amogh, and Kaxil had all provided feedback. Dennis is working through Ash's design comments and expects to have a proposal within the hour.
    • All remaining work — including wiring for get connection, get variable, get asset, get XCom — is done locally and ready to be submitted within an hour of the blocking PR being merged.
    • Anish is working on a PR to unify workload handling, which is largely complete but will need a small refactor once the blocking PR lands.
    • Every executor needs to be updated, and volunteers have been assigned for each, with most having draft PRs already open.
    • Given the timing, the team agreed that the connection access piece is likely to land in 3.2.1 rather than 3.2, which can reasonably be treated as a bug fix.
  • Multi-team (Niko Oliveira)
    • UAT was largely completed last week, with just one remaining open issue related to CI.
    • All features scoped for 3.2 are in, and the team is holding off on any new development to avoid unintentional merges into the 3.2 branch.
    • Phase 2 development targeting 3.3 will start soon. Ash noted that by default nothing will get merged into 3.2 without an explicit request, so the team can start development without risk of accidentally landing 3.3 work in 3.2.
  • Python Async Operator docs (David Blain)
    • David was not on the call. Deferred to next call.
• Discussion Topics
  • AIP-99 Progress and Demo (Pavan Kumar)
    • Pavan gave a live demo of AIP-99, the Common Data Access Patterns + AI work, which generated a lot of enthusiasm from the team.
    • For those unfamiliar, AIP-99 uses a Pythonic AI framework under the hood to enable unified, AI-assisted access to different data sources via existing Airflow hooks. The goal is to allow agents to connect to any database or data source Airflow supports, in a consistent way.
    • The demo covered two example DAGs. The first was a data analyst agent using a SQL toolset connected to a Postgres database with Human-in-the-loop (HITL) review enabled. The agent fetched data, responded to natural language queries, and paused for human input before proceeding — all within the task instance UI. The current implementation runs on a worker with XCom as the backend for agent/human communication.
    • The second demo showed a file analysis agent, using the Airflow Object Store interface to read a sample image and return a structured analysis of its key components.
    • Amogh suggested that when HITL review is active, there should be a clearer visual indicator in the UI so users know the task is waiting on their input.
    • Ash and Daniel noted that the experience feels conceptually different from a typical DAG run, though agreed that naming this distinction is hard enough that leaving it as-is for now is reasonable.
    • Pavan noted the work is approaching a 0.1 release, planned for the following week, and that recorded demos will be shared.
    • Shubham expressed excitement about the demo and the potential use cases.
  • Time based, scope-restricted Personal Access Tokens / PATs (Dheeraj Turaga)
    • Dheeraj raised the challenge that users are currently generating JWT tokens and passing them to LLMs to interact with Airflow programmatically, but those tokens are short-lived. The ask is for longer-lived tokens with admin-level controls — including the ability to revoke tokens and to scope them to read-only or limited permissions, to prevent LLMs from taking unintended write actions in production.
    • Ash noted that this is a meaningful ask but complicated by the Auth Manager architecture, where each auth manager has its own mechanisms and token lifecycle handling would likely need to be implemented per auth manager.
    • Karthikeyan highlighted the specific challenge of a user wanting read/write access for UI interactions but read-only access for LLM interactions — something tokens don't currently capture. He also shared a link to an existing token invalidation issue (#47952) and a related Claude Code issue around token refresh.
    • Jarek noted that token revocation has recently been added (PR #61339 merged by Pierre), and that token refresh is already possible while a token is valid. He suggested that an agent could potentially be instructed to refresh its token, though Ash noted this doesn't address the case where a token has fully expired.
    • Bugra suggested separating the concerns of token lifetime extension and token revocation, as they interact with different layers — token lifetime is tied to the auth manager, while revocation may be interceptable at the base auth manager level.
    • The general consensus was that this is a valid and important ask that requires deliberate design, particularly around security and scope controls. Jarek noted this is part of why he had reservations about MCP support — these security design questions need to be thought through carefully.
    • Dheeraj agreed to start an async discussion on the dev list to continue the conversation.
  • DualStatus Manager backwards compatibility (Christos Bisias)
    • This was an unplanned topic raised by Christos. A lazy consensus vote on the dev list to remove the DualStats Manager interface passed last Saturday, and Christos is working on a PR to remove it.
    • The question was whether it could be removed now given that it appeared in a recent provider release.
    • Ash clarified that since provider code referencing the DualStats Manager has already been released on PyPI, the import cannot be removed until those provider versions are no longer supported — roughly when the minimum provider version reaches Airflow 3.3, which is approximately 3 minor versions away.
    • The implementation can be changed (e.g. aliased to the new Stats Manager or made a no-op), but the import path must remain intact to avoid breaking released code.
    • Ash suggested Christos check whether the existing provider usages are wrapped in a try/except ImportError — if so, removal may be possible sooner. If they rely on a version check instead, the import must stay.
    • Daniel suggested that aliasing DualStats Manager to Stats Manager would be the simplest path forward, since the underlying behavior has already been moved.
    • Ash closed by reminding everyone to test Beta 2, with RC1 targeted for Monday. He also noted that 3.3 is planned to be a shorter cycle of roughly 3 months.

9 April 2026

Proposed Agenda

Attendees

Summary

• Vikram opened the retrospective noting that spring break and general post-release fatigue meant a lighter attendance than usual.
• Catch-up on action items from last call
  • None
• Airflow 3.2 Retrospective
  • Release Manager perspective (Rahul Vats)
    • Rahul opened by thanking all contributors who helped with development and testing for 3.2.
    • Rahul proposed introducing a recognition program for community members who file bugs during beta and RC phases, similar in spirit to the existing "PR of the Month," to incentivize earlier and broader community testing.
    • Jarek noted that the model used for provider releases works well: a GitHub issue is created that tags contributors and explicitly asks them to test the specific things they contributed. He suggested adapting this for minor releases, with a more compact version of the issue given the larger scope. He noted that for the most recent provider release, around 80% of tagged contributors confirmed they had tested their changes. Jarek also suggested reaching out directly to users who had filed issues that were fixed in the release, asking them to test the beta or RC on their staging environments — since those users are already motivated to see their fix land.
    • Jens highlighted that the key gap is getting testing done in production-like environments rather than artificial small-scale setups, since many issues only surface at real scale. He suggested building a more comprehensive integration test suite that covers known corner cases and can run longer, more complex scenarios.
    • Rahul agreed and noted that a few issues during RC1 were related to object storage on cloud — scenarios that exist in the E2E test suite but weren't being run against real cloud environments. He suggested reviewing gaps in E2E coverage as a follow-up.
    • Jens also noted that the Easter break timing reduced available testing resources significantly, and suggested that having an additional beta before cutting the RC, combined with targeted contributor outreach, would give the community more time to catch issues earlier.
  • Documentation-first proposal (Elad Kalif)
    • Elad raised two observations. First, that a large number of non-feature commits and bug fixes are bundled into minor releases, adding risk. He suggested considering whether some of those could go into a separate minor release to reduce scope.
    • Second, and more significantly, Elad noted that documentation for major features was arriving very late in the cycle, often close to the beta cut, making it very difficult to test features effectively. He proposed requiring that documentation be written upfront, even before implementation in some cases, so that testers can understand the intended behavior and verify against it.
    • Vikram said he was a strong fan of this idea, noting that with AI tools it is now very easy to generate and maintain documentation concurrently with code, and therefore there is little excuse for late docs. He suggested this should be a prerequisite for a feature being merged or included in the first beta, at least for significant features.
    • Jarek agreed and added that he has used an AI agent to continuously update documentation while developing, which works very well and can even generate screenshots. He noted this represents a shift in development mindset but is very achievable.
    • Rahul noted this would also reduce the last-minute documentation rush before RCs.
    • Action item: Elad to write up the proposal for requiring documentation alongside significant feature development as a prerequisite for beta inclusion, to be shared on the dev list.
  • Security landscape update (Jarek Potiuk)
    • Jarek shared a broader update on the security landscape that had affected the 3.2 release process.
    • A supply chain security incident occurred in recent weeks involving LiteLLM, where maintainers were impersonated. The Google provider had a dependency on LiteLLM and was briefly affected. The vulnerable version was pulled from PyPI within about two hours. 
    • Separately, Jarek shared that Anthropic is investing $1.5M into ASF security initiatives, with $250K from Alpha Omega as a seed, as part of a new ASF Responsible AI initiative announced the day before the call. The goal is to raise $3M this year and $10M overall to fund tooling and access to advanced AI security models. He explained that the emerging security model involves AI that can proactively find and generate patches for security vulnerabilities, turning the security process from reactive to proactive. Bug bounty programs have already been suspended by several organizations as AI can now find vulnerabilities faster than human researchers.
    • ASF will receive early access to Anthropic models currently under embargo. The details of how access will be distributed are still being worked out at the ASF board level, with a decision expected at the board meeting in about a week and a half. A VP of AI Tooling role is expected to be appointed to lead the initiative.
    • Dheeraj asked whether ASF maintainers would receive direct access to Anthropic's preview models. Jarek confirmed that access is being arranged, either directly from Anthropic or through Alpha Omega, though it will be limited to those involved in security work due to embargo constraints.
    • Vikram noted that Airflow's cooldown protection is now in place broadly and asked the team to keep an eye on further developments.
  • UI / API feedback (Brent Bovenzi)
    • Brent noted that the increased PR velocity from the community on UI has been a mixed experience, helpful for getting features like additional filters and search shipped, but requiring more effort to reject low-quality PRs. He and Pierre have had to be stricter in their review standards.
    • Brent also noted that the pace of feature work has sometimes meant less time for thoughtful UX design.
  • Jens's upgrade issue follow-up
    • Vikram asked Jens about the GitHub issues he had planned to file from the Bosch upgrade experience. Jens said he had filed the easy ones and some had already been fixed in parallel, but had not completed the full list due to work pressures.
• Discussion Topics:
  • AIP-103 Task State Management (Vikram)
    • Vikram noted he is looking forward to working with XD and Jake on AIP-103 Task State Management, with a target of the 3.3 release. He thanked everyone who gave feedback and voted on this.
    • Jake is also working on an updated incarnation of the Asset Watermarking AIP, rebasing it on the Task State Management work.
    • Vikram noted he plans to clean up the related Confluence pages, as there are currently around 4 to 5 related AIPs that are not well organized. The goal is to track them as a cohesive epic rather than unrelated items.
  • AIP-72 Multi-language support / Java Task SDK update (Vikram)
    • Vikram noted that TP has been making significant progress on the Java Task SDK, which represents the first delivery of non-Python multi-language support for the Task SDK.
    • It felt too rushed to include in 3.2, but Vikram has scheduled a demo for the next dev call in two weeks. He wanted to make sure the community was aware of this work, as it had not been discussed broadly as a group.
  • AIP-94 CLI Decoupling update (Bugra Ozturk)
    • Bugra shared that now that 3.2 is released, he has started creating tickets for AIP-94 (decoupling the CLI using Airflow CTL), all with the 3.3 milestone. He flagged this for awareness and asked the community to keep an eye out for new issues being created.
    • Vikram thanked Bugra and said he would update the wiki page in the next couple of days to reflect this.

23 April 2026

Proposed Agenda

Attendees

Summary

• Note: The call only got through roughly half the agenda. AIP-99, AIP-103, MCP (AIP-91), and AI Assistant (AIP-101) are deferred to the next dev call
• Catch-up on action items from last call:
  • Airflow 3.3 wiki update (Vikram)
    • Vikram shared that the wiki update is in progress but not yet complete. He walked through the current state of the Airflow 3.3 scope page.
    • Items currently tracked for 3.3 include:
      • Continued Asset Partitions work (segment-based partitions and remaining date-based use cases),
      • AIP-103 Task State Management,
      • Pluggable retry policies (a positive vote is expected),
      • Deadline Alerts and Multi-team follow-on work,
      • AIP-99 agentic work targeting a 1.0 release, J
      • ava and Golang Task SDK work, and
      • AIP-94 CLI decoupling via Airflow CTL.
    • Vikram noted that Jens has a couple of follow-on AIPs from the 3.2 cycle that he hadn't yet captured, and asked Jens to add or tag them. Jens confirmed he still needs to write the AIP, which relates to pooling, deferring, and task-level limits, so no number has been assigned yet.
    • Bugra noted in chat that AIP-94 was missing from the wiki and would add it after the call. He also confirmed in the call that tickets for AIP-94 are now being created with a 3.3 milestone, following the 3.2 release.
    • Vikram said he will update the wiki to reflect AIP-94.
    • Vikram stated the target for 3.3 is to release by mid-June at the latest, ideally early June. He expressed a desire to move toward a roughly every-two-months release cadence going forward.
    • Jens suggested the approach should be timeboxed, with any features not ready being deferred to a later release rather than delaying the whole milestone. Vikram agreed.
    • Rajesh asked about what would be showcased at Airflow Summit in late August. Vikram noted that the agentic AI work and task state management were his personal focus areas from a Summit relevance perspective, and added that a 3.4 release in August before Summit is possible if the cadence holds.
    • Ash endorsed more frequent, smaller releases, noting they are less painful to produce and easier to upgrade to from the user side. He also noted that the client-server separation should help users upgrade more frequently going forward.
    • Ash and Jarek agreed to follow up async on the possibility of completing the airflow to airflow-core rename and related import cleanup for 3.3, with Amogh likely involved. Jarek noted this work is very close to his heart and he would like to dedicate more time to it. Jens added that cleanly separating the API server deployment from the DAG parser, and clarifying the provider split between plugin code and operators, should also be documented, ideally following the documentation-first approach discussed in the last retro. Jarek agreed and noted there is an unpublished internal document from Ash that could serve as a starting point.
  • Pierre shared in chat that there was a successful Airflow community meetup in Paris the evening before, with around 80 attendees, which was notably larger than usual for tech meetups in the city.
• Airflow 3.3 Development Updates:
  • AIP-72 Java Task SDK demo (Jason Liu / TP Chung)
    • TP and Jason were unable to join due to time zone constraints. A pre-recorded demo by Jason Liu was shared by Phani Kumar during the call. The recording is also available here.
    • Ash noted there were a couple of last-minute naming changes in the code that the demo had not yet been re-recorded to reflect, including a change to the stub(language=) parameter name.
    • The demo showcased the Java Task SDK, covering three key interfaces: Task (with an execute method), access to Airflow features including XCom, connections, and variables, and a DAG bundle interface for exposing DAGs to the DAG processor.
    • Two approaches for defining DAG structure were demonstrated. The first uses a Python stub DAG with @task.stub to reference Java tasks, which was the approach supported in the earlier Golang SDK work. The second is a pure Java DAG, where the DAG structure is defined entirely in Java and metadata from the JAR file is used by the DAG processor.
    • A key highlight of the demo was a true multi-language DAG showing Python and Java tasks running within the same DAG, with XCom passing data bidirectionally between the two languages. Task dependencies in the pure Java DAG are defined via a simple addTask API with explicit dependency lists.
    • Error handling was also demonstrated, with Java-side stack traces forwarded through the Task SDK to Airflow logs.
    • The overall reaction from the team was very enthusiastic, with Jarek describing it as "really cool" and Jens noting how impressive the Java support was.
    • Ash explained that architecturally, a Python coordinator process picks up the task and hands it off to Java for execution, handling log capture and secrets mediation so that the secrets backend does not need to be reimplemented in each language.
    • Several open questions were raised and will be shared with Jason and TP for async follow-up, including: how Java DAG bundles handle additional JAR dependencies (fat jar vs. classpath), whether the DAG processor needs the JAR present for pure Java DAGs or whether serialized DAG payloads are used instead, and whether the same approach applies to Go. Ash noted a PR or dev list post from Jason and TP is expected soon. Vikram noted he will make sure the questions are shared with Jason and TP for async response.
• Discussion Topics:
  • Agentic PR Triage skill (Jarek Potiuk)
    • Jarek gave a demo of a skill-based agentic approach he has developed for PR triage, built on Claude Code. The demo experienced some slowness due to GitHub rate limits during the live session, but the overall approach and results were explained.
    • The triage skill is implemented entirely as English-language Markdown files describing classification criteria, conditions, response templates, tone rules, and batching/caching behavior. GraphQL queries are embedded inline to pull PR data. The skill was itself generated by Claude based on an earlier code implementation.
    • The workflow is conversational: a maintainer opens Claude Code in the Airflow repo and issues natural language commands such as "triage oldest PRs" or "triage PRs I am a reviewer of." The agent prefetches batches of PRs in the background while the maintainer reviews the current batch, making it fast enough to process around 300 PRs in a couple of hours.
    • A key design principle is that the agent never sends a comment without explicit maintainer confirmation. Jarek emphasized this as a golden rule.
    • Results from Jarek's first full triage run: out of 200 PRs triaged, 130 were closed (over 60% where contributors did not respond), 70 were merged, 75 received responses, and 149 were marked as ready for review. Around 40% of the remaining 300 contributor PRs are now in draft status awaiting action.
    • Brent raised concerns about the impersonal nature of AI-generated comments sent from a personal account, and the risk of discouraging genuine contributors. He noted that some automated reviews add noise rather than value, particularly when they simply restate pending comments already left by a human reviewer. Aritra suggested that adding an explicit banner or disclosure indicating AI generation would be a good middle ground and would be easy to implement as a single prompt change. Jarek agreed and said he would add this.
    • Jens thanked Jarek for tackling the backlog but raised a specific case where a colleague's PR for a critical production triggerer issue was converted to draft by the triage agent despite having active maintainer attention. Jarek acknowledged this and asked for feedback to be posted in the dedicated Slack channel he created for triage issues, so rules can be refined.
    • Kaxil shared his complementary approach, which uses GitHub Copilot for opt-in PR reviews and a more deterministic skill setup with hooks, bash scripts, and evals to prevent the agent from acting without confirmation. He noted that agents are probabilistic and that hooks are important for enforcing determinism in critical steps. He mentioned a blog post about this approach is expected soon. The key distinction between the two approaches is that Kaxil's is focused on pre-review AI assistance with manual control before posting, while Jarek's is focused on post-submission automated triage with human oversight at confirmation time.
    • Vikram noted that entire companies are being built to solve this problem and encouraged the team to remain open to other open source or commercial alternatives in addition to Jarek's skill.
    • Kaxil shared a link to the existing code review instructions file in the Airflow repo: .github/instructions/code-review.instructions.md.

7 May 2026

Proposed Agenda