Proposed Agenda

• Catch-up on action items from last call
  • TBD
• Swim lane updates:
  • UI Test Framework (Rahul Vats)
  • UI / API swim lane update (Pierre / Brent)
  • Task SDK update (Amogh)
• Airflow 3.2 development updates:
  • Asset Partitions (Wei Lee / TP):
  • Deadline Alerts (Dennis Ferruzzi):
  • Multi-team (Niko / Vincent):
  • Python Async Operator docs (David Blain):
• Discussion topics:
  
  • Helm chart release - Bugra
  • Security / Isolation level for 3.2 - Jarek
  • TBD

Attendees

• Vikram Koka

• Shahar Epstein

• Natanel

• Elad Kalif

• Brent Bovenzi

• Jens Scheffler

• Pratiksha Badheka

• Ash Berlin-Taylor

• Vincent Beck

• Dennis Ferruzzi

• Eugene Kostieiev

• Tamara Fingerlin

• Pavan Kumar (gopidesupavan)

• Dheeraj Turaga

• Amogh Rajesh Desai

• Niko Oliveira

• Rahul Vats

• Phani Kumar

• Jarek Potiuk

• Kaxil Naik

• Bugra Ozturk

• Jed Cunningham

• Karthikeyan Singaravelan

• Rajeshwar Bishundeo

• Christos Bisias

• Zach Gottesman

Summary

• Swim lane updates:
  • UI Test framework (Rahul Vats):
    • Rahul shared that the UI test framework has reached a significant milestone, now at 121 test cases after removing pagination and sorting cases from the meta issue, which has now been closed with all scenarios marked as complete.
    • Current work is focused on addressing test flakiness and reducing execution time by increasing the number of workers from the current default. 
    • Rahul plans to create a new meta issue next week for contributions to unit tests as a follow-up to this work.
    • On the performance front, Rahul and Pierre have created a meta issue for large DAG and task performance improvements, with 5 issues already closed and around 4 still open. The target is to close the remaining items before the 3.2 release. 
  • AIP-72 Task SDK update (Amogh)
    • Amogh shared that the AIP-72 project board in Github is officially closed today, having completed over 300 issues and PRs, which was significantly larger in scope than initially anticipated. Phase 2 planning for expanding to DAG processing and Triggerer are underway with initial thinking of focusing on the 3.3 timeframe. 
    • Analysis of the remaining imports shows minimal dependencies, with the main blocker being that Dag.test still requires Airflow Core, and some dependencies based on the Deadline Alerts executor work. 
  • UI / API swim lane update (Pierre / Brent)
    • Brent shared that UI plugins dependency work has been completed and that the Data Partitions UI work is in progress for 3.2. 
    • Graph view performance optimization for large Dags is in flight and is a stretch goal for the 3.2 release.
• Airflow 3.2 development updates:
  • AIP-76 Asset Partitions (Wei Lee / TP):
    • Wei shared a recorded video of the Asset Partitions work which demonstrated completed work for Date based partitioned DAGs. 
    • The outstanding element of work seemed to be documentation.
  • Deadline Alerts (Dennis Ferruzzi): 
    • Dennis noted that a PR has been pending review for almost a month. Amogh provided detailed feedback requiring significant changes, and while Jens had approved the PR, additional reviews were still needed. The team agreed that Dennis could proceed with follow-up PRs for incremental improvements in parallel.
  • Multi-team (Niko / Vincent):
    
    • Niko shared that the multi-team work had entered UAT phase with ongoing, active community testing. All features scoped for 3.2 were implemented and marked as "experimental" as previously discussed. With respect to executor support, Niko shared that Celery was completed, Kubernetes was in progress, and Edge discussions were ongoing. 
    • Niko also clarified that multi-team configuration required deployment manager involvement and was not intended to be dynamically configurable.
  • Release Management (Rahul / Kaxil):
    • Rahul shared that the 3.2 timeline was on track, with the beta branch having been cut. 
    • Vikram shared that approximately 1,500 meaningful commits were included between 3.1 and 3.2, excluding CI and dev tooling, which made this significantly larger than a normal, minor release. 
• Discussion topics:
  
  • Helm chart release (Bugra)
    • Bugra presented the current state of the Helm chart, with version 1.19 recently released and proposed moving to a major version i.e. 2.0 to drop deprecated features. 
    • Jed recommended at least one more 1.x release with proper deprecation warnings in place before making such a break. Jens supported a 1.20 release followed by a 2.0 clean up. 
    • The decision was to have the core Helm chart contributors schedule a dedicated call to align on the release strategy. Bugra took this action item.
  • Security / Isolation level for 3.2 (Jarek)
    • Jarek raised the current gap in security isolation, specifically that there is no isolation between tasks running on the same machine. 
    • Ash that said he had a PR implementing a feature for worker process memory protection, with additional work needed for DAG processor and triggerer isolation. 
    • After significant discussion, the team decided to defer the follow-on work to 3.3 given the complexity and the required integration with the DAG processor and Triggerer changes already targeted for that release.