Proposed Agenda

Attendees

  • Vikram Koka

  • Ash Berlin-Taylor

  • Jens Scheffler

  • Rajesh Bishundeo

  • Eugene Kostielev

  • Kalyan Reddy Kasireddy

  • Pavan Kumar

  • Houda Cherkaoul

  • Vincent Beck

  • Ismael Cherkaoul

  • Ephraim Anierobi

  • Rahul Vats

  • Bugra Ozturk

  • Arunav Gupta

  • Pierre Jeambrun

  • Kenten Danas

  • Cedrik Neumann

  • Brent Bovenzi

  • Jed Cunningham

  • Shahar Epstein

  • Jarek Potiuk

  • Shubham Raj

  • Karthikeyan Singaravelan

  • Daniel Standish

  • Tamara Fingerlin

  • Abhishek Bhakat

  • Daniel Imberman

  • Aritra Basu

  • Daniel Hegberg

  • Vishnu Chilukoori

  • Shubham Mehta

Summary

  • Follow-up on action items from the last call:
    • Airflow 3.0 minimum version to be Python 3.10 Unknown User (ash) 
      • Moving to Python 3.10 is not a blocker.
      • Ash said that he didn't have time to make this change in time for the 3.0 release. 
      • Therefore, decided to bump to after the 3.0 release and do this as part of the standard Python support cadence.
  • Development Updates and Presentations:
    • Update on AIP-72 Task Execution Interface aka Task SDK 
      • Short circuit and Branch operators Unknown User (shahar) 
      • Shahar walked through the changes required for these operators.
      • Specifically, the change from direct DB access to using the TaskSDK for updating XComs and the state of the task. 
    • Test plan update Rahul Vats
      • Rahul walked through testing status with Beta1, including testing migrations (upgrades and downgrades). 
      • Rahul said that confidence level with respect to the various AIPs was increasing and that he expected the Grid of confidence by AIP to get a lot more colorful over the next week.  
      • He said that since the "DB isolation will be enforced starting Beta2", he expected more test DAGs to start failing. 
      • When asked about UI test automation, he said that the UI testing was manual and could not be automated yet because it was changing every day. 
      • The team was very appreciative of Rahul's update.
    • Milestone update Unknown User (vikramkoka) 
      • Feature completeness and road to RCs
      • Vikram shared the updated Airflow 3 dev milestones and said that he wanted the beta 4 release targeted for the week of March 17th, to be feature complete and the final beta release. 
      • Vikram asked for all open PRs to be completed, tested, and merged before beta 4. He urged contributors to ask for help if needed to meet this milestone.
      • Vikram asked for patience for the community over the next few weeks, since open issues could not be allowed to linger and would be reassigned if the assignee was unresponsive.
      • There was general consensus from the team. 
      • Vikram also brought up AIP-68 Extended Plugin interface for React Views, saying that this would need to be deferred to 3.1 because of timing. There was general agreement on this as well. 
    • DevEx improvements as result of provider's distribution separation and devel-common Unknown User (potiuk)
      • Jarek shared a quick update on the recent DevEx improvements for Airflow development based on UV.
      • He shared how this would make Provider development a lot easier than previously the case as well as benefiting TaskSDK development going forward.
  • Discussion topics:
    • Packaging update - open items (Unknown User (ash) , Unknown User (jedcunningham) Unknown User (potiuk) )
      • A quick update shared was about making the Airflow repo a meta package, with Core Airflow becoming a project within the Airflow repo. 
    • Edge provider naming issue (and PEP 752 as long term solution) Unknown User (potiuk) 
      • Jarek brought up the issue that this name is already taken on the PyPI registry and that this is related to the PEP 752 discussion. 
      • When this PEP is implemented, ASF will take the "Apache" name, but this is still in progress with the POC to be implemented this week. 
      • The immediate action is to hopefully get this name (Edge) from the PyPI registry. 
    • FAB next steps - Unknown User (potiuk) 
      • Jarek confirmed that he is working on enabling Airflow to use FAB 5.0, since this is holding us back from upgrading SqlAlchemy. 
      • Jarek outlined multiple options: 
        • Option 1: Wait for 5.0, but that is 2 to 3 months
        • Option 2: Take the needed parts of FAB and incorporate into the FAB provider; Then upgrade to SqlAlchemy 2.
        • Option 3: Stay with SqlAlchemy 1 for now, but the security issues with it are problematic
      • The team agreed that option 1 was a non-starter, and that we should on option 2, but be prepared to switch to option 3, if we run into issues with the SqlAlchemy upgrade being so close to release. 
  • No labels