Proposed Agenda

Attendees

  • Vikram Koka

  • Jens Scheffler

  • Jarek Potiuk

  • Ash Berlin-Taylor

  • Rajesh Bishundeo

  • Ambika Garg

  • Jed Cunningham

  • Kenten Danas

  • Kalyan R

  • Amogh Rajesh Desai

  • Rahul Vats

  • Tamara Fingerlin

  • Shahar Epstein

  • Zach Gottesman

  • Pavan Kumar

  • Bugra Ozturk

  • Eugene Kostieiev

  • Ephraim Anierobi

  • Elad Kalif

  • Kaxil Naik

  • Daniel

  • Abhishek Bhakat

  • Vincent Beck

  • Aritra Basu

  • Kathikeyan Singaravelan

  • Brent Bovenzi

  • Ian Buss

  • Pierre Jeambrun

  • Dennis Ferruzzi

  • Shubham Raj

  • Vishnu Chilukoori

  • Cedrik Neumann

Summary

  • Development Updates and Presentations
    • AIP-81 CLI Authentication Design Update Buğra Öztürk Unknown User (jedcunningham) 
      • Bugra and Jed explained how AIP-81 would create a completely separate remote CLI package, which will be "airflow-ctl".
      • For local execution, all of the CLI commands (which are DB backed) will just work. 
      • All of the merged code for remote CLI execution will be moved into airflow-ctl as part of beta4 and will also simplify packaging and release timelines. 
    • AIP-72 Task Execution Interface aka Task SDK Update Unknown User (ash) Unknown User (kaxilnaik) Unknown User (amoghdesai) 
      • Kaxil gave an update on the AIP-72 detailing that JWT Integration, Custom XCom backend, and Secrets backend work is all completed. 
      • He added that the Triggerer backend now works without going through the public API, to avoid all the authentication passing complexity.
      • Kaxil said that the Task Notifiers is the one open item, which TP is currently working on to finish.
      • Amogh added that the Secrets backend also enables a separate secrets backend for Workers as compared to the Secrets backend configured for the Scheduler. 
      • Ash added that each Task now has a JWT token that is associated with it, bound to a Task Try ID. 
      • Ash also added that the Execution API versioning was now merged for the Task Execution Interface and implemented Calendar versioning, based on Cadwyn.
      • Ash added that this was only for the Task Execution Interface, not for the public API, nor for UI internal API. 
      • Shahar added that he had heard that some AWS provider system tests were breaking right and that he would be investigating. 
    • Test plan update Rahul Vats
      • Rahul shared the updated Airflow 3 Test plan and walked through a quick progress update by AIP. 
      • Rahul asked everyone on the team to put on their testing hat, treat beta4 as being the internal testing vehicle, and to test it out, so that we can find more bugs before release.
      • Rahul also added that AIP-81 needed to be tested for CLI regressions. 
  • Discussion Topics:
    • Packaging Update Jarek 
      • Jarek highlighted the big PR which introduces the "airflow-core" package within the main airflow repo.
      • He highlighted how there is no longer an "airflow" folder in the main repo.
      • Instead, there is "airflow-core" which has all the key items inside it including the pyproject home.
      • The second thing we have is an airflow meta package which has all the configuration for builds and additional elements for packaging and distribution. 
      • Jarek said that this was still work in progress, with a few things to be fixed, but was mostly done.
      • Jarek also spoke about how UV works with the new packaging structure and stressed how this simplifies Airflow development.
    • Cleanup of Airflow 3 open items in Github Unknown User (eladkal) 
      • Elad asked all the AIP owners to review the list of Github issues originally targeted for Airflow 3 and still open
      • Specifically, these are airflow3.0:candidate and airflow3.0:breaking
      • Elad made the point that these should either be closed as having been done or moved to a subsequent release tag as part of the Airflow 3 RCs. 
  • Action items
    • What's critical to finish up
      • Jens brought this up in the context of the UI, if there are open critical items to be completed before 3.0 release
      • Brent took the action item to highlight this async and shout out if anything needed help
    • Change dev calls to be weekly until release
      • Elad suggested that we make the dev calls weekly, since we are very close to release.
      • Vikram to update the calendar invite to be weekly and to send out a reminder if a meeting is required that week. 
  • No labels