Proposed Agenda

  • Catch-up on action items from last call
    • None
  • Development Updates and Presentations (Please keep the updates to less than 10 minutes):
    • Airflow 3.1 beta release update (Kaxil)
    • Airflow 3.1 test plan update (Rahul)
    • Language translations update (Shahar)
  • Discussion topics:

Attendees

  • Vikram Koka

  • Shahar Epstein

  • Elad Kalif

  • Dennis Ferruzzi

  • Brent Bovenzi

  • Phani Kumar

  • TP Chung

  • Jed Cunningham

  • Ephraim Aneirobi

  • Christos Bislas

  • Pierre Jeambrun

  • Eugene Kostielev

  • Vincent Beck

  • Amogh Rajesh Desai

  • Rahul Vats

  • Bugra Ozturk

  • Karthikeyan Singaravelan

  • Niko

  • Rajeshwar Bishundeo

  • Kenten Danas

  • Kaxil Naik

  • Cedrik Neumann

  • Wei Lee

Summary

  • Development Updates and Presentations:
    • Airflow 3.1 test plan update (Rahul)
      • Rahul shared that the overall testing status was positive across most components. 
      • He shared that the Task SDK regression testing had passed without import issues, the UI filters implementation were tested for audit log and XComm filters, and that the "Human in the loop" feature testing was on track. 
      • Rahul also said that AIP-81 testing ran into authentication issues with token/username-password. Bugra shared that the issue is being investigated and that the current suspicion is that it is environment related. 
      • Rahul was concerned about "UI Plugins" requiring retesting after recent changes by Brent and Pierre. 
      • Rahul also asked Dennis to update the testing status for "Deadline Alerts".
    • Airflow 3.1 beta release update (Kaxil)
      • Kaxil shared that the Airflow 3.1 beta version released earlier in the week did include the two major PRs highlighted in the last dev call i.e. TP's serialized DAG implementation changes and Kaxil's changes for defaults. 
      • As a result of the above, the server-side code now only uses serialized DAGs and Task Groups are decoupled between server and SDK. 
      • Also, migration testing from 2.11 to 3.1 beta has successfully been tested. 
    • Deadline Alerts (Dennis)
      • Dennis shared that async support was being added to Notifiers.
      • He also said that UI aspects were deferred to post 3.1 and that synchronous callbacks were in development, targeted for Airflow 3.2
    • Language translations update (Shahar)
      • Shahar shared that the language translations are at 97% coverage overall and we have been able to add new translations including Catalan and Simplified Chinese. Portuguese and Italian are still in PR, but may still be able to make it before release
      • And that the the translation freeze has been extended until the 3.1 release to get to 100% coverage
      • Amazing work here, I am astonished that we are now up to 14 language translations! 
  • Discussion topics:
    • UI plugins recommendation (Pierre)
      • Pierre raised a discussion topic around marking React app plugins as experimental. 
      • Pierre said that the edge provider plugin work by Jens revealed needs for optimization and would probably require changes in build tools and dependency sharing. This would probably only affect very complex UI plugins, but it would be safer at this time to mark this part of the UI plugins work as "experimental". 
      • Based on this recommendation by Pierre and Brent, the team agreed with this recommendation. 
    • Airflow CTL update (Bugra)
      • Bugra said that the first beta was released for "airflow ctl" and that testing was ongoing with limited community interaction to date. 
      • Bugra said that a follow-on release candidate is planned on this, rather than a second beta. 
      • Following on, Bugra created a new AIP to decouple remote commands from the Airflow core CLI, which included a transition approach. 
      • Vikram, Jarek, and Kaxil raised concerns about existing automation dependencies based on the preexisting airflow CLI commands. 
      • Vikram recommended that Bugra delay asking for a vote on this AIP until after the 3.1 release, so that the team has the cognitive capacity to process the changes. 
  • Release preparation and related action items:
    • Kaxil asked for the AIP (feature) owners to review the Airflow 3.1 draft release notes. Specifically including:
      • Human in the loop (Wei, et al)
      • Deadline alerts documentation (Dennis / Rajesh):
        • Mention of the async vs. sync differences and what's coming in 3.1 vs. later
        • Migration guide additions to explain the transition from SLA callbacks (in 2.x) to Deadline Alerts in 3.1 
      • UI React plugins documentation (PIerre / Brent):
        • Including what is experimental vs. not
    • Rahul asked Bugra to get back on the "airflow ctl" bugfix and testing
    • Rahul asked Dennis to update the Deadline Alerts testing
    • Vikram asked the team to review and cleanup the priority blocker issues (issues with label priority:upgrade_to_airflow3) and the 3.1 blocking issues
    • Elad asked Dennis to confirm what was needed for Notifier provider release in conjunction with the 3.1 release