DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
Proposed Agenda
- Catch-up on action items from last call
- None
- Development Updates and Presentations (Please keep the updates to less than 10 minutes):
- Airflow 3.1 beta release update (Kaxil)
- Airflow 3.1 test plan update (Rahul)
- Language translations update (Shahar)
- Discussion topics:
- Airflow 3 adoption blockers (issues with label priority:upgrade_to_airflow3)
- TBD
Attendees
Vikram Koka
Shahar Epstein
Elad Kalif
Dennis Ferruzzi
Brent Bovenzi
Phani Kumar
TP Chung
Jed Cunningham
Ephraim Aneirobi
Christos Bislas
Pierre Jeambrun
Eugene Kostielev
Vincent Beck
Amogh Rajesh Desai
Rahul Vats
Bugra Ozturk
Karthikeyan Singaravelan
Niko
Rajeshwar Bishundeo
Kenten Danas
Kaxil Naik
Cedrik Neumann
Wei Lee
Summary
- Development Updates and Presentations:
- Airflow 3.1 test plan update (Rahul)
- Rahul shared that the overall testing status was positive across most components.
- He shared that the Task SDK regression testing had passed without import issues, the UI filters implementation were tested for audit log and XComm filters, and that the "Human in the loop" feature testing was on track.
- Rahul also said that AIP-81 testing ran into authentication issues with token/username-password. Bugra shared that the issue is being investigated and that the current suspicion is that it is environment related.
- Rahul was concerned about "UI Plugins" requiring retesting after recent changes by Brent and Pierre.
- Rahul also asked Dennis to update the testing status for "Deadline Alerts".
- Airflow 3.1 beta release update (Kaxil)
- Kaxil shared that the Airflow 3.1 beta version released earlier in the week did include the two major PRs highlighted in the last dev call i.e. TP's serialized DAG implementation changes and Kaxil's changes for defaults.
- As a result of the above, the server-side code now only uses serialized DAGs and Task Groups are decoupled between server and SDK.
- Also, migration testing from 2.11 to 3.1 beta has successfully been tested.
- Deadline Alerts (Dennis)
- Dennis shared that async support was being added to Notifiers.
- He also said that UI aspects were deferred to post 3.1 and that synchronous callbacks were in development, targeted for Airflow 3.2
- Language translations update (Shahar)
- Shahar shared that the language translations are at 97% coverage overall and we have been able to add new translations including Catalan and Simplified Chinese. Portuguese and Italian are still in PR, but may still be able to make it before release
- And that the the translation freeze has been extended until the 3.1 release to get to 100% coverage
- Amazing work here, I am astonished that we are now up to 14 language translations!
- Airflow 3.1 test plan update (Rahul)
- Discussion topics:
- UI plugins recommendation (Pierre)
- Pierre raised a discussion topic around marking React app plugins as experimental.
- Pierre said that the edge provider plugin work by Jens revealed needs for optimization and would probably require changes in build tools and dependency sharing. This would probably only affect very complex UI plugins, but it would be safer at this time to mark this part of the UI plugins work as "experimental".
- Based on this recommendation by Pierre and Brent, the team agreed with this recommendation.
- Airflow CTL update (Bugra)
- Bugra said that the first beta was released for "airflow ctl" and that testing was ongoing with limited community interaction to date.
- Bugra said that a follow-on release candidate is planned on this, rather than a second beta.
- Following on, Bugra created a new AIP to decouple remote commands from the Airflow core CLI, which included a transition approach.
- Vikram, Jarek, and Kaxil raised concerns about existing automation dependencies based on the preexisting airflow CLI commands.
- Vikram recommended that Bugra delay asking for a vote on this AIP until after the 3.1 release, so that the team has the cognitive capacity to process the changes.
- UI plugins recommendation (Pierre)
- Release preparation and related action items:
- Kaxil asked for the AIP (feature) owners to review the Airflow 3.1 draft release notes. Specifically including:
- Human in the loop (Wei, et al)
- Deadline alerts documentation (Dennis / Rajesh):
- Mention of the async vs. sync differences and what's coming in 3.1 vs. later
- Migration guide additions to explain the transition from SLA callbacks (in 2.x) to Deadline Alerts in 3.1
- UI React plugins documentation (PIerre / Brent):
- Including what is experimental vs. not
- Rahul asked Bugra to get back on the "airflow ctl" bugfix and testing
- Rahul asked Dennis to update the Deadline Alerts testing
- Vikram asked the team to review and cleanup the priority blocker issues (issues with label priority:upgrade_to_airflow3) and the 3.1 blocking issues
- Elad asked Dennis to confirm what was needed for Notifier provider release in conjunction with the 3.1 release
- Kaxil asked for the AIP (feature) owners to review the Airflow 3.1 draft release notes. Specifically including: