Versions Compared

Old Version 43

changes.mady.by.user Unknown User (vikramkoka)

Saved on

compared with

New Version 44

changes.mady.by.user Unknown User (vikramkoka)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Finished meeting summary from last dev call, added agenda for next

...


8 May 2025

Proposed Agenda

...

  • Development Updates and Presentations:
    • Airflow 3.1.x patch release update (Ephraim Anierobi)
      • Ephraim said that 3.1.2 had been released successfully.
      • Jarek reported that there was one issue reported right after about disappearing logs which may be critical and require a follow-on patch release. 
      • Rahul chimed in to say that this log issue was reproducible and a fix had also been identified and tested. 
      • There was agreement that this may require a 3.1.3 very soon, instead of waiting for the 2 week release cycle.
      • This is currently scheduled for this week and added to the Airflow 3.x wiki page
    • UI performance issues (Pierre Jeambrun)
      • Auth issues (Vincent Beck)
      • Expanding Task SDK Integration test framework with more tests (Amogh)
      Discussion topics:
    • Issue triage process
      • Thoughts on how to document DB access options in Airflow 3 upgrade docs (Amogh)
        • Pierre reported that a number of N+1 query problems had been identified, were being resolved, and guard rails being put in place. The root cause was serialization layer lazy-loading relationships in loops. 
        • Pierre also referenced an issue that had identified missing indexes as a source of slowness and this was being resolved by new index creation. Vikram raised his concern that new index creation could cause issues in the "DB migration" part of an Airflow upgrade. Ash concurred with the concern and proposed a solution to make index creation part of the API server/ Scheduler startup rather than as part of the migration. 
        • Brent added that the Grid view performance remains challenging and that additional optimization work was being planned after the N+1 fixes were complete. 
        • There was also discussion about FastAPI configuration changes because of scaling differences from the Flask approach. This triggered a need for updating the documentation to recommend scaling approach recommendations. 
      • Auth issues (Vincent Beck)
        • Vincent reported that issues related to Auth were being resolved and that he had taken this on at Vikram's request. 
      • Expanding Task SDK Integration test framework with more tests (Amogh)
        • There was a quick ask for help from Amogh requesting community contributions to the Task SDK integration test framework. 
        • Amogh said that the complexity was higher than previous efforts and may require a SIG on slack for coordination. 
    • Discussion topics:
      • Issue triage process (Vikram)
        • Vikram followed up on his email summary of issues sent to the dev list earlier, saying that the "needs triage" label was applied to 73 of the 284 open issues related to Airflow 3. And that this still seemed to be applied even after a PR had been created to address the issue. 
        • Jarek chimed in to say that this was unintentional and that at least he himself often forgot to assign or remove labels during the review process. 
        • Vikram proposed adoption of issues into logical swim lanes, with volunteer owners for those lanes such as:
          • Auth issues: Vincent leading
          • UI / API issues: Pierre and Brent leading
          • Data aware scheduling: TP and Wei leading
          • Edge Worker: Jens leading
        • There was some discussion around this with senior contributors such as Ash, saying that they look at everything, not based on individual areas. However, Vincent and Jens chimed in saying that this would be useful for them to focus attention. Brent chimed in saying that sometimes issues were mislabeled with the UI tag, but that was solvable by reassigning the UI-labelled issues post-initial triage. 
        • An aspiring contributor commented that these swim lane labels would also be useful for issues tagged with "Good first issue", so that they could pick something to work on based on their own skills and interests. 
        • Elad pointed out that this needs to be tried out in practice and if it works, also applied to PRs, since there are many PRs sitting waiting for approval. At this point, we have hit a record of 343 open PRs in the project! 
      • Thoughts on how to document DB access options in Airflow 3 upgrade docs (Amogh)
        • Amogh said that the Database access options topic had raised a lot of discussion on the PR. 
        • As a result, Amogh started a dev list discussion and was looking for input. Based on that, a lazy consensus would be started middle of next week. 


    Proposed Agenda

    Expand