Versions Compared

Old Version 42

changes.mady.by.user Unknown User (vikramkoka)

Saved on

compared with

New Version 43

changes.mady.by.user Unknown User (vikramkoka)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: adding summary from last dev call

...

  • Catch-up on action items from last call:
    • DAG import issues (Dheeraj)
      • Dheeraj said that he had re-tested the upgrade process and that the RUFF based utilities had significantly improved DAG compatibility from Airflow 2 to Airflow 3, when run with autofix, with over 50% of all the DAGs successfully parsed with Airflow 3, without needing manual changes. 
      • Dheeraj went to to say that the remaining issues requiring manual fixes were with: 
        • airflow.utils days_ago method,
        • DB create session no longer being available, because of direct database access removal
        • Simple HTTP Operator deprecation and Bash Operator being moved 
      • Dheeraj's summary was that the migration timeline after using the utilities would be about 3-4 days to achieve 80-90% DAG compatibility
      • The only remaining issue in his mind was UI performance, where it seemed that there was a noticeable slowdown as compared to Airflow 2.x
      • This report raised a fair amount of questions and discussion in the meeting itself. It was very helpful for the rest of the team to hear Dheeraj's feedback!
  • Development Updates and Presentations:
    • Airflow 3.1.x patch release update (Ephraim Anierobi)
      • Ephraim said that 3.1.2 had been released successfully.
      • Jarek reported that there was one issue reported right after about disappearing logs which may be critical and require a follow-on patch release. 
      • Rahul chimed in to say that this log issue was reproducible and a fix had also been identified and tested. 
      • There was agreement that this may require a 3.1.3 very soon, instead of waiting for the 2 week release cycle.
      • This is currently scheduled for this week and added to the Airflow 3.x wiki page
    Development Updates and Presentations:
    • Airflow 3.1.x patch release update (Ephraim Anierobi)
    • UI performance issues (Pierre Jeambrun)
    • Auth issues (Vincent Beck)
    • Expanding Task SDK Integration test framework with more tests (Amogh)
  • Discussion topics:
    • Issue triage process
    • Thoughts on how to document DB access options in Airflow 3 upgrade docs (Amogh)

...