Versions Compared

Old Version 36

changes.mady.by.user Unknown User (kaxilnaik)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (kaxilnaik)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
maxLevel2

#1: 10 Aug 2020 - Planning & Scoping

...

  • Scheduler HA

    • Progress

      • MySQL 8 tests have been added
      • Adoption of orphaned tasks have been added to Kubernetes and to the new CeleryKubernetesExecutor (PR)
      • All (but one) Tests are passing
      • Instructions to run the PR with Multiple Schedulers using docker-compose have been added (link to the comment)
      • Received some reviews on Monday -- Should be addressed by today/tomorrow
    • All Big/Critical Concerns have been addressed (escape hatch for locks, MySQL 8 tests, Moving callbacks to DagFileProcessor)
    • Room to improve code readability which isn't a blocker and would be addressed in follow-up PR
    • Ash is in talks with Kevin to test his Giant DAG to verify Scheduler Performance to address one of Kevin Yang's concern
    • There would be a separate PR for retrying transactions to avoid deadlocks as suggested by MySQL guide on handling deadlocks
  • API
    • Kamil and Omair are hoping to complete work on the Task Instance Endpoint PR by the end of this week
    • James is working to map existing permissions to the UI, hoping to complete by end of the week too. Link to PRs
  • Backport Providers
    • Backport packages 2020.10.5 will be released without cncf.kubernetes. We will create a separate release for cncf.kubernetes that will address the issues Kamil found in RCs. Link to VOTE thread.
  • Airflow 1.10.13 Release
  • Airflow 2.0 Alpha
    • Revisit on Thursday based on the progress with Scheduler HA & API whether we would be able to release first alpha this Friday.
  •  Others:
    • Kaxil to create Github Issues for the feedback received on the Documentation Improvement Thread and create threads asking for VOTES for previously discussed items like removing pickles.
    • Vikram will write up, share and all can review on what we mean by ALPHA and BETAs.

...

Discussion and Key Decisions