Versions Compared

Old Version 34

changes.mady.by.user Unknown User (kaxilnaik)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (kaxilnaik)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
maxLevel2

#1: 10 Aug 2020 - Planning & Scoping

...

  • Scheduler HA

    • Progress

      • MySQL 8 tests have been added
      • Adoption of orphaned tasks have been added to Kubernetes and to the new CeleryKubernetesExecutor (PR)
      • All (but one) Tests are passing
      • Instructions to run the PR with Multiple Schedulers using docker-compose have been added (link to the comment)
      • Received some reviews on Monday -- Should be addressed by today/tomorrow
    • All Big/Critical Concerns have been addressed (escape hatch for locks, MySQL 8 tests, Moving callbacks to DagFileProcessor)
    • Room to improve code readability which isn't a blocker and would be addressed in follow-up PR
    • Ash is in talks with Kevin to test his Giant DAG to verify Scheduler Performance to address one of Kevin Yang's concern
    • There would be a separate PR for retrying transactions to avoid deadlocks as suggested by MySQL guide on handling deadlocks
  • API
    • Kamil and Omair are hoping to complete work on the Task Instance Endpoint PR by the end of this week
    • James is working to map existing permissions to the UI, hoping to complete by end of the week too. Link to PRs
  • Backport Providers
    • Backport packages 2020.10.5 will be released without cncf.kubernetes. We will create a separate release for cncf.kubernetes that will address the issues Kamil found in RCs. Link to VOTE thread.
  • Airflow 1.10.13 Release
  • Airflow 2.0 Alpha
    • Revisit on Thursday based on the progress with Scheduler HA & API whether we would be able to release first alpha this Friday.
  •  Others:
    • Kaxil to create Github Issues for the feedback received on the Documentation Improvement Thread and create threads asking for VOTES for previously discussed items like removing pickles.
    • Vikram will write up, share and all can review on what we mean by ALPHA and BETAs.

...

  • Airflow 2.0 Release Candidate: 
    • Targeting 2.0.0rc1 Wednesday - 9 Dec.
  • Upgrade Checks
    • Targeting 1.1.0rc1 Friday - 11 Dec.
  • Providers 1.0.0rc1
    • Targeting with 2.0.0rc1 on Wednesday - 9 Dec.
  • Airflow 1.10.14

    • 1.10.14rc4 has been cut and submitted for a vote. If vote passes, 1.10.14 would be released on Thursday 10 Dec

  • Deprecating Features
    • Follow strict SEMVER.
    • Features won’t be removed until a major version even if they are deprecated
    • Document a policy about that in CONTRIBUTING.rst or somewhere more appropriate
    • Ash to write a proposal about having a major release (e.g 3.0 release) with just removed features

#17: 14 Dec 2020

Attendees

Name

Company

Kaxil NaikAstronomer

Vikram Koka

Astronomer

Daniel ImbermanAstronomer
Ash Berlin-TaylorAstronomer

Jarek Potiuk

Polidea

John JacksonAmazon
RafalGoogle

Discussion and Key Decisions