Page History

...

• 2.0 Beta1
  • Milestone: https://github.com/apache/airflow/milestone/10. 
  • We will cut out beta1 at some point this week as all the outstanding items in the milestone should be done in a day or two.
  • While announcing the beta we will ask users to check out the documentation too, so that we can identify missing gaps.
    • We need to specifically add instructions (Both Slack and Email) for users who find this missing notes to at least creates Github Issues describing what is missing and ideally if they can create PRs to fix it.
    • Python wheel, tar.gz and Dockerfile for beta1 will be published to facilitate easier testing
  • PRs to enable Black and Pyupgrade across the code-base will be rebased and merged before publishing beta1. This might cause some PRs to have conflicts and we will recommend users to squash all commits in their PRs so that it is easy to resolve conflicts as it should just be the matter of running pre-commit hook for black and pyupgrade.
• Separating Upgrade checks from 1.10.13
  • upgrade-check tool will now be released separately from Airflow 1.10.13. The main reason for deciding to do so is that we will be able to add more upgrade checks even after 1.10.13 is released.
  • Github Issue: https://github.com/apache/airflow/issues/11112 . Please add any suggestions or ask any question over there. If needed, we can do a separate call otherwise async discussion will continue on that Github issue.
• Airflow 1.10.13
  • Milestone: https://github.com/apache/airflow/milestone/7
  • Triage and Release process has started for 1.10.13. We aim to release it in the next 1-3 weeks. 
  • Releasing in in the next two weeks will allow people testing upgrade from 1.10.13 to Airflow 2.0.0beta2 which would be critical.

#12: 9 Nov 2020

Attendees

Discussion and Key Decisions

• 2.0 Beta1
  • Beta: https://github.com/apache/airflow/milestone/10
  • Airflow 2.0.0b1 was cut out on Monday, 9 Nov 2020
• 2.0 Beta2
  • Airflow 2.0.0b2 was cut out on Tuesday, 10 Nov 2020 after a bug was discovered that prevented testing of beta1
  • Testing: To everyone who wants to help with Airflow 2.0.0 beta testing, the following are the main areas we need more help from the community:
    • Kubernetes Executor and KubernetesPodOperator: 
    • CeleryKubernetesExecutor
    • KEDA AutoScaling + CeleryKubernetesExecutor
• 2.0 Beta3
  • Main Purpose
    • Test upgrade from 1.10.12 / 1.10.13 to 2.0.0b3
    • Regression from beta2
    • Providers discovery from beta2
• Separating Upgrade Checks from v1-10-test / 1.10.13
  • A reminder to Committers and any-one interested to add comments on https://github.com/apache/airflow/issues/11112 to decide whether we will keep the source code for the upgrade-check plugin inside airflow-repo or a separate repo

• Airflow 1.10.13

  • Milestone: https://github.com/apache/airflow/milestone/7
  • Should be released in the next 2-3 weeks. 
• New Time for Dev calls
  • The regular Monday dev calls will now happen an hour earlier i.e. 5:30 PM GMT every Monday

#13: 16 Nov 2020

Attendees

Discussion and Key Decisions

• Make KubernetePodOperator Backwards-compatible
  • PR: https://github.com/apache/airflow/pull/12384
  • Release cncf.kubernetes backport-provider package with that PR in to make it backwards compatible with 1.10.x
• Docs
  • Add Docs explaining the different components of Airflow, especially for users migrating from 1.10.x who are not following community news:
    • Providers
    • Backport-providers
    • Airflow Core
    • Helm Chart
    • Docker Image
  • Existing docs for Providers are at https://airflow.readthedocs.io/en/latest/provider-packages.html
• Upgrade Checks
  • Github Issue: https://github.com/apache/airflow/issues/11112
  • The package will be released separately to 1.10.13.
  • However, there is an on-going discussion whether the sources should remain in v1-10-test branch or not.
    • Call to discuss this will happen this Thursday, 19 Nov at 3-4 PM GMT. If anyone is interested to join, use this Zoom link
    • We will also set a date when this separate package should be released and when the alpha/beta will be out to test with Airflow 1.10.x
    • After the decision, this information will be available on the planning page.
• Providers Packages
  • PR for adding new Connections for a Provider using provider.yaml file is out. Each provider folder now contains a YAML file that is used for documentation, PR that introduced it: https://github.com/apache/airflow/pull/12304
  • This will be extended to discover Extra links too.
  • For now, this should be sufficient but in future, we can take a look at Pytest's plugin mechanism suggested by Ash.
• 2.0 Beta3
  • 2.0.0beta3 will be cut by the end of this week
  • Regular betas will be cut every week until we cut RCs

• Airflow 1.10.13

  • Milestone: https://github.com/apache/airflow/milestone/7
  • Jarek has cherry-picked more than 300 commits to v1-10-test that syncs breeze, Helm Chart changes with Master, updated dependencies and test fixes.
  • We are targeting RCs this Friday