Airflow Dev Calls

6 Jan 2021

Attendees

Summary

• Airflow 2.0.1 / Open Issue from 2.0:
  • Milestone: Airflow 2.0.1 Milestone · GitHub
  • Add ‘priority’ labels to all the open issues in 2.0.1 Milestone
  • Release 2.0.1 as soon as the reported bugs are fixed
  • Positive feedback from Scribd – Going PROD with 2.0 in 2 weeks. No major bugs reported by Astronomer, Google / Polidea and Amazon.
• Upgrade Checks
  • Release 1.2.0 soon with a fix for https://github.com/apache/airflow/issues/13293
  • Add a Rule to Check version of Database (MySQL / Postgres / SQLite)
• Providers
  • Elasticsearch and Google (breaking change - 2.0.0) providers would be released soon or at least RCs would be cut next week
• Release Process - Semantic Versioning
  • Changing the version of dependency should not count as breaking change.
  • Ash is working on the doc which will be PRd to Airflow soon
  • Still no unanimous agreement on whether "Major Version should contain new features or just removal of deprecation"
  • Highlight this process to all the committers (chances are some committers are not regularly active). The most important one is that we will strictly follow SEMVER.
• Airflow 2.1 - cadence, candidates and planning
  • Patch releases 2-3 weeks (Might be driven by the bugs and their severity)
  • Minor releases would be released every two months
  • No cadence for Major releases!
  • What happens to the PR that introduces breaking changes??
  • We should not merge that PR until it is a breaking change, instead we should make the PR backwards compatible
  • Or wait until next major release (assign appropriate Github Milestone)

20 Jan 2021

Attendees

Summary

• Airflow 2.0.1:
  • Milestone: Airflow 2.0.1 Milestone · GitHub
  • Target week of Feb 8 for release
• Providers Release
  • Next set of Providers will be released next week once https://github.com/apache/airflow/pull/13767 is merged with appropriate SEMVER version (e.g Google would be a major release while ES would be a patch release)
  • Backport Providers will be released for the ones that have some significant changes after or at the same time as the providers
  • Snowflake released a new version (v2.3.8) and the Snowflake provider which will use this version will be released too as currently all versions for Snowflake providers are YANKED.
    • This will be released after https://github.com/apache/airflow/pull/12188 is merged as it contains some incompatibilities with Azure Blob Storage
• Upgrade Checks
  • Release 1.2.0 soon with a fix for https://github.com/apache/airflow/issues/13293
  • Add a check for importing Operator/Sensor via Plugins if it does not already exist
  • Add docs in 1.10.15 about using upgrade-checks to upgrade to 2
• Helm Chart Release
  • We need to agree on what we call the first version: 0.0.1 vs 2.0.0
  • We need to review the current Helm chart before it is released as a lot of changes have been merged since it was first merged.
  • Current Test Coverage is not that great - ideally, we would like to cover more test cases before it can be released 
  • We hope to release it after 2.0.1 so bugs fixes are in by then and 2.0.1 can become the default version for the Helm Chart

Next Meeting Agenda

• Airflow 1.10.15
• TBC