DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
- Pending Item:
- John Jackson to create Github Issue to improve Airflow's observability story
- Vikram Koka to create Github Issue/PR to add a guide to help new contributors make doc-only contributions
- Ash Berlin-Taylor to complete the Draft PR on Release Docs / Version Policy. Jarek Potiuk to follow it up to add version policy for Providers
- Github Issues Stats
- Vikram presented some stats and charts around our Github Issues
- Kubernetes was an area that needs attention.
- Vikram presented some stats and charts around our Github Issues
- Apache Airflow 1.10.15
- Milestone: https://github.com/apache/airflow/milestone/22?closed=1
- Tentative Date: 1 Mar 2021
- Helm Chart
- Project Board: https://github.com/apache/airflow/projects/7 to track the work
- Tentative Release Date: Early April
- The first release will be 1.0.0
Agenda for next week:
3 March 2021
Attendees
Name | Company |
|---|---|
| Kaxil Naik | Astronomer |
| Vikram Koka | Astronomer |
| Kevin Yang | AirBnb |
| Jarek Potiuk | Snowflake |
| Daniel Imberman | Astronomer |
| Dennis Akpenyi | Astronomer |
| John Jackson | AWS |
| Ash Berlin-Taylor | Astronomer |
| Ian Stanton | Astronomer |
| James Timmins | Astronomer |
| Dennis KVDM | Astronomer |
| Ephraim Anierobi | Astronomer |
Summary
- Add a policy document around all our Policies / rules in the Github Repo itself
- This should cover any decision we take as PMC / Committers, for example:
- Versioning Policy - Core, Providers and all others
- Committers have a binding vote on AIP Releases
- Lazy Consensus thread will be created on the mailing list for each decision agreed on the Dev Call
- Providers Release Policy
- Batch vs Ad-hoc:
- Release Manager would default to release Providers in Batch
- ad-hoc releases are OK (i.e. if there is a critical bug that needs fixing in a single provider)
- Frequency:
- For Batch release, we will release every month (starting of the month - 1 to 7 most likely)
- Just a note that it generally takes around a week for the vote to pass even though we have 72 hours min period
- Testing
- License and Signature Checks are mandatory (following the ASF rules)
- For Providers, not all changes require strict testing -- you make a judgement based on the changes for a particular provider
- For Airflow core it is different, smoke tests should be carried out and we rely on community testing too (Kaxil to create a separate Mailing list thread to discuss this further since there wasn't 100% agreement)
- Doc-only changes
- When we have doc-only changes for Providers (during batch-release), we should still release a new version. Majority on the Dev call agreed that releasing docs asap is good instead of waiting for the next release with a code-change.
- Feedback on AIPs
- AIP-39 Richer scheduler_interval
- Foreign keys between TaskInstance and DagRun will be changed from execution_date and dag_id to dagrun_id.
- This and any other changes with columns will need migration that might be slow for deployments with many Dags and large TIs or DagRuns in Metadata DB.
- Kevin Yang from Airbnb said that they have tried something similar but instead of dagrun_id they used DagRun.id (auto-incrementing id for each DagRun) which had a performance impact.
- AIP-38 Modern Web Application is already being Voted on: link
- TaskGroups support in Tree View
- We agreed that this would be good to have since we are planning to remove SubDags for TaskGroups
- A separate Mailing list thread has been created for more details and further discussion: link
- TaskGroups for Tree View (Kevin Yang) Provider Release CadenceAd-hoc Release vs Regular Releases