The Release Candidate artefacts we vote upon should be the exact ones we vote against, without any modification than renaming – i.e. the contents of the files must be the same between voted release canidate and final release. Because of this the version in the built artefacts that will become the official Apache releases must not include the rcN suffix.
Set environment variables
# Set Version export VERSION=1.10.2rc1 # Set AIRFLOW_REPO_ROOT to the path of your git repo export AIRFLOW_REPO_ROOT=$(pwd) # Example after cloning git clone https://github.com/apache/airflow.git airflow cd airflow export AIRFLOW_REPO_ROOT=$(pwd) |
airflow/version.py (without the RC tag)Tag your release
git tag ${VERSION} |
Clean the checkout: the sdist step below will
git clean -fxd |
Tarball the repo
git archive --format=tar.gz ${VERSION} --prefix=apache-airflow-${VERSION}/ -o apache-airflow-${VERSION}-source.tar.gz |
Generate sdist
NOTE: Make sure your checkout is clean at this stage - any untracked or changed files will otherwise be included in the file produced.
python setup.py compile_assets sdist |
Rename the sdist
mv dist/apache-airflow-${VERSION%rc?}.tar.gz apache-airflow-${VERSION}-bin.tar.gz |
Generate SHA512/ASC (If you have not generated a key yet, generate it by following instructions on http://www.apache.org/dev/openpgp.html#key-gen-generate-key)
${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-${VERSION}-source.tar.gz && ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-${VERSION}-bin.tar.gz |
Push Tags
git push --tags |
Push the artefacts to ASF dev dist repo
# First clone the repo
svn checkout https://dist.apache.org/repos/dist/dev/airflow airflow-dev
# Create new folder for the release
cd airflow-dev
svn mkdir ${VERSION}
# Move the artefacts to svn folder & commit
mv ${AIRFLOW_REPO_ROOT}/apache-airflow-${VERSION}* airflow-dev/${VERSION}/
cd ${VERSION}
svn add *
svn commit -m "Add artefacts for Airflow ${VERSION}" |
At this point we have the artefact that we vote on, but as a convenience to developers we also want to publish "snapshots" of the RC builds to pypi for installing via pip. To do this we need to
airflow/version.py to include the RC suffix.python setup.py compile_assets sdistgit checkout airflow/version.pyIt is important to stress that this snapshot is not intended for users.
You will need to sign the release artifacts with your pgp key. After you have created a key, make sure you
# Create PGP Key gpg --gen-key # Checkout ASF dist repo svn checkout https://dist.apache.org/repos/dist/release/airflow cd airflow # Add your GPG pub key to KEYS file. Replace "Kaxil Naik" with your name (gpg --list-sigs "Kaxil Naik" && gpg --armor --export "Kaxil Naik" ) >> KEYS # Commit the changes svn commit -m "Add PGP keys of Airflow developers" |
See this for more detail on creating keys and what is required for signing releases.
http://www.apache.org/dev/release-signing.html#basic-facts
Once the RC is built (both source and binary), put them in the dev SVN repository:
https://dist.apache.org/repos/dist/dev/airflow/
Use the dev/airflow-jira script to generate a list of Airflow JIRAs that were closed in the release.
Send out a vote to the dev@airflow.incubator.apache.org mailing list:
Hey all, I have cut Airflow 1.9.0 RC7. This email is calling a vote on the release, which will last for 72 hours. Consider this my (binding) +1. Airflow 1.9.0 RC7 is available at: https://dist.apache.org/repos/dist/dev/airflow/1.9.0rc7/ apache-airflow-1.9.0rc7-source.tar.gz is a source release that comes with INSTALL instructions. apache-airflow-1.9.0rc7-bin.tar.gz is the binary Python "sdist" release. Public keys are available at: https://dist.apache.org/repos/dist/release/airflow/KEYS The release contains the following JIRAs: https://issues.apache.org/jira/browse/AIRFLOW-1 https://issues.apache.org/jira/browse/AIRFLOW-2 https://issues.apache.org/jira/browse/AIRFLOW-3 https://issues.apache.org/jira/browse/AIRFLOW-4 ... Cheers, <your name> |
Once the vote has been passed, you will need to send a result vote to dev@airflow.incubator.apache.org:
Hello, Apache Airflow (incubating) 1.9.0 (based on RC8) has been accepted. 5 “+1” binding votes received: - Maxime Beauchemin (binding) - Chris Riccomini (binding) - Bolke de Bruin (binding) - Joy Gao (binding) - Fokko Driesprong (binding) My next step is to open a thread with the IPMC. Cheers, <your name> |
And you will also need to send a vote request to general@incubator.apache.org:
Hello Incubator PMC’ers, The Apache Airflow community has voted and approved the proposal to release Apache Airflow 1.9.0 (incubating) based on 1.9.0 Release Candidate 8. We now kindly request the Incubator PMC members to review and vote on this incubator release. Airflow is a platform to programmatically author, schedule, and monitor workflows. Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress, and troubleshoot issues when needed. When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative. Artifacts are available at: https://dist.apache.org/repos/dist/dev/airflow Public keys are available at: https://dist.apache.org/repos/dist/release/airflow apache-airflow-1.9.0rc8-source.tar.gz <https://dist.apache.org/repos/dist/dev/airflow/1.9.0rc8/apache-airflow-1.9.0rc8-source.tar.gz> is a source release that comes with INSTALL instructions. Along with it, for convenience, find the binary Python "sdist" as apache-airflow-1.9.0rc8-bin.tar.gz <https://dist.apache.org/repos/dist/dev/airflow/1.9.0rc8/apache-airflow-1.9.0rc8-bin.tar.gz> Vote thread: https://mail-archives.apache.org/mod_mbox/incubator-airflow-dev/201712.mbox/%3CCABYbY7ecu8F8sp%3DgvZrvW3knj%3DQpdCUZu-Jh02KngjGOezhBKA%40mail.gmail.com%3E Git tag: https://github.com/apache/incubator-airflow/releases/tag/1.9.0rc8 The vote will be open for at least 72 hours and until the necessary number of votes are reached. Members please be sure to indicate "(Binding)" with your vote which will help in tallying the vote(s). * Here is my +1 (non-binding) * Cheers, <your name> |
Lastly, you'll need to send the general@ vote result to both general@incubator.apache.org and dev@airflow.incubator.apache.org:
The vote to release Airflow 1.9.0-incubating, having been open for 11 days is now closed. There were three binding +1s and no -1 votes. +1 (binding): Chris Douglas Justin Mclean Jakob Homan The release is approved. Thanks to all those who voted. Cheers, <your name> |
Create a ~/.pypirc file:
$ cat ~/.pypirc [distutils] index-servers = pypi pypitest [pypi] username=your-username password=********* [pypitest] repository=https://test.pypi.org/legacy/ username=your-username password=********* |
(more details here.)
Set proper permissions for the pypirc file:
$ chmod 600 ~/.pypirc |
Install the package in PyPi's test environment:
python setup.py compile_assets sdist upload -r pypitest |
Install the package in PyPi's production environment:
python setup.py compile_assets sdist upload -r pypi |
Again, confirm that the package is available here:
https://pypi.python.org/pypi/apache-airflow
Get a diff between the last version and the current version:
$ git log 1.8.0..1.9.0 --pretty=oneline |
Notify dev@airflow.incubator.apache.org that the artifacts have been published:
Dear Airflow community, Airflow 1.9.0 was just released. The source release as well as the binary "sdist" release are available here: https://dist.apache.org/repos/dist/release/airflow/1.9.0-incubating/ We also made this version available on PyPi for convenience (`pip install apache-airflow`): https://pypi.python.org/pypi/apache-airflow Find the CHANGELOG here for more details: https://github.com/apache/incubator-airflow/blob/master/CHANGELOG.txt Cheers, <your name> |
Update the Annouce page on the wiki
Source releases:
|