Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Publish "binary" wheel files for quicker install

...

  • Tag your release

    Code Block
    languagebash
    themeMidnight
    git tag ${VERSION}


  • Clean the checkout: the sdist step below will

    Code Block
    languagebash
    themeMidnight
    git clean -fxd


  • Tarball the repo

    Code Block
    languagebash
    themeMidnight
    git archive --format=tar.gz ${VERSION} --prefix=apache-airflow-${VERSION}/ -o apache-airflow-${VERSION}-source.tar.gz


  • Generate sdist
    NOTE: Make sure your checkout is clean at this stage - any untracked or changed files will otherwise be included in the file produced.

    Code Block
    languagebash
    themeMidnight
    python setup.py compile_assets sdist bdist_wheel


  • Rename the sdist

    Code Block
    languagebash
    themeMidnight
    mv dist/apache-airflow-${VERSION%rc?}.tar.gz apache-airflow-${VERSION}-bin.tar.gz
    mv dist/apache_airflow-${VERSION%rc?}-py2.py3-none-any.whl apache_airflow-${VERSION}-py2.py3-none-any.whl


  • Generate SHA512/ASC (If you have not generated a key yet, generate it by following instructions on http://www.apache.org/dev/openpgp.html#key-gen-generate-key)

    Code Block
    languagebash
    themeMidnight
    ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-${VERSION}-source.tar.gz && 
    ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-${VERSION}-bin.tar.gz
    ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache_airflow-${VERSION}-py2.py3-none-any.whl


  • Push Tags

    Code Block
    languagebash
    themeMidnight
    git push --tags


  • Push the artefacts to ASF dev dist repo

    Code Block
    languagebash
    themeMidnight
    # First clone the repo
    svn checkout https://dist.apache.org/repos/dist/dev/airflow airflow-dev
    
    # Create new folder for the release
    cd airflow-dev
    svn mkdir ${VERSION}
    
    # Move the artefacts to svn folder & commit
    mv ${AIRFLOW_REPO_ROOT}/apache{-,_}airflow-${VERSION}* airflow-dev/${VERSION}/
    cd ${VERSION}
    svn add *
    svn commit -m "Add artefacts for Airflow ${VERSION}" 


...

  • Edit the airflow/version.py to include the RC suffix.
  • python setup.py compile_assets sdist bdist_wheel
  • Follow the steps in Releasing Airflow
  • Throw away the change - we don't want to commit this: git checkout airflow/version.py

...

  • Create a ~/.pypirc file:

    Code Block
    languagebash
    $ cat ~/.pypirc 
    [distutils]
    index-servers =
      pypi
      pypitest
    
    [pypi]
    username=your-username
    password=*********
    
    [pypitest]
    repository=https://test.pypi.org/legacy/
    username=your-username
    password=*********

    (more details here.)

  • Set proper permissions for the pypirc file:

    Code Block
    languagebash
    $ chmod 600 ~/.pypirc


  • Confirm that airflow/version.py is set properly.
  • Install the package in PyPi's test environment:

    Code Block
    languagebash
    themeMidnight
    python setup.py compile_assets sdist bdist_wheel upload -r pypitest


  • Verify that the test package looks good by downloading it and installing it into a virtual environment. The package download link is available at:
    https://test.pypi.org/project/apache-airflow/#files
  • Install the package in PyPi's production environment:

    Code Block
    languagebash
    themeMidnight
    python setup.py compile_assets sdist bdist_wheel upload -r pypi


  • Again, confirm that the package is available here:
    https://pypi.python.org/pypi/apache-airflow

...

  • Get a diff between the last version and the current version:

    Code Block
    languagebash
    $ git log 1.8.0..1.9.0 --pretty=oneline


  • Update CHANGELOG.md with the details, and commit it.

Notifying developers of release

  • Notify dev@airflow.apache.org that the artifacts have been published:

    Code Block
    titleAirflow 1.10.2 is released
    collapsetrue
    Dear Airflow community,
    
    I'm happy to announce that Airflow 1.10.2 was just released.
    
    The source release, as well as the binary "sdist" release, are available
    here:
    
    https://dist.apache.org/repos/dist/release/airflow/1.10.2/
    
    We also made this version available on PyPi for convenience (`pip install apache-airflow`):
    
    https://pypi.python.org/pypi/apache-airflow
    
    The documentation is available on:
    https://airflow.apache.org/
    https://airflow.apache.org/1.10.2/
    https://airflow.readthedocs.io/en/1.10.2/
    https://airflow.readthedocs.io/en/stable/
    
    Find the CHANGELOG here for more details:
    
    https://airflow.apache.org/changelog.html#airflow-1-10-2-2019-01-19
    
    Cheers,
    <your name>


  • Update the Announce page on the wiki

Publishing documentation

Bulk migrate JIRAs

Old stuff that doesn't apply anymore

Expand
titleView legacy instructions

Committers should use the following commands and steps to create an Apache Airflow release:

  1. Run dev/airflow-jira on the test branch
    dev/airflow-jira compare 1.8.1

  2. Commit RCs here
    svn co https://dist.apache.org/repos/dist/dev/airflow

    Make sure CHANGELOG and UPDATING.md have been updated.

  3. Remove previous versions from  https://dist.apache.org/repos/dist/dev/airflow (`svn rm` -> `svn commit`)

  4. Run a travis build, confirm it passes (duh.)
  5. Build airflow
    python setup.py sdist

  6. Generate MD5
    gpg --print-md MD5 apache-airflow-1.8.1rc0+apache.incubating.tar.gz > apache-airflow-1.8.1rc0+apache.incubating.tar.gz.md5

  7. Generate the SHA512
    gpg --print-md SHA512 apache-airflow-1.8.1rc0+apache.incubating.tar.gz > apache-airflow-1.8.1rc0+apache.incubating.tar.gz.sha

  8. Upload your GPG public key to https://pgp.mit.edu

  9. Add your GPG pub key to https://dist.apache.org/repos/dist/release/airflow/KEYS , follow the instructions at the top of the file

  10. Sign Airflow tar.gz
    gpg --armor --output apache-airflow-1.8.1rc0+apache.incubating.tar.gz.asc --detach-sig apache-airflow-1.8.1rc0+apache.incubating.tar.gz

  11. Do votes in dev@airflow.incubator.apache.org and general@incubator.apache.org
    https://mail-archives.apache.org/mod_mbox/incubator-airflow-dev/201705.mbox/%3CCABYbY7eZk2SYHs3ni31h94uwwRdcn4dkJwumYtUMsfFLq-gwjg%40mail.gmail.com%3E
    https://mail-archives.apache.org/mod_mbox/incubator-general/201705.mbox/%3CCABYbY7cnACrrsrFtdwtq_-4AokMAe8CBOcWSvNz%2BhgB9DfVEvQ%40mail.gmail.com%3E

  12. Commit final release here
    svn co https://dist.apache.org/repos/dist/release/airflow

  13. Merge the test branch into the stable branch
    git pull apache v1-8-test
    git pull apache v1-8-stable
    git co v1-8-stable
    git merge v1-8-test
    git tag 1.8.1
    git push apache v1-8-stable
    git push apache 1.8.1

  14. Follow these instructions to publish to PyPi
    Note: You must remove the +incubating from the airflow/version.py file.

Source releases:
  • ...
  • Generate MD5s
    gpg --print-md MD5 apache-airflow-1.8.1rc0+apache.incubating.tar.gz > apache-airflow-1.8.1rc0+apache.incubating.tar.gz.md5

  • Generate the SHA512
    gpg --print-md SHA512 apache-airflow-1.8.1rc0+apache.incubating.tar.gz > apache-airflow-1.8.1rc0+apache.incubating.tar.gz.sha

  • Upload your GPG public key to https://pgp.mit.edu

  • Add your GPG pub key to https://dist.apache.org/repos/dist/release/airflow/KEYS , follow the instructions at the top of the file

  • Sign Airflow tar.gz
    gpg --armor --output apache-airflow-1.8.1rc0+apache.incubating.tar.gz.asc --detach-sig apache-airflow-1.8.1rc0+apache.incubating.tar.gz