Page History

The core of the release process is the build-vote-fix cycle. Each cycle produces one release candidate. The Release Manager repeats this cycle until the community approves one release candidate, which is then finalized.

Build and stage Java and Python artifacts

Set up a few environment variables to simplify Maven commands that follow. This identifies the release candidate being built. Start with RC_NUM equal to 1 and increment it for each candidate.

RC_NUM="1"
TAG="release-${RELEASE_VERSION}-rc${RC_NUM}"

Now, create a release branch ( this step can not be skipped for minor releases):

$ cd tools
tools $ OLD_VERSION=$CURRENT_SNAPSHOT_VERSION NEW_VERSION=$RELEASE_VERSION RELEASE_CANDIDATE=$RC_NUM releasing/create_release_branch.sh

Tag the release commit:

git tag -s ${TAG} -m "${TAG}"

We now need to do several things:

• Create the source release archive
• Deploy jar artefacts to the Apache Nexus Repository, which is the staging area for deploying the jars to Maven Central
• Build PyFlink wheel packages (since 1.11)

You might want to create a directory on your local machine for collecting the various source and binary releases before uploading them. Creating the binary releases is a lengthy process but you can do this on a another machine (for example, in the "cloud"). When doing this, you can skip signing the release files on the remote machine, download them to your local machine and sign them there.

First, we build the source release:

tools $ RELEASE_VERSION=$RELEASE_VERSION releasing/create_source_release.sh

Next, we stage the maven artifacts:

tools $ releasing/deploy_staging_jars.sh

Review all staged artifacts (https://repository.apache.org/). They should contain all relevant parts for each module, including pom.xml, jar, test jar, source, test source, javadoc, etc. Carefully review any new artifacts.

Close the staging repository on Apache Nexus. When prompted for a description, enter “Apache Flink, version X, release candidate Y”.

Then, you need to build the PyFlink wheel packages.(since 1.11) 

1. Set up an azure pipeline in your own Azure account. You can refer to Azure Pipelines for more details on how to set up azure pipeline for a fork of the Flink repository. Note that a google cloud mirror in Europe is used for downloading maven artifacts, therefore it is recommended to set your Azure organization region to Europe to speed up the downloads.

2. Push the release candidate branch to your forked personal Flink repository, e.g.

   Code Block
   language bash
   tools $ git push <remote> refs/heads/release-${RELEASE_VERSION}-rc${RC_NUM}:release-${RELEASE_VERSION}-rc${RC_NUM}

   
   

3. Trigger the Azure Pipelines manually to build the PyFlink wheel packages
   
   1. Go to your Azure Pipelines Flink project → Pipelines
   2. Click the "New pipeline" button on the top right
   3. Select "GitHub" → your GitHub Flink repository → "Existing Azure Pipelines YAML file"
   4. Select your branch → Set path to "/azure-pipelines.yaml" → click on "Continue" → click on "Variables"
   5. Then click "New Variable" button, fill the name with "MODE", and the value with "release". Click "OK" to set the variable and the "Save" button to save the variables, then back on the "Review your pipeline" screen click "Run" to trigger the build.
   6. You should now see a build where only the "CI build (release)" is running
      
4. Download the PyFlink wheel packages from the build result page after the jobs of "build_wheels mac" and "build_wheels linux" have finished.
   1. Download the PyFlink wheel packages
      1. Open the build result page of the pipeline
      2. Go to the `Artifacts` page (build_wheels linux -> 1 artifact)
      3. Click `wheel_Darwin_build_wheels mac` and `wheel_Linux_build_wheels linux` separately to download the zip files

   2. Unzip these two zip files

      Code Block
      language bash
      $ cd /path/to/downloaded_wheel_packages $ unzip wheel_Linux_build_wheels\ linux.zip $ unzip wheel_Darwin_build_wheels\ mac.zip

      
      

   3. Create directory `dist` under the directory of flink-python

      Code Block
      language bash
      $ cd <flink-dir> $ mkdir flink-python/dist

      
      

   4. Move the unzipped wheel packages to the directory of flink-python/dist

      Code Block
      language bash
      $ mv /path/to/wheel_Darwin_build_wheels\ mac/* flink-python/dist/ $ mv /path/to/wheel_Linux_build_wheels\ linux/* flink-python/dist/ $ cd tools

      
      

Finally, we create the binary convenience release files:

tools $ RELEASE_VERSION=$RELEASE_VERSION releasing/create_binary_release.sh

If you want to run this step in parallel on a remote machine you have to make the release commit available there (for example by pushing to a repository). This is important: the commit inside the binary builds has to match the commit of the source builds and the tagged release commit. When building remotely, you can skip gpg signing by setting  SKIP_GPG=true. You would then sign the files manually after downloading them to your machine:

for f in flink-*-bin*.tgz; do gpg --armor --detach-sig $f; done

gpg --armor --detach-sig apache-flink-*.tar.gz

The release manager need to make sure the PyPI project `apache-flink` and `apache-flink-libraries` has enough available space for the python artifacts. The remaining space must be larger than the size of `tools/releasing/release/python`. Login with the PyPI admin account (account info is only available to PMC members) and check the remaining space in project settings. 

Request an increase if there's not enough space. Note, it could take some days for PyPI to review our request. 

Stage source and binary releases on dist.apache.org

Copy the source release to the dev repository of dist.apache.org.

1. If you have not already, check out the Flink section of the dev repository on dist.apache.org via Subversion. In a fresh directory:

   Code Block
   language bash
   svn checkout https://dist.apache.org/repos/dist/dev/flink --depth=immediates

   
   

2. Make a directory for the new release:

   Code Block
   language bash
   mkdir flink/flink-${RELEASE_VERSION}-rc${RC_NUM}

   Copy Flink source/binary distributions, hashes, and GPG signature and the python subdirectory:

   Code Block
   mv <flink-dir>/tools/releasing/release/* flink/flink-${RELEASE_VERSION}-rc${RC_NUM}

   
   

3. Add and commit all the files.

   Code Block
   language bash
   cd flink svn add flink-${RELEASE_VERSION}-rc${RC_NUM} svn commit -m "Add flink-${RELEASE_VERSION}-rc${RC_NUM}"

   
   

4. Verify that files are present

(Push the release tag)

If you haven't pushed the release tag yet, here's the command:

git push <remote> refs/tags/release-${RELEASE_VERSION}-rc${RC_NUM}

Propose a pull request for website updates

The final step of building the candidate is to propose a website pull request containing the following changes:

• update docs/data/flink.yml
  
  • Add a new major version or update minor version as required
  • Update release_archive.yml
  • update version references in quickstarts (q/ directory) as required (outdated?)
• add a blog post announcing the release in _posts
• add a organized release notes page under docs/content/release-notes and docs/content.zh/release-notes (like https://nightlies.apache.org/flink/flink-docs-release-1.15/release-notes/flink-1.15/). The page is based on the non-empty release notes collected from the issues, and only the issues that affect existing users should be included (e.g., instead of new functionality). It should be in a separate PR since it would be merged to the flink project. 

Don’t merge the PRs before finalizing the release.

Checklist to proceed to the next step

1. Maven artifacts deployed to the staging repository of repository.apache.org
2. Source distribution deployed to the dev repository of dist.apache.org
3. Website pull request proposed to list the release
4. (major only) Check docs/config.toml to ensure that
   • the version constants refer to the new version
   • the baseurl does not point to flink-docs-master  but flink-docs-release-X.Y instead

You can (optionally) also do additional verification by:

1. Check hashes (e.g. shasum -c *.sha512)

2. Check signatures (e.g. gpg --verify flink-1.2.3-source-release.tar.gz.asc flink-1.2.3-source-release.tar.gz)
3. grep for legal headers in each file.
4. If time allows check the NOTICE files of the modules whose dependencies have been changed in this release in advance, since the license issues from time to time pop up during voting. See Verifying a Flink Release "Checking License" section.

Once the release has been finalized, the last step of the process is to promote the release within the project and beyond. Please wait for 24h after finalizing the release in accordance with the ASF release policy.

Update japicmp configuration

Update the japicmp reference version and wipe exclusions / enable API compatibility checks for @PublicEvolving APIs on the corresponding SNAPSHOT branch with the update_japicmp_configuration.sh script (see below).

For a new major release (x.y.0), run the same command also on the master branch for updating the japicmp reference version and removing out-dated exclusions in the japicmp configuration.

Make sure that all Maven artifacts are already pushed to Maven Central. Otherwise, there's a risk that CI fails due to missing reference artifacts.

tools $ NEW_VERSION=$RELEASE_VERSION releasing/update_japicmp_configuration.sh
tools $ cd ..
$ git add *
$ git commit -m "Update japicmp configuration for $RELEASE_VERSION"

Merge website pull request

Merge the website pull request to list the release. Make sure to regenerate the website as well, as it isn't build automatically.

Remove outdated versions

dist.apache.org

For a new major release remove all release files older than 2 versions, e.g., when releasing 1.7, remove all releases <= 1.5.

For a new bugfix version remove all release files for previous bugfix releases in the same series, e.g., when releasing 1.7.1, remove the 1.7.0 release.

1. If you have not already, check out the Flink section of the release repository on dist.apache.org via Subversion. In a fresh directory:

   Code Block
   language bash
   svn checkout https://dist.apache.org/repos/dist/release/flink --depth=immediates cd flink

   
   

2. Remove files for outdated releases and commit the changes.

   Code Block
   language bash
   svn remove flink-<version_to_remove> svn commit

   
   

3. Verify that files  are removed

   Warning
   Remember to remove the corresponding download links from the website.

   
   

CI

Disable the cron job for the now-unsupported version from (tools/azure-pipelines/build-apache-repo.yml) in the respective branch.

Apache mailing lists

Announce on the dev@ mailing list that the release has been finished.

Announce on the release on the user@ mailing list, listing major improvements and contributions.

Announce the release on the announce@apache.org mailing list.

From: Release Manager
To: dev@flink.apache.org, user@flink.apache.org, user-zh@flink.apache.org, announce@apache.org
Subject: [ANNOUNCE] Apache Flink 1.2.3 released

The Apache Flink community is very happy to announce the release of Apache Flink 1.2.3, which is the third bugfix release for the Apache Flink 1.2 series.

Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications.

The release is available for download at:
https://flink.apache.org/downloads.html

Please check out the release blog post for an overview of the improvements for this bugfix release:
<blob post link>

The full release notes are available in Jira:
<jira release notes link>

We would like to thank all contributors of the Apache Flink community who made this release possible!

Feel free to reach out to the release managers (or respond to this thread) with feedback on the release process. Our goal is to constantly improve the release process. Feedback on what could be improved or things that didn't go so well are appreciated.

Regards,
Release Manager

Recordkeeping

Use reporter.apache.org to seed the information about the release into future project reports.

(Note: Only PMC members have access report releases. If you do not have access, ask on the mailing list for assistance.)

Flink blog

Major or otherwise important releases should have a blog post. Write one if needed for this particular release. Minor releases that don’t introduce new major functionality don’t necessarily need to be blogged (see flink-web PR #581 for Flink 1.15.3 as an example for a minor release blog post).

Please make sure that the release notes of the documentation (see section "Review and update documentation") are linked from the blog post of a major release.
We usually include the names of all contributors in the announcement blog post. Use the following command to get the list of contributors:

git log --pretty="%an%n%cn" <fromCommit>..<toCommit> | sort | uniq | tr "\n" "," | sed 's/,/, /g'

Social media

Tweet, post on Facebook, LinkedIn, and other platforms. Ask other contributors to do the same.

Flink Release Wiki page

Add a summary of things that went well or that went not so well during the release process. This can include feedback from contributors but also more generic things like the release have taken longer than initially anticipated (and why) to give a bit of context to the release process.

End of Life for Unsupported versions

For major versions only. As per our support policy for old Flink versions when we release a new 1.x version we should start a discussion thread to end support for old versions.

1. Check if there are any resolved critical/blocker issues in the version exiting support. For example if you have released 1.17.0 then 1.15 is now end of life. 
2. Open a discussion thread to either 1/ start the final bugfix release or 2/ close the support window (example https://lists.apache.org/thread/csm81mlcktobl00p37x9t2jkpwmj9m58)

Checklist to declare the process completed

1. Website pull request to list the release merged
2. Release announced on the user@ mailing list.
3. Blog post published, if applicable.
4. Release recorded in reporter.apache.org.
5. Release announced on social media.
6. Completion declared on the dev@ mailing list.
7. Update Homebrew: https://docs.brew.sh/How-To-Open-a-Homebrew-Pull-Request (seems to be done automatically - at least for minor releases  for both minor and major releases)
8. Update quickstart scripts in flink-web, under the q/ directory
9. Updated the japicmp configuration
   • corresponding SNAPSHOT branch japicmp reference version set to the just released version, and API compatibiltity checks for @PublicEvolving  was enabled
   • (major only) master branch japicmp reference version set to the just released version
   • (major only) master branch japicmp exclusions have been cleared
10. Update the list of previous version in docs/config.toml on the master branch.
11. Set show_outdated_warning: true in docs/config.toml in the branch of the previous Flink version

12. Update stable and master alias in https://github.com/apache/flink/blob/master/.github/workflows/docs.yml

    Code Block
    if [ "${currentBranch}" = "master" ]; then echo "flink_alias=release-1.16" >> ${GITHUB_ENV} elif [ "${currentBranch}" = "release-1.14" ]; then echo "flink_alias=stable" >> ${GITHUB_ENV} fi -> if [ "${currentBranch}" = "master" ]; then echo "flink_alias=release-1.16" >> ${GITHUB_ENV} elif [ "${currentBranch}" = "release-1.15" ]; then echo "flink_alias=stable" >> ${GITHUB_ENV} fi

    
    

13. Update docs to "stable" in docs/config.toml in the branch of the just-released version:
    • Change Version from x.y-SNAPSHOT to x.y.z, i.e. 1.6-SNAPSHOT to 1.6.0
    • Change VersionTitle from x.y-SNAPSHOT to x.y, i.e. 1.6-SNAPSHOT to 1.6
    • Change Branch from master to release-x.y, i.e. master to release-1.6
    • Change baseURL from //ci.apache.org/projects/flink/flink-docs-master to //ci.apache.org/projects/flink/flink-docs-release-x.y
    • Change javadocs_baseurl from //ci.apache.org/projects/flink/flink-docs-master to //ci.apache.org/projects/flink/flink-docs-release-x.y
    • Change IsStable to true
14. Update CURRENT_VERSION in TypeSerializerUpgradeTestBase  with the new version. This will likely fail some tests because snapshots are missing for that version. Generate them, for example in TypeSerializerUpgradeTestBase. 
15. (major/minor only) Update migration tests in master to cover migration from new version: (search for usages of FlinkVersion)
    
    • AbstractOperatorRestoreTestBase
    • CEPMigrationTest
    • BucketingSinkMigrationTest
    • FlinkKafkaConsumerBaseMigrationTest
    • ContinuousFileProcessingMigrationTest
    • WindowOperatorMigrationTest
    • StatefulJobSavepointMigrationITCase
    • StatefulJobWBroadcastStateMigrationITCase
16. (major only) Opened discussion thread for End of Life for Unsupported versions