Page History

Before your first release, you should perform one-time configuration steps. This will set up your security keys for signing the release and access to various release repositories.

To prepare for each release, you should audit the project status in the JIRA issue tracker, and do necessary bookkeeping. Finally, you should create a release branch from which individual release candidates will be built.

One-time setup instructions

GPG Key

You need to have a GPG key to sign the release artifacts. Please be aware of the ASF-wide release signing guidelines. If you don’t have a GPG key associated with your Apache account, please create one according to the guidelines.

Determine your Apache GPG Key and Key ID, as follows:

gpg --list-keys

This will list your GPG keys. One of these should reflect your Apache account, for example:

--------------------------------------------------
pub   2048R/845E6689 2016-02-23
uid                  Nomen Nescio <anonymous@apache.org>
sub   2048R/BA4D50BE 2016-02-23

Here, the key ID is the 8-digit hex string in the pub line: 845E6689.

Now, add your Apache GPG key to the Flink’s KEYS file in the release repository at dist.apache.org. Follow the instructions listed at the top of these files. (Note: Only PMC members have write access to the release repository. If you end up getting 403 errors ask on the mailing list for assistance.)

Configure git to use this key when signing code by giving it your key ID, as follows:

git config --global user.signingkey 845E6689

You may drop the --global option if you’d prefer to use this key for the current repository only.

You may wish to start gpg-agent to unlock your GPG key only once using your passphrase. Otherwise, you may need to enter this passphrase hundreds of times. The setup for gpg-agent varies based on operating system, but may be something like this:

eval $(gpg-agent --daemon --no-grab --write-env-file $HOME/.gpg-agent-info)
export GPG_TTY=$(tty)
export GPG_AGENT_INFO

Access to Apache Nexus repository

Configure access to the Apache Nexus repository, which enables final deployment of releases to the Maven Central Repository.

1. You log in with your Apache account.
2. Confirm you have appropriate access by finding org.apache.flink under Staging Profiles.
3. Navigate to your Profile (top right dropdown menu of the page).
4. Choose User Token from the dropdown, then click Access User Token. Copy a snippet of the Maven XML configuration block.

5. Insert this snippet twice into your global Maven settings.xml file, typically ${HOME}/.m2/settings.xml. The end result should look like this, where TOKEN_NAME and TOKEN_PASSWORDare your secret tokens:

   Code Block
   language xml title settings.xml
   <settings> <servers> <server> <id>apache.releases.https</id> <username>TOKEN_NAME</username> <password>TOKEN_PASSWORD</password> </server> <server> <id>apache.snapshots.https</id> <username>TOKEN_NAME</username> <password>TOKEN_PASSWORD</password> </server> </servers> </settings>

   
   

Website development setup

Get ready for updating the Flink website by following the website development instructions.

GNU Tar Setup for Mac

Skip this step if you are not using a Mac. The default tar application on Mac does not support GNU archive format and defaults to Pax. This bloats the archive with unnecessary metadata that can result in additional files when decompressing (see 1.15.2-RC2 vote thread). Install gnu-tar and create a symbolic link to use in preference of the default tar program.

brew install gnu-tar
ln -s /usr/local/bin/gtar /usr/local/bin/tar
which tar

Create a new version in JIRA

When contributors resolve an issue in JIRA, they are tagging it with a release that will contain their changes. With the release currently underway, new issues should be resolved against a subsequent future release. Therefore, you should create a release item for this subsequent release, as follows:

1. In JIRA, navigate to the Flink > Administration > Versions.
2. Add a new release: choose the next minor version number compared to the one currently underway, select today’s date as the Start Date, and choose Add.

(Note: Only PMC members have access to the project administration. If you do not have access, ask on the mailing list for assistance.)

Triage release-blocking issues in JIRA

There could be outstanding release-blocking issues, which should be triaged before proceeding to build a release candidate. We track them by assigning a specific Fix version field even before the issue resolved.

The list of release-blocking issues is available at the version status page. Triage each unresolved issue with one of the following resolutions:

• If the issue has been resolved and JIRA was not updated, resolve it accordingly.
• If the issue has not been resolved and it is acceptable to defer this until the next release, update the Fix Version field to the new version you just created. Please consider discussing this with stakeholders and the dev@ mailing list, as appropriate.
  • When using "Bulk Change" functionality of Jira
    • First, add the newly created version to Fix Version for all unresolved tickets that have old the old version among its Fix Versions. 
    • Afterwards, remove the old version from the Fix Version. 
• If the issue has not been resolved and it is not acceptable to release until it is fixed, the release cannot proceed. Instead, work with the Flink community to resolve the issue.

Review and update documentation

There are a few pages in the documentation that need to be reviewed and updated for each release.

• Ensure that there exists a release notes page for each non-bugfix release (e.g., 1.5.0) in ./docs/release-notes/, that it is up-to-date, and linked from the start page of the documentation.
• Upgrading Applications and Flink Versions: https://ci.apache.org/projects/flink/flink-docs-master/ops/upgrading.html
• please extend this list.

Cross team testing

For user facing features that go into the release we'd like to ensure they can actually be used by Flink users. To achieve this the release managers ensure that an issue for cross team testing is created in the Apache Flink Jira. This can and should be picked up by other community members to verify the functionality and usability of the feature.
The issue should contain some entry points which enables other community members to test it. It should not contain documentation on how to use the feature as this should be part of the actual documentation. The cross team tests are performed after the feature freeze. Documentation should be in place before that. Those tests are manual tests, so do not confuse them with automated tests.
To sum that up:

• User facing features should be tested by other contributors
• The scope is usability and sanity of the feature
• The feature needs to be already documented
• The contributor creates an issue containing some pointers on how to get started (e.g. link to the documentation, suggested targets of verification)
• Other community members pick those issues up and provide feedback
• Cross team testing happens right after the feature freeze

Unless the pages have not been updated before, please create a JIRA ticket and mark it as release blocker.

Setup environment variables

Set up a few environment variables to simplify commands that follow. (We use bash Unix syntax in this guide.)

RELEASE_VERSION="1.5.0"
SHORT_RELEASE_VERSION="1.5"
CURRENT_SNAPSHOT_VERSION="$SHORT_RELEASE_VERSION-SNAPSHOT"
NEXT_SNAPSHOT_VERSION="1.6-SNAPSHOT"
SHORT_NEXT_SNAPSHOT_VERSION="1.6"

Review Release Notes in JIRA

JIRA automatically generates Release Notes based on the Fix Version field applied to issues. Release Notes are intended for Flink users (not Flink committers/contributors). You should ensure that Release Notes are informative and useful.

Open the release notes from the version status page by choosing the release underway and clicking Release Notes.

You should verify that the issues listed automatically by JIRA are appropriate to appear in the Release Notes. Specifically, issues should:

• Be appropriately classified as Bug, New Feature, Improvement, etc.
• Represent noteworthy user-facing changes, such as new functionality, backward-incompatible API changes, or performance improvements.
• Have occurred since the previous release; an issue that was introduced and fixed between releases should not appear in the Release Notes.
• Have an issue title that makes sense when read on its own.

Adjust any of the above properties to the improve clarity and presentation of the Release Notes.

Ensure that the JIRA release notes are also included in the release notes of the documentation (see section "Review and update documentation").

Content of Release Notes field from JIRA tickets 

To get the list of "release notes" field from JIRA, you can ran the following script using JIRA REST API (notes the maxResults limits the number of entries):

curl -s https://issues.apache.org/jira//rest/api/2/search?maxResults=200&jql=project%20%3D%20FLINK%20AND%20%22Release%20Note%22%20is%20not%20EMPTY%20and%20fixVersion%20%3D%20${RELEASE_VERSION} | jq '.issues[]|.key,.fields.summary,.fields.customfield_12310192' | paste - - -

jq  is present in most Linux distributions and on MacOS can be installed via brew.

Verify Java and Maven Version

All of the following steps require to use Maven 3.2.5 and Java 8. Modify your PATH environment variable accordingly if needed.

Clone flink into a fresh workspace

Create a new directory for this release and clone the Flink repo from github to ensure you have a clean workspace. This step is optional.

Verify that a Release Build Works

Run mvn -Prelease clean install to ensure that the build processes that are specific to that profile are in good shape. This step is optional.

Verify that no exclusions were erroneously added to the japicmp plugin that break compatibility guarantees

Check the exclusions for the japicmp-maven-plugin in the root pom for exclusions that:

• for minor releases, break source compatibility for  @Public APIs
• for patch releases, break source/binary compatibility for @Public  / @PublicEvolving  APIs

Any such exclusion must be properly justified, in advance.

Create a release branch

Release candidates are built from a release branch. As a final step in preparation for the release, you should create the release branch, push it to the code repository (you should probably do this once the whole process is done), and update version information on the original branch.

Most of the following commands have to be executed in the tools directory, we will prefix the command prompt to make this explicit.

tools $ releasing/create_snapshot_branch.sh
tools $ git checkout master
tools $ OLD_VERSION=$CURRENT_SNAPSHOT_VERSION NEW_VERSION=$NEXT_SNAPSHOT_VERSION releasing/update_branch_version.sh

v1_12("1.12"),
v1_13("1.13"),
v1_14("1.14"),
v1_15("1.15"),
v1_16("1.16");

<flink.version>1.18-SNAPSHOT</flink.version>

tools $ git checkout release-$SHORT_RELEASE_VERSION

The rest of this guide assumes that commands are run in the root (or tools directory) of a repository on the branch of the release version with the above environment variables set.

Checklist to proceed to the next step

1. Release Manager’s GPG key is published to dist.apache.org
2. Release Manager’s GPG key is configured in git configuration
3. Release Manager's GPG key is configured as the default gpg key. 
4. Release Manager has org.apache.flink listed under Staging Profiles in Nexus
5. Release Manager’s Nexus User Token is configured in settings.xml
6. There are no release blocking JIRA issues
7. Release Notes in JIRA have been audited and adjusted
8. Update upgrade compatibility table (docs/ops/upgrading.md).
9. Update Release Overview in Confluence
10. (major only) Release branch has been created and pushed
    1.  Cron job has been added on the release branch in (tools/azure-pipelines/build-apache-repo.yml)
11. (major only) Originating branch has the version information updated to the new version
12. (major only) New version is added to the FlinkVersion Enum.
13. (major only) Make sure flink-docker has dev-x.y branch and docker e2e tests run against this branch
14. (major only) docs/config.toml has been updated appropriately.
15. (major only) The documentation for the new major release is visible under https://nightlies.apache.org/flink/flink-docs-release-$SHORT_RELEASE_VERSION (after at least one doc build finishes).
16. (major only) The documentation for the new major release does not contain "-SNAPSHOT" in its version title, and all links refer to the corresponding version docs instead of master.
17. (major only) The dev-x.y branch of ($CURRENT_SNAPSHOT_VERSION) have been created in the Flink Benchmark repo.
18. (major only) The flink.version property of Flink Benchmark repo has been updated to the latest snapshot version.