Versions Compared

Old Version 137

changes.mady.by.user Piotr Nowojski

Saved on

compared with

New Version 138

changes.mady.by.user Piotr Nowojski

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Expand
titlePrepare for the release

Before your first release, you should perform one-time configuration steps. This will set up your security keys for signing the release and access to various release repositories.

To prepare for each release, you should audit the project status in the JIRA issue tracker, and do necessary bookkeeping. Finally, you should create a release branch from which individual release candidates will be built.

One-time setup instructions

GPG Key

You need to have a GPG key to sign the release artifacts. Please be aware of the ASF-wide release signing guidelines. If you don’t have a GPG key associated with your Apache account, please create one according to the guidelines.

Determine your Apache GPG Key and Key ID, as follows:

Code Block
languagebash
gpg --list-keys
This will list your GPG keys. One of these should reflect your Apache account, for example:
Code Block
languagebash
--------------------------------------------------
pub   2048R/845E6689 2016-02-23
uid                  Nomen Nescio <anonymous@apache.org>
sub   2048R/BA4D50BE 2016-02-23

Here, the key ID is the 8-digit hex string in the pub line: 845E6689.

Now, add your Apache GPG key to the Flink’s KEYS file in the release repository at dist.apache.org. Follow the instructions listed at the top of these files. (Note: Only PMC members have write access to the release repository. If you end up getting 403 errors ask on the mailing list for assistance.)

Configure git to use this key when signing code by giving it your key ID, as follows:

Code Block
languagebash
git config --global user.signingkey 845E6689

You may drop the --global option if you’d prefer to use this key for the current repository only.

You may wish to start gpg-agent to unlock your GPG key only once using your passphrase. Otherwise, you may need to enter this passphrase hundreds of times. The setup for gpg-agent varies based on operating system, but may be something like this:

Code Block
languagebash
eval $(gpg-agent --daemon --no-grab --write-env-file $HOME/.gpg-agent-info)
export GPG_TTY=$(tty)
export GPG_AGENT_INFO

Access to Apache Nexus repository

Configure access to the Apache Nexus repository, which enables final deployment of releases to the Maven Central Repository.

  1. You log in with your Apache account.
  2. Confirm you have appropriate access by finding org.apache.flink under Staging Profiles.
  3. Navigate to your Profile (top right dropdown menu of the page).
  4. Choose User Token from the dropdown, then click Access User Token. Copy a snippet of the Maven XML configuration block.

  5. Insert this snippet twice into your global Maven settings.xml file, typically ${HOME}/.m2/settings.xml. The end result should look like this, where TOKEN_NAME and TOKEN_PASSWORDare your secret tokens:

    Code Block
    languagexml
    titlesettings.xml
    <settings>
   <servers>
     <server>
       <id>apache.releases.https</id>
       <username>TOKEN_NAME</username>
       <password>TOKEN_PASSWORD</password>
     </server>
     <server>
       <id>apache.snapshots.https</id>
       <username>TOKEN_NAME</username>
       <password>TOKEN_PASSWORD</password>
     </server>
   </servers>
 </settings>


Website development setup

Get ready for updating the Flink website by following the website development instructions.

Create a new version in JIRA

When contributors resolve an issue in JIRA, they are tagging it with a release that will contain their changes. With the release currently underway, new issues should be resolved against a subsequent future release. Therefore, you should create a release item for this subsequent release, as follows:

  1. In JIRA, navigate to the Flink > Administration > Versions.
  2. Add a new release: choose the next minor version number compared to the one currently underway, select today’s date as the Start Date, and choose Add.

(Note: Only PMC members have access to the project administration. If you do not have access, ask on the mailing list for assistance.)

Triage release-blocking issues in JIRA

There could be outstanding release-blocking issues, which should be triaged before proceeding to build a release candidate. We track them by assigning a specific Fix version field even before the issue resolved.

The list of release-blocking issues is available at the version status page. Triage each unresolved issue with one of the following resolutions:

  • If the issue has been resolved and JIRA was not updated, resolve it accordingly.
  • If the issue has not been resolved and it is acceptable to defer this until the next release, update the Fix Version field to the new version you just created. Please consider discussing this with stakeholders and the dev@ mailing list, as appropriate.
  • If the issue has not been resolved and it is not acceptable to release until it is fixed, the release cannot proceed. Instead, work with the Flink community to resolve the issue.

Review and update documentation

There are a few pages in the documentation that need to be reviewed and updated for each release.

Unless the pages have not been updated before, please create a JIRA ticket and mark it as release blocker.

Review Release Notes in JIRA

JIRA automatically generates Release Notes based on the Fix Version field applied to issues. Release Notes are intended for Flink users (not Flink committers/contributors). You should ensure that Release Notes are informative and useful.

Open the release notes from the version status page by choosing the release underway and clicking Release Notes.

You should verify that the issues listed automatically by JIRA are appropriate to appear in the Release Notes. Specifically, issues should:

  • Be appropriately classified as Bug, New Feature, Improvement, etc.
  • Represent noteworthy user-facing changes, such as new functionality, backward-incompatible API changes, or performance improvements.
  • Have occurred since the previous release; an issue that was introduced and fixed between releases should not appear in the Release Notes.
  • Have an issue title that makes sense when read on its own.

Adjust any of the above properties to the improve clarity and presentation of the Release Notes.

Ensure that the JIRA release notes are also included in the release notes of the documentation (see section "Review and update documentation").

Content of Release Notes field from JIRA tickets 

To get the list of "release notes" field from JIRA, you can ran the following script using JIRA REST API:

Code Block
curl -s https://issues.apache.org/jira//rest/api/2/search?jql=project%20%3D%20FLINK%20AND%20%22Release%20Note%22%20is%20not%20EMPTY%20and%20fixVersion%20%3D%201.11.0 | jq '.issues[]|.key,.fields.summary,.fields.customfield_12310192' | paste - - -

Were last parameter passed in curl in jql is fixVersion%20%3D%201.11.0  defines tickets for which fix version will be searched for (in this case it is 1.11.0) .  

jq  is present in most Linux distributions and on MacOS can be installed via brew.

Verify that a Release Build Works

Run mvn -Prelease clean install to ensure that the build processes that are specific to that profile are in good shape.

Create a release branch

Release candidates are built from a release branch. As a final step in preparation for the release, you should create the release branch, push it to the code repository (you should probably do this once the whole process is done), and update version information on the original branch.

Set up a few environment variables to simplify Maven commands that follow. (We use bash Unix syntax in this guide.)

Code Block
languagebash
RELEASE_VERSION="1.5.0"
SHORT_RELEASE_VERSION="1.5"
CURRENT_SNAPSHOT_VERSION="$SHORT_RELEASE_VERSION-SNAPSHOT"
NEXT_SNAPSHOT_VERSION="1.6-SNAPSHOT"
SHORT_NEXT_SNAPSHOT_VERSION="1.6"

Most of the following commands have to be executed in the tools directory, we will prefix the command prompt to make this explicit.

Major release

Expand
titleMajor release

Major release

If you are doing a new major release, you need to update Flink version in two repositories

Flink repository

Create a branch for the new version that we want to release before updating the master branch to the next development version:

Code Block
languagebash
tools $ releasing/create_snapshot_branch.sh
tools $ git checkout master
tools $ OLD_VERSION=$CURRENT_SNAPSHOT_VERSION NEW_VERSION=$NEXT_SNAPSHOT_VERSION releasing/update_branch_version.sh

The newly created branch needs to be pushed to the official repository. Afterwards fork off the dev-master  a dev-x.y  branch in the https://github.com/apache/flink-docker repository. Make sure {{flink-docker/testing/run_travis_tests}} points to a correct snapshot version.

After pushing the new major release branch, as the last step you should also update the documentation build bot to also build the documentation for the new release branch. Check Managing Documentation on details on how to do that. You may also want to manually trigger a build to make the changes visible as soon as possible.

Flink Benchmarks repository

Inside https://github.com/apache/flink-benchmarks repository you need to manually update the flink.version property inside the main pom.xml file. It should be pointing to the most recent snapshot version ($NEXT_SNAPSHOT_VERSION). For example:

Code Block
languagebash
<flink.version>1.13-SNAPSHOT</flink.version>
Minor release

 Expand
titleMinor release
Minor release
If you're creating a new minor release you do not need to modify Flink Benchmarks. You will skip the "Major release" step and simply check out the the already existing branch for that version:
 Code Block
languagebash
tools $ git checkout release-$SHORT_RELEASE_VERSION
tools $ OLD_VERSION=$CURRENT_SNAPSHOT_VERSION NEW_VERSION=$NEXT_SNAPSHOT_VERSION releasing/update_branch_version.sh

The rest of this guide assumes that commands are run in the root (or tools directory) of a repository on the branch of the release version with the above environment variables set.
Checklist to proceed to the next step
  1. Release Manager’s GPG key is published to dist.apache.org
  2. Release Manager’s GPG key is configured in git configuration
  3. Release Manager has org.apache.flink listed under Staging Profiles in Nexus
  4. Release Manager’s Nexus User Token is configured in settings.xml
  5. JIRA release item for the subsequent release has been created
  6. There are no release blocking JIRA issues
  7. Release Notes in JIRA have been audited and adjusted
  8. Release branch has been created and pushed if it is a major release.
  9. Originating branch has the version information updated to the new version
  10. (major/minor only) Jenkins deployment updated to create snapshot artifacts for release branch (see here)
  11. (major/minor only) Cron end-to-end-tests branch setup for release branch
  12. (major only) Make sure flink-docker has dev-x.y branch and docker e2e tests run against this branch
  13. (major/minor only) Update upgrade compatibility table (docs/ops/upgrading.md.
  14. docs/config.toml has been updated appropriately.
  15. The new documentation for major releases is visible under https://ci.apache.org/projects/flink/flink-docs-release-$SHORT_RELEASE_VERSION (after at least one doc build finishes).
  16. The new documentation for major releases do not contain "-SNAPSHOT" in its version title, and all links refer to the corresponding version docs instead of master.
...