...
Expand | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Before your first release, you should perform one-time configuration steps. This will set up your security keys for signing the release and access to various release repositories. To prepare for each release, you should audit the project status in the JIRA issue tracker, and do necessary bookkeeping. Finally, you should create a release branch from which individual release candidates will be built. One-time setup instructionsGPG KeyYou need to have a GPG key to sign the release artifacts. Please be aware of the ASF-wide release signing guidelines. If you don’t have a GPG key associated with your Apache account, please create one according to the guidelines. Determine your Apache GPG Key and Key ID, as follows:
This will list your GPG keys. One of these should reflect your Apache account, for example:
Here, the key ID is the 8-digit hex string in the Now, add your Apache GPG key to the Flink’s
Configure
You may drop the You may wish to start
Access to Apache Nexus repositoryConfigure access to the Apache Nexus repository, which enables final deployment of releases to the Maven Central Repository.
Website development setupGet ready for updating the Flink website by following the website development instructions. GNU Tar Setup for MacSkip this step if you are not using a Mac. The default tar application on Mac does not support GNU archive format and defaults to Pax. This bloats the archive with unnecessary metadata that can result in additional files when decompressing (see 1.15.2-RC2 vote thread). Install gnu-tar and create a symbolic link to use in preference of the default tar program.
Create a new version in JIRAWhen contributors resolve an issue in JIRA, they are tagging it with a release that will contain their changes. With the release currently underway, new issues should be resolved against a subsequent future release. Therefore, you should create a release item for this subsequent release, as follows:
(Note: Only PMC members have access to the project administration. If you do not have access, ask on the mailing list for assistance.) Triage release-blocking issues in JIRAThere could be outstanding release-blocking issues, which should be triaged before proceeding to build a release candidate. We track them by assigning a specific The list of release-blocking issues is available at the version status page. Triage each unresolved issue with one of the following resolutions:
Review and update documentationThere are a few pages in the documentation that need to be reviewed and updated for each release.
Cross team testingFor user facing features that go into the release we'd like to ensure they can actually be used by Flink users. To achieve this the release managers ensure that an issue for cross team testing is created in the Apache Flink Jira. This can and should be picked up by other community members to verify the functionality and usability of the feature.
Unless the pages have not been updated before, please create a JIRA ticket and mark it as release blocker.Setup environment variablesSet up a few environment variables to simplify commands that follow. (We use
Review Release Notes in JIRAJIRA automatically generates Release Notes based on the Open the release notes from the version status page by choosing the release underway and clicking Release Notes. You should verify that the issues listed automatically by JIRA are appropriate to appear in the Release Notes. Specifically, issues should:
Adjust any of the above properties to the improve clarity and presentation of the Release Notes. Ensure that the JIRA release notes are also included in the release notes of the documentation (see section "Review and update documentation"). Content of Release Notes field from JIRA ticketsTo get the list of "release notes" field from JIRA, you can ran the following script using JIRA REST API (notes the maxResults limits the number of entries):
Verify Java and Maven VersionAll of the following steps require to use Maven 3.28.5 6 and Java 8. Modify your PATH environment variable accordingly if needed.
Clone flink into a fresh workspaceCreate a new directory for this release and clone the Flink repo from github to ensure you have a clean workspace. This step is optional. Verify that a Release Build WorksRun Verify that no exclusions were erroneously added to the japicmp plugin that break compatibility guaranteesCheck the exclusions for the japicmp-maven-plugin in the root pom for exclusions that:
Any such exclusion must be properly justified, in advance. Create a release branchRelease candidates are built from a release branch. As a final step in preparation for the release, you should create the release branch, push it to the code repository (you should probably do this once the whole process is done), and update version information on the original branch. Most of the following commands have to be executed in the
The rest of this guide assumes that commands are run in the root (or tools directory) of a repository on the branch of the release version with the above environment variables set. Checklist to proceed to the next step
|
...
Expand | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The core of the release process is the build-vote-fix cycle. Each cycle produces one release candidate. The Release Manager repeats this cycle until the community approves one release candidate, which is then finalized. Build and stage Java and Python artifactsSet up a few environment variables to simplify Maven commands that follow. This identifies the release candidate being built. Start with
Now, create a release branch ( this step can not be skipped for minor releases):
Tag the release commit:
We now need to do several things: You can use We now need to do several things:
You might want to create a directory on your local machine for collecting the various source and binary releases before uploading them. Creating the binary releases is a lengthy process but you can do this on a another machine (for example, in the "cloud"). When doing this, you can skip signing the release files on the remote machine, download them to your local machine and sign them there. First, we build the source release:
Next, we stage the maven artifacts:
Review all staged artifacts in the staging repositories(https://repository.apache.org/#stagingRepositories). They should contain all relevant parts for each module, including Close the staging repository on Apache Nexus. When prompted for a description, enter “Apache Flink, version X, release candidate Y”. Then, you need to build the PyFlink wheel packages.(since 1.11)
Finally, we create the binary convenience release files:
If you want to run this step in parallel on a remote machine you have to make the release commit available there (for example by pushing to a repository). This is important: the commit inside the binary builds has to match the commit of the source builds and the tagged release commit. When building remotely, you can skip gpg signing by setting
The release manager need to make sure the PyPI project `apache-flink` and `apache-flink-libraries` has enough available space for the python artifacts. The remaining space must be larger than the size of `tools/releasing/release/python`. Login with the PyPI admin account (account info is only available to PMC members) and check the remaining space in project settings. Request an increase if there's not enough space. Note, it could take some days for PyPI to review our request. Stage source and binary releases on dist.apache.orgCopy the source release to the dev repository of
(Push the release tag) If you haven't pushed the release tag yet, here's the command:
Propose a pull request for website updatesThe final step of building the candidate is to propose a website pull request containing the following changes:
Don’t merge the PRs before finalizing the release. Checklist to proceed to the next step
You can (optionally) also do additional verification by:
|
...
Expand | |||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||
Once the release candidate has been reviewed and approved by the community, the release should be finalized. This involves the final deployment of the release candidate to the release repositories, merging of the website changes, etc. Deploy Python artifacts to PyPI (Since 1.9)Release manager should create a PyPI account and ask the PMC add this account to pyflink collaborator list with Maintainer role (The PyPI admin account info can be found here. NOTE, only visible to PMC members) to deploy the Python artifacts to PyPI. The artifacts could be uploaded using twine(https://pypi.org/project/twine/). To install twine, just run:
Download the python artifacts from dist.apache.org and upload it to pypi.org: Note: Please ensure that the version of
Download the python artifacts from dist.apache.org and upload it to pypi.org:
If upload failed or incorrect for some reason(e.g. network transmission problem), you need to delete the uploaded release package of the same version(if exists) and rename the artifact to apache-flink-${RELEASE_VERSION}.post0.tar.gz, then re-upload. Note: re-uploading to pypi.org must be avoided as much as possible because it will cause some irreparable problems. If that happens, users cannot install the apache-flink package by explicitly specifying the package version, i.e. the following command "pip install apache-flink==${RELEASE_VERSION}" will fail. Instead they have to run "pip install apache-flink" or "pip install apache-flink==${RELEASE_VERSION}.post0" to install the apache-flink package. Deploy artifacts to Maven Central RepositoryUse the Apache Nexus repository to release the staged binary artifacts to the Maven Central repository. In the Deploy source and binary releases to dist.apache.orgCopy the source and binary releases from the
(Note: Only PMC members have access to the release repository. If you do not have access, ask on the mailing list for assistance.) Remove old release candidates from dist.apache.orgRemove the old release candidates from https://dist.apache.org/repos/dist/dev/flink using Subversion.
Git tagCreate and push a new Git tag for the released version by copying the tag for the final release candidate, as follows:
Mark the version as released in JIRAIn JIRA, inside version management, hover over the current release and a settings menu will appear. Click (Note: Only PMC members have access to the project administration. If you do not have access, ask on the mailing list for assistance.) If PRs have been merged to the release branch after the the last release candidate was tagged, make sure that the corresponding Jira tickets have the correct Fix Version set. Publish the Dockerfiles for the new releaseNote: the official Dockerfiles fetch the binary distribution of the target Flink version from an Apache mirror. After publishing the binary release artifacts, mirrors can take some hours to start serving the new artifacts, so you may want to wait to do this step until you are ready to continue with the "Promote the release" steps below. Follow the instructions in the flink-docker repo to build the new Dockerfiles and send an updated manifest to Docker Hub so the new images are built and published.
Checklist to proceed to the next step
|
...
Expand | ||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||
Once the release has been finalized, the last step of the process is to promote the release within the project and beyond. Please wait for 24h after finalizing the release in accordance with the ASF release policy. Update japicmp configurationUpdate the japicmp reference version and wipe exclusions / enable API compatibility checks for For a new major release (x.y.0), run the same command also on the master branch for updating the japicmp reference version and removing out-dated exclusions in the japicmp configuration. Make sure that all Maven artifacts are already pushed to Maven Central. Otherwise, there's a risk that CI fails due to missing reference artifacts.
Merge website pull requestMerge the website pull request to list the release. Make sure to regenerate the website as well, as it isn't build automatically. Remove outdated versionsdist.apache.orgFor a new major release remove all release files older than 2 versions, e.g., when releasing 1.7, remove all releases <= 1.5. For a new bugfix version remove all release files for previous bugfix releases in the same series, e.g., when releasing 1.7.1, remove the 1.7.0 release.
CIDisable the cron job for the now-unsupported version from (tools/azure-pipelines/build-apache-repo.yml) in the respective branch. Apache mailing listsAnnounce on the dev@ mailing list that the release has been finished. Announce on the release on the user@ mailing list, listing major improvements and contributions. Announce the release on the announce@apache.org mailing list.
RecordkeepingUse reporter.apache.org to seed the information about the release into future project reports. (Note: Only PMC members have access report releases. If you do not have access, ask on the mailing list for assistance.) Flink blogMajor or otherwise important releases should have a blog post. Write one if needed for this particular release. Minor releases that don’t introduce new major functionality don’t necessarily need to be blogged (see flink-web PR #581 for Flink 1.15.3 as an example for a minor release blog post). Please make sure that the release notes of the documentation (see section "Review and update documentation") are linked from the blog post of a major release.
Social mediaTweet, post on Facebook, LinkedIn, and other platforms. Ask other contributors to do the same. Flink Release Wiki pageAdd a summary of things that went well or that went not so well during the release process. This can include feedback from contributors but also more generic things like the release have taken longer than initially anticipated (and why) to give a bit of context to the release process. End of Life for Unsupported versionsFor major versions only. As per our support policy for old Flink versions when we release a new 1.x version we should start a discussion thread to end support for old versions.
Checklist to declare the process completed
|
...