Apache MiNiFi Release Guidelines
The purpose of this document is to capture and describe the steps involved in producing an official release of Apache NiFi MiNiFI for both Java and C++ versions. It is written specifically to someone acting in the capacity of a Release Manager (RM).
Table of Contents |
---|
Background Material
These documents are necessary for all committers to be familiar with
These documents are necessary for someone acting as the RM
These documents are helpful for general environmental setup to perform releases
The objective
Our aim is to produce an official Apache release.
The following is a list of the sorts of things that will be validated and are the basics to check when evaluating a release for a vote.
What to validate and how to Validate a release
- Are LICENSE and NOTICE file present in the source root and complete?
- Specifically look in the *-sources.zip artifact and ensure these items are present at the root of the archive.
- Evaluate the sources and dependencies. Does the overall LICENSE and NOTICE appear correct? Do all licenses fit within the ASF approved licenses?
- Here is an example path to a sources artifact:
- Is there a README available that explains how to build the application and to execute it?
- Look in the *-sources.zip artifact root for the readme.
- Are the signatures and hashes correct for the source release?
- Validate the hashes of the sources artifact do in fact match:
- Validate the signature of the source artifact. Here is an example path:
- Do all sources have necessary headers?
- Unzip the sources file into a directory and execute
mvn install -Pcontrib-check
- Unzip the sources file into a directory and execute
- Are there no unexpected binary files in the release?
- The only thing we'd expect would be potentially test resources files.
- Does the app (if appropriate) execute and function as expected?
The flow of a release (an outline)
- The community is contributing to a series of JIRA tickets assigned to the next release
- The number of tickets open/remaining for that next release approaches zero
- A member of the community suggests a release and initiates a discussion
- Someone volunteers to be an RM for the release (can be a committer but apache guides indicate preference is a PMC member)
- A release candidate is put together and a vote sent to the team.
- If the NiFi community rejects the vote the issues noted are resolved and another RC is generated
- If the NiFi community accepts the vote then the release is 'releasable' and can be placed into the appropriate 'dist' location, maven artifacts released from staging.
The mechanics of the release
Prepare your environment
Follow the steps outlined in the Quickstart Guide
At this point you're on the latest 'master' branch and are able to build the entire application
...
Create the next version in JIRA if necessary so work can continue towards that release.
Create meaningful release notes for this version if not already created. Enter them here
Create new branch off 'master' named after the JIRA ticket. Here we'll use a branch off of 'master' with git checkout -b NIFI-270-RC1
Verify that Maven has sufficient heap space to perform the build tasks. Some plugins and parts of the build consumes a surprisingly large amount of space. These settings have been shown to work MAVEN_OPTS="-Xms1024m -Xmx3076m -XX:MaxPermSize=256m"
...
Variable reference substitutions
Throughout this guide, references must be made to names and values that will vary from release to release. For clarity those variable values have been written like Bash variable references. When a term like "/tmp/src/minifi-${MINIFI_VERSION}" is seen in an instruction or email template it should be replaced with "/tmp/src/minifi-0.5.0" when working the release of "Apache NiFi MiNiFi 0.5.0".
- Subtitutions used in tasks and email templates
Reference Example value Description
========= ============== ===========
${BRANCH} 0.5.0-SNAPSHOT the development branch on which the release is based.
${MINIFI_VERSION} 0.5.0 the version currently in development on the release branch.
${NEXT_VERSION} 0.6.0-SNAPSHOT the future version for development on the release branch.
${JIRA_TICKET} MINIFI-458 the JIRA ticket created by the release manager for the release tasks.
${RC} 2 the Release Candidate index start at 1 for the first release candidate.
${RC_TAG_COMMIT_ID} the commit ID of the RC tag created during the Maven release process.
${RM_USERID} johndoe the Apache account ID of Release Manager.
${RELEASE_TAG} rel/minifi-0.5.0 the Git repository tag for the source code as released.
${VOTE_THREAD_URL} [0.5.0 vote thread][050-rc2-vote] the URL for the Apache Pony Mail archive of the release vote thread.
To be practical but avoid confusion with future release details, these example values reflect the previous release MiNiFi 0.5.0 RC2 release details.
The mechanics of the release
Prepare your environment
Follow the steps outlined in the Quickstart Guide
At this point you're on the latest 'master' branch and are able to build the entire application
- Create a JIRA ticket for the release tasks and use that ticket number for the commit messages. For example we'll consider MINIFI-458 as our ticket. Also have in mind the release version you are planning for. For example we'll consider '0.5.0'.
- Create the next version in JIRA if necessary so work can continue towards that release.
- Create meaningful release notes for this version if not already created. Enter them here
- Create new branch off 'master' named after the JIRA ticket. Here we'll use a branch off of 'master' with
git checkout -b MINIFI-458-RC1
Ensure your Maven settings.xml has been updated as shown below. There are other ways to ensure your PGP key is available for signing as well
Code Block |
---|
...
<profile>
<id>signed_release</id>
<properties>
<mavenExecutorId>forked-path</mavenExecutorId>
<gpg.keyname>YOUR GPG KEY ID HERE</gpg.keyname>
<gpg.passphrase>YOUR GPG PASSPHRASE HERE</gpg.passphrase>
</properties>
</profile>
...
<servers>
<server>
<id>repository.apache.org</id>
<username>YOUR USER NAME HERE</username>
<password>YOUR MAVEN ENCRYPTED PASSWORD HERE</password>
</server>
</servers>
...
|
...
<profile>
<id>signed_release</id>
<properties>
<mavenExecutorId>forked-path</mavenExecutorId>
<gpg.keyname>YOUR GPG KEY ID HERE</gpg.keyname>
<gpg.passphrase>YOUR GPG PASSPHRASE HERE</gpg.passphrase>
</properties>
</profile>
...
<servers>
<server>
<id>repository.apache.org</id>
<username>YOUR USER NAME HERE</username>
<password>YOUR MAVEN ENCRYPTED PASSWORD HERE</password>
</server>
</servers>
...
Ensure the the full application build and tests all work by executing mvn -T 2.5C clean install
for a parallel build. Once that completes you can startup and test the application by cd nifi-assembly/target
then run bin/nifi.sh start
in the nifi build. The application should be up and running in a few seconds at http://localhost:8080/nifi
run "bin/minifi.sh start"
in the MiNiFi build.
Evaluate and ensure the appropriate license headers are present on all source files. Ensure LICENSE and NOTICE files are complete and accurate.
Developers should always be keeping these up to date as they go along adding source and modifying dependencies to keep this burden manageable.
This command mvn install -Pcontrib-check
should be run as well to help validate. If that doesn't complete cleanly it must be addressed.
...
If all looks good then push the branch to origin git push origin NIFIMINIFI-270458
For reviewing of the release candidate - The sources, hashes, signature, and the convenience binary, its hashes, and signature should be placed here:
- https://dist.apache.org/repos/dist/dev/nifiminifi-0.5.0.1/
For each convenience binary
- Generate ascii armored detached signature by running `gpg -a -b --digest-algo=SHA512 nifi-0.0.1-bin.tar.gz`
- Generate md5ascii armored hashdetached summarysignature by running `md5sum`gpg nifi-0.0.1-bin.tar.gz | awk '{ printf substr($0,0,32)}' > -a -b --digest-algo=SHA512 nifi-0.0.1-bin.tar.gz.md5`gz`
- Generate sha1 hash summary by running `sha1sum nifi-0.0.1-bin.tar.gz | awk '{ printf substr($0,0,40)}' > nifi-0.0.1-bin.tar.gz.sha1`
- Upload the bin, asc, sha1, md5 for each binary convenience build to the same location as the source release
...
- Move convenience binaries and related artifacts from dist/dev to dist/release:
svn move -m "NIFI-1122" https://dist.apache.org/repos/dist/dev/nifi/nifi-0.0.1 https://dist.apache.org/repos/dist/release/nifi/0.0.1
In repository.apache.org go to the staging repository and select
release
and follow instructions on the site.Merge the release branch into master
Update the NiFi website to point to the new download(s). Remove older release artifacts from download page (leave the current release and the previous one). For the release just previous to this new one change the links to point to the archive location. See current page as an example of the needed URL changes. In addition to updating the download page as described delete artifacts other than the current/new release from the dist/nifi SVN storage. They are already in the archive location so no need to do anything else.
Update the Migration Guide on the Wiki.
Update the NiFi Web Page to indicate NEWS of the release as appropriate
From a nifi.tar.gz collect the docs/html/* files and svn commit them to https://svn.apache.org/repos/asf/nifi/site/trunk/docs/nifi-docs/html/
From a nifi.tar.gz collect the nifi-framework-nar.nar/META-INF/bundled-dependencies/nifi-web-api.war/docs/rest-api/* files and svn commit them to https://svn.apache.org/repos/asf/nifi/site/trunk/docs/nifi-docs/rest-api/
Run an instance of nifi
Copy nifi/work/docs/components/* and svn commit to https://svn.apache.org/repos/asf/nifi/site/trunk/docs/nifi-docs/components/
wget http://localhost:8080/nifi-docs/documentation and svn commit to https://svn.apache.org/repos/asf/nifi/site/trunk/docs/nifi-docs/index.html
In Jira mark the release version as 'Released' and 'Archived' through 'version' management in the 'administration' console.
Create a proper signed tag of the released codebase. If the approved RC tag was 'nifi-0.0.1-RC1' then create a signed release tag of 'rel/nifi-0.0.1'. For instructions on setting up to sign your tag see here. To create a signed release tag enter
git tag -s rel/nifi-0.0.1 -m "NIFI-XYZ Signed release tag for approved release of nifi 0.0.1" COMMIT-ID-OF-RC-TAG
Wait 24 hours then send release announcement.
- See here for an understanding of why you need to wait 24 hours
- Then create an announcement like the one shown below addressed to 'announce@apache.org, dev@nifi..apache.org' with a reply-to of 'dev@nifi.apache.org'.
- The email has to be sent from an apache.org email address and should be by the release manager of the build.
SUBJECT: [ANNOUNCE] Apache NiFi 0.0.1 release
BODY:
Hello
The Apache NiFi team would like to announce the release of Apache NiFi 0.0.1.
Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data. Apache NiFi was made for dataflow. It supports highly configurable directed graphs of data routing, transformation, and system mediation logic.
More details on Apache NiFi can be found here:
http://nifi.apache.org/
The release artifacts can be downloaded from here:
http://nifi.apache.org/download.html
Maven artifacts have been made available here:
https://repository.apache.org/content/repositories/releases/org/apache/nifi/
Issues closed/resolved for this list can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12329373
Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.0.1
Thank you
The Apache NiFi team
Release Supporting and Helper Resources
Sample NiFi and MiNiFi Configuration to transmit data from MiNiFi to NiFi via Site to Site
The following archive contains a flow.xml.gz to configure a flow with a known input port UUID to communicate with a configuration for MiNiFI C++ with the included flow.yml.
Hashes
md5sum: d38ef00f894cbcc3fde69b51ad5f8fe8 871b465492eb6cb0d9072c50b08fe4b1
sha1sum: 6de5a2d9638b869895172e428d472e164a7f2407 5b3797924eee1a59421ff216c542cb28c3564bfe
sha256sum: b346c7a453802c4925f45776bfe22ce9f1de4fe50af7b9a52d0affe5ad8cf865 9323165a2086053f8d1ad5478e2b7cc97f01fbc38ba133afe77badba1a446833
Signature
Signed with the key at http://people.apache.org/keys/committer/aldrin.asc
...