Apache MiNiFi Release Guidelines

The purpose of this document is to capture and describe the steps involved in producing an official release of Apache NiFi MiNiFI Java. It is written specifically to someone acting in the capacity of a Release Manager (RM).

Background Material

The objective

Our aim is to produce an official Apache release.
The following is a list of the sorts of things that will be validated and are the basics to check when evaluating a release for a vote.

What to validate and how to Validate a release

The flow of a release (an outline)

Variable reference substitutions


Throughout this guide, references must be made to names and values that will vary from release to release. For clarity those variable values have been written like Bash variable references. When a term like "/tmp/src/minifi-${MINIFI_VERSION}" is seen in an instruction or email template it should be replaced with "/tmp/src/minifi-0.5.0" when working the release of "Apache NiFi MiNiFi 0.5.0".

The mechanics of the release

Prepare your environment

Follow the steps outlined in the Quickstart Guide

At this point you're on the latest 'master' branch and are able to build the entire application


Ensure your Maven settings.xml has been updated as shown below. There are other ways to ensure your PGP key is available for signing as well

...
        <profile>
           <id>signed_release</id>
           <properties>
               <mavenExecutorId>forked-path</mavenExecutorId>
               <gpg.keyname>YOUR GPG KEY ID HERE</gpg.keyname>
               <gpg.passphrase>YOUR GPG PASSPHRASE HERE</gpg.passphrase>
           </properties>
       </profile>
       ...
       <servers>
          <server>
              <id>repository.apache.org</id>
              <username>YOUR USER NAME HERE</username>
              <password>YOUR MAVEN ENCRYPTED PASSWORD HERE</password>
          </server>
       </servers>
       ...

        ...
        <profile>
           <id>signed_release</id>
           <properties>
               <mavenExecutorId>forked-path</mavenExecutorId>
               <gpg.keyname>YOUR GPG KEY ID HERE</gpg.keyname>
               <gpg.passphrase>YOUR GPG PASSPHRASE HERE</gpg.passphrase>
           </properties>
       </profile>
       ...
       <servers>
          <server>
              <id>repository.apache.org</id>
              <username>YOUR USER NAME HERE</username>
              <password>YOUR MAVEN ENCRYPTED PASSWORD HERE</password>
          </server>
       </servers>
       ...

Ensure the the full application build and tests all work by executing mvn -T 2.5C clean install for a parallel build. Once that completes you can startup and test the application by cd nifi-assembly/target then run "bin/minifi.sh start" in the MiNiFi build.

Evaluate and ensure the appropriate license headers are present on all source files. Ensure LICENSE and NOTICE files are complete and accurate.
Developers should always be keeping these up to date as they go along adding source and modifying dependencies to keep this burden manageable.
This command mvn install -Pcontrib-check should be run as well to help validate. If that doesn't complete cleanly it must be addressed.

Now its time to have maven prepare the release so execute mvn release:prepare -Psigned_release -DscmCommentPrefix="NIFI-270-RC1 " -Darguments="-DskipTests". Maven will ask:

What is the release version for "Apache NiFi"? (org.apache.nifi:nifi) 0.0.1: :

Just hit enter to accept the default.

Maven will then ask:

What is SCM release tag or label for "Apache NiFi"? (org.apache.nifi:nifi) nifi-0.0.1: :

Enter nifi-0.0.1-RC1 or whatever the appropriate release candidate (RC) number is. Maven will then ask:

What is the new development version for "Apache NiFi"? (org.apache.nifi:nifi) 0.0.2-SNAPSHOT: :

Just hit enter to accept the default.

Now that preparation went perfectly it is time to perform the release and deploy artifacts to staging. To do that execute

mvn release:perform -Psigned_release -DscmCommentPrefix="NIFI-270-RC1 " -Darguments="-DskipTests"

That will complete successfully and this means the artifacts have been released to the Apache Nexus staging repository. You will see something like

[INFO] * Closing staging repository with ID "orgapachenifi-1011".

So if you browse to https://repository.apache.org/#stagingRepositories login with your Apache committer credentials and you should see orgapachenifi-1011. If you click on that you can inspect the various staged artifacts.

Validate that all the various aspects of the staged artifacts appear correct

If all looks good then push the branch to origin git push origin MINIFI-458

For reviewing of the release candidate - The sources, hashes, signature, and the convenience binary, its hashes, and signature should be placed here:

- https://dist.apache.org/repos/dist/dev/minifi-0.5.0/


For each convenience binary

- Generate ascii armored detached signature by running `gpg -a -b --digest-algo=SHA512 nifi-0.0.1-bin.tar.gz`
- Generate sha1 hash summary by running `sha1sum nifi-0.0.1-bin.tar.gz | awk '{ printf substr($0,0,40)}' >  nifi-0.0.1-bin.tar.gz.sha1`
- Upload the bin, asc, sha1, md5 for each binary convenience build to the same location as the source release


If anything isn't correct about the staged artifacts you can drop the staged repo from repository.apache.org and delete the local tag in git. If you also delete the local branch and clear your local maven repository under org/apache/nifi then it is as if the release never happened. Before doing that though try to figure out what went wrong. So as described here you see that you can pretty easily test the release process until you get it right. The mvn versions:set and mvn versions:commit commands can come in handy to help do this so you can set versions to something clearly release test related.

Now it's time to initiate a vote within the NiFi community. Send the vote request to dev@nifi.apache.org with a subject of [VOTE] Release Apache NiFi 0.0.1. The following template can be used:

Hello
I am pleased to be calling this vote for the source release of Apache NiFi
nifi-0.0.1.

The source zip, including signatures, digests, etc. can be found at:
https://repository.apache.org/content/repositories/orgapachenifi-1011

The Git tag is nifi-0.0.1-RC1
The Git commit ID is 72abf18c2e045e9ef404050e2bffc9cef67d2558
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=72abf18c2e045e9ef404050e2bffc9cef67d2558

Checksums of nifi-0.0.1-source-release.zip:
MD5: 5a580756a17b0573efa3070c70585698
SHA1: a79ff8fd0d2f81523b675e4c69a7656160ff1214

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/joewitt.asc

KEYS file available here:
https://dist.apache.org/repos/dist/release/nifi/KEYS

8 issues were closed/resolved for this release:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12329307

Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.0.1

The vote will be open for 72 hours. 
Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build from source, and test.  The please vote:

[ ] +1 Release this package as nifi-0.0.1
[ ] +0 no opinion
[ ] -1 Do not release this package because because...


A release vote is majority rule. So wait 72 hours and see if there are at least 3 binding (PMC members) +1 votes and no more negative votes than positive. If the vote passes then send a vote result email. Send the email to dev@nifi.apache.org with a subject of[RESULT][VOTE] Release Apache NiFi 0.0.1. Use a template such as:

Hello

The release passes with

X +1 (binding) votes
Y -1 (binding) votes

Thanks to all who helped make this release possible.

Here is the PMC vote thread: [INSERT URL OF PMC Vote Thread]


Now all the voting is done and the release is good to go.

Here are the steps of the release once the release is approved:

  1. Move convenience binaries and related artifacts from dist/dev to dist/release:
    svn move -m "NIFI-1122" https://dist.apache.org/repos/dist/dev/nifi/nifi-0.0.1 https://dist.apache.org/repos/dist/release/nifi/0.0.1 

  2. In repository.apache.org go to the staging repository and select release and follow instructions on the site.

  3. Merge the release branch into master

  4. Update the NiFi website to point to the new download(s). Remove older release artifacts from download page (leave the current release and the previous one). For the release just previous to this new one change the links to point to the archive location. See current page as an example of the needed URL changes. In addition to updating the download page as described delete artifacts other than the current/new release from the dist/nifi SVN storage. They are already in the archive location so no need to do anything else.

  5. Update the Migration Guide on the Wiki.

  6. Update the NiFi Web Page to indicate NEWS of the release as appropriate

  7. From a nifi.tar.gz collect the docs/html/* files and svn commit them to https://svn.apache.org/repos/asf/nifi/site/trunk/docs/nifi-docs/html/

  8. From a nifi.tar.gz collect the nifi-framework-nar.nar/META-INF/bundled-dependencies/nifi-web-api.war/docs/rest-api/* files and svn commit them to https://svn.apache.org/repos/asf/nifi/site/trunk/docs/nifi-docs/rest-api/

  9. Run an instance of nifi

  10. Copy nifi/work/docs/components/* and svn commit to https://svn.apache.org/repos/asf/nifi/site/trunk/docs/nifi-docs/components/

  11. wget http://localhost:8080/nifi-docs/documentation and svn commit to https://svn.apache.org/repos/asf/nifi/site/trunk/docs/nifi-docs/index.html

  12. In Jira mark the release version as 'Released' and 'Archived' through 'version' management in the 'administration' console.

  13. Create a proper signed tag of the released codebase. If the approved RC tag was 'nifi-0.0.1-RC1' then create a signed release tag of 'rel/nifi-0.0.1'. For instructions on setting up to sign your tag see here. To create a signed release tag enter git tag -s rel/nifi-0.0.1 -m "NIFI-XYZ Signed release tag for approved release of nifi 0.0.1" COMMIT-ID-OF-RC-TAG

  14. Wait 24 hours then send release announcement.

SUBJECT:  [ANNOUNCE] Apache NiFi 0.0.1 release
BODY:
Hello

The Apache NiFi team would like to announce the release of Apache NiFi 0.0.1.

Apache NiFi is an easy to use, powerful, and reliable system to process and distribute data.  Apache NiFi was made for dataflow.  It supports highly configurable directed graphs of data routing, transformation, and system mediation logic.

More details on Apache NiFi can be found here:
http://nifi.apache.org/

The release artifacts can be downloaded from here:
http://nifi.apache.org/download.html

Maven artifacts have been made available here:
https://repository.apache.org/content/repositories/releases/org/apache/nifi/

Issues closed/resolved for this list can be found here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12329373

Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.0.1

Thank you
The Apache NiFi team

 

Release Supporting and Helper Resources

Sample NiFi and MiNiFi Configuration to transmit data from MiNiFi to NiFi via Site to Site

The following archive contains a flow.xml.gz to configure a flow with a known input port UUID to communicate with a configuration for MiNiFI C++ with the included flow.yml. 

minifi-sample-config.tgz

Hashes

Signature

Signed with the key at http://people.apache.org/keys/committer/aldrin.asc

minifi-sample-config.tgz.gpg