Versions Compared

Old Version 26

changes.mady.by.user Ferenc Gerlits

Saved on

compared with

New Version Current

changes.mady.by.user Martin Zink

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

provided in the vote email thread

Note:  This document is a work in progress and heavily derived from the NiFi Release guide generated by the community.  The idea is that with processes fully established for each project we can consolidate at a later point.

...

Substitutions used in tasks and email templates:
ReferenceExample valueDescription
${BRANCH}mainthe development branch on which the release is based
${MINIFI_VERSION}0.10.0the version currently in development on the release branch, to be released



${
NEXT_VERSION}0.11.0the future version for development on the release branch${
JIRA_TICKET}MINIFICPP-1234the JIRA ticket created by the release manager for the release tasks
${RC}1the Release Candidate index start at 1 for the first release candidate
${RC_TAG_COMMIT_ID}
the commit ID of the RC tag created during the Maven release process
${RM_USERID}johndoethe Apache account ID of Release Manager
${RELEASE_TAG}rel/minificpp-0.10.0the Git repository tag for the source code as released
${VOTE_THREAD_URL}[0.10.0 vote thread][0100-rc2-vote]the URL for the Apache Pony Mail archive of the release vote thread


To be practical but avoid confusion with future release details, these example values reflect the release details of MiNiFi C++ 0.10.0 RC2.

...

  1. Create a JIRA ticket for the release tasks for version ${MINIFI_VERSION}.
    _The resulting JIRA ticket number is referred to as  ${JIRA_TICKET}  in this guide.
  2. Create the next version in JIRA, if it doesn't already exist, so work can continue towards that release.  Versions can be managed at https://issues.apache.org/jira/projects/MINIFICPP?selectedItem=com.atlassian.jira.jira-projects-plugin%3Arelease-page
  3. Create meaningful release notes for this version if not already created.  Enter them here https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotesAdd them to the Release Notes - MiNiFi (C++) on page on the MiNiFi wiki.
    1. May be useful to add links to processor documentation so it is easier to diff releases. 
  4. Create a new branch off ${BRANCH} named after the JIRA ticket.
    $ git checkout -b ${JIRA_TICKET}-RC${RC} ${BRANCH}
  5. Verify that you have the needed dependencies to build and run MiNiFi
  6. Ensure the the full application builds, all tests work, and source passes linting by executing the following:
    TODO/DISCUSS:  Since we don't have a dependency management framework we will include only base extensions.
    $ mkdir build && cd build &&  cmake cmake -DENABLE_COAP=ON -DSKIP_TESTS= -DUSE_SHARED_LIBS=ON -DPORTABLE=ON -DBUILD_ROCKSDB=ON -DEXCLUDE_BOOST=ON -DBUILD_IDENTIFIER= -DCMAKE_BUILD_TYPE=Release -DFAIL_ON_WARNINGS= .. && make package_source && make package && make test && make linter && make docker

  7. Startup and test the application with from the build folder:
    $ tar xvzf nifi-minifi-cpp-${MINIFI_VERSION}-bin.tar.gz && ./nifi-minifi-cpp-${MINIFI_VERSION}/bin/minifi.sh start

  8. Evaluate and ensure the appropriate license headers are present on all source files.

  9. Ensure LICENSE and NOTICE files are complete and accurate. (Developers should always be keeping these up to date as they go along adding source and modifying dependencies to keep this burden manageable.)

  10. If the validated artifacts all look good then create a tag and push the branch to the ASF repository.

    $ git tag -s "minifi-cpp-${MINIFI_VERSION}-RC${RC}" -m "${JIRA_TICKET} RC${RC} release candidate of NiFi MiNiFi C++ ${MINIFI_VERSION}" ${JIRA_TICKET}-RC${RC}
    $ git push --tag asf ${JIRA_TICKET}-RC${RC}
Generate convenience binaries
TODO/DISCUSS:  Since we don't have a dependency management framework we will include only base extensions.
  • on OS X:
    $ make package
  • on Linux:
    • $ make centos
    • $ make debian
    • $ make fedora
    • $ make u20    # for Ubuntu focal

Generate the convenience binary:
$ make centos
$ mv nifi-minifi-cpp-${MINIFI_VERSION}-bin-{centos,linux}.tar.gz


Create the signature and hashes for the source release and convenience binary files

...

:


Code Block
for artifact in $(find . -type f -name '*.tar.gz')
; 
do
     echo $artifact;
    echo  “ Generating ASCII armored GPG signature ”;
     gpg -a -b 

do
  echo $artifact;
  echo  " Generating ASCII armored GPG signature "
  gpg -a -b --digest-algo=SHA512 ${artifact};
     echo  “ Generating sha1 sum hash ”;
    gsha1sum  ${artifact} |  cut -d" " -f1 > ${artifact}.sha1
     echo  “ Generating sha25 sum hash ”;
     gsha256sum  ${artifact} |  cut -d" " -f1 > 

  echo  " Generating sha512 sum hash "
  sha512sum  ${artifact} |  cut -d" " -f1 > ${artifact}.sha512
  echo  " Generating sha256 sum hash "
  sha256sum  ${artifact} |  cut -d" " -f1 > ${artifact}.sha256
   

done


    1. ASCII armored GPG signatures (--digest-algo=SHA512 select the SHA512 hash algorithm). Configure GPG to always prefer stronger hashes.
      $ gpg -a -b --digest-algo=SHA512 nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz    # produces nifi-minifi-cpp${MINIFI_VERSION}-source.tar.gz.asc
      $ gpg -a -b --digest-algo=SHA512 nifi-minifi-cpp-${MINIFI_VERSION}-bin.tar.gz          # produces nifi-minifi-cpp-${MINIFI_VERSION}-bin.tar.gz.asc
      $ gpg -a -b --digest-algo=SHA512 nifi-minifi-cpp-${MINIFI_VERSION}-bin.tar.gz          # produces nifi-minifi-cpp-${MINIFI_VERSION}-bin.tar.gz.asc
    2. Generate SHA1 SHA256 hash summaries.
      $ sha1sum shasum -a 256 nifi-${MINIFI_VERSION}-source-release.zip | cut -d" " -f1 >  nifi-${RELEASAEMINIFI_VERSION}-source-release.zip.sha1sha256
      $ sha1sum shasum -a 256 nifi-${MINIFI_VERSION}-bin.tar.gz | cut -d" " -f1 >  nifi-${RELEASAEMINIFI_VERSION}-bin.tar.gz.sha1sha256
      $ sha1sum shasum -a 256 nifi-${MINIFI_VERSION}-bin.zip | cut -d" " -f1 >  nifi-${RELEASAEMINIFI_VERSION}-bin.zip.sha1sha256
    3. Generate SHA256 hash SHA512 hash summaries.
      $ shasum -a 256 sha512sum nifi-${MINIFI_VERSION}-source-release.zip | cut -d" " -f1 >  nifi-${MINIFI_VERSIONRELEASAE}-source-release.zip.sha256sha512
      $ shasum -a 256 sha512sum nifi-${MINIFI_VERSION}-bin.tar.gz | cut -d" " -f1 >  nifi-${MINIFI_VERSIONRELEASAE}-bin.tar.gz.sha256sha512
      $ shasum -a 256 sha512sum nifi-${MINIFI_VERSION}-bin.zip | cut -d" " -f1 >  nifi-${MINIFI_VERSIONRELEASAE}-bin.zip.sha256sha512


For reviewing

...

the release candidate,

...

upload the source

...

and the convenience

...

binary tarballs, along with their hashes and signatures

...

, to the ASF distribution repo using subversion:

Code Block
svn co https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp
cd nifi-minifi-cpp
mkdir ${MINIFI_VERSION}
cp nifi-minifi-cpp-* ${MINIFI_VERSION}/
svn add ${MINIFI_VERSION}

...


svn commit

(you need to be a committer to do this, and svn will prompt you for your Apache password).

Step 4. Step 4. Error recovery (RM)

If anything isn't correct about the staged artifacts you can drop the staged repo from repository.apache.org and delete the local tag in git. If you also delete the local branch and clear your local maven repository under org/apache/nifi then it is as if the release never happened. Before doing that though try to figure out what went wrong so the Release Guide can be updated or corrected if necessary.

...

  1. RM sends a vote request email to the NiFi Developers Mailing List.

  2. RM sends the following helper email to the NiFi Developers Mailing List.

    1. TO: dev@nifi.apache.org
    2. FROM: ${RM_USERID}@apache.org
    3. SUBJECT:  Apache NiFi MiNiFi C++ ${MINIFI_VERSION} RC${RC} Release Helper Guide

      Hello Apache NiFi community,

      Please find the associated guidance to help those interested in validating/verifying the release so they can vote.

      # Download latest KEYS file:
      https://dist.apache.org/repos/dist/release/nifi/KEYS

      # Import keys file:
      gpg --import KEYS

      # Pull down nifi-minifi-cpp-${MINIFI_VERSION} source release artifacts for review:

      wget https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp/${MINIFI_VERSION}/nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz
      wget https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp/${MINIFI_VERSION}/nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz.asc
      wget https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp/${MINIFI_VERSION}/nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz.sha1 sha256
      wget https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp/${MINIFI_VERSION}/nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz.sha256 sha512

      # Verify the signature
      gpg --verify nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz.asc

      # Verify the hashes (sha1sha256, sha256sha512) match the source and what was provided in the vote email thread
      sha1sum sha256sum nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz
      sha256sum sha512sum nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz

      # Extract nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz
      tar xvzf       nifi-minifi-cpp-${MINIFI_VERSION}-source.tar.gz

      # Verify the build works including tests and linter checks
      cd nifi-minifi-cpp-${MINIFI_VERSION}-source
      mkdir build && cd build && cmake .. && make package && make test && make linter
      # or:
      # ./bootstrap.sh && cd build && make package && make test && make linter

      # On Windows:
      # Install dependencies as documented: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=139627733
      # cd nifi-minifi-cpp-${MINIFI_VERSION}-source
      # win_build_vs.bat build /P

      # Verify the contents contain a good README, NOTICE, and LICENSE.

      # Verify the git commit ID is correct

      # Verify the RC was branched off the correct git commit ID

      # Look at the resulting convenience binary as found in build/nifi-minifi-cpp-${MINIFI_VERSION}-bin.tar.gz

      # Make sure the README, NOTICE, and LICENSE are present and correct

      # Run the resulting convenience binary and make sure it works as expected

      # Send a response to the vote thread indicating a +1, 0, -1 based on your findings.

      Thank you for your time and effort to validate the release!
  3. Developers in the community review the release candiate and reply to the vote email with their vote.

  4. After 72 hours if

    • at least 3 binding (PMC members) cast +1 votes, and
    • the positive binding votes out number any negative binding votes
  5. the vote passes and the release candidate is officially released. If the vote does not pass, corrections are made on the release branch and a new release candidate is put forward for a new vote.
  6. RM sends vote result email.

    • TO: dev@nifi.apache.org
    • FROM: ${RM_USERID}@apache.org
    • SUBJECT:  [RESULT][VOTE] Release Apache NiFi MiNiFi C++ ${NIFI_VERSION}

      Apache NiFi Community,

      I am pleased to announce that the ${MINIFI_VERSION} release of Apache NiFi MiNiFi C++ passes with
        X +1 (binding) votes
        Y -+1 (non-binding) votes
        0 0 votes
        0 -1 votes

      Thanks to all who helped make this release possible.

      Here is the PMC vote thread: ${VOTE_THREAD_URL}

...

  1. Move convenience binaries and related artifacts from dist/dev to dist/release: (a member of the PMC needs to run this)

    $ svn move -m " ${JIRA_TICKET}"   https://dist.apache.org/repos/dist/dev/nifi/nifi-minifi-cpp/ ${MINIFI_VERSION} https://dist.apache.org/repos/dist/release/nifi/nifi-minifi-cpp/ ${MINIFI_VERSION}

  2. Update the MiNiFi website to point to the new download(s) by creating a pull request on https://github.com/apache/nifi-site and following the instructions in README.md.  Remove older release artifacts from download page (leave the current release and the previous one). For the release just previous to this new one change the links to point to the archive location. See current page as an example of the needed URL changes. In addition to updating the download page as described In addition to updating the download page as described delete artifacts other than the current/new release from the dist/nifi SVN storage. They are already in the archive location so no need to do anything else.

  3. Update the NiFi Web Page to indicate NEWS of the release as appropriate

  4. Create a proper signed tag of the released codebase based on the RC Tag craeted created during the Maven release process.

    $ git tag -s rel/minifi-cpp-${MINIFI_VERSION} -m "${JIRA_TICKET} signed release tag for approved release of NiFi MiNiFi C++ ${MINIFI_VERSION}" ${RC_TAG_COMMIT_ID}
    For instructions on setting up to sign your tag see  here .

  5. Push the release tag to the official ASF repository.

    $ git push asf rel/minifi-cpp-${MINIFI_VERSION}

  6. Publish the convenience binaries to docker hub (TODO: extend this guide with details)

  7. Update the release notes with the final date of the release.

  8. After the release has been complete for 24 hours send the release announcement.

...

    • be the template included below extended with the newly released features.

...



    • Hello

      The Apache NiFi team would like to announce the release of Apache NiFi MiNiFi C++ ${MINIFI_VERSION}.

...


    • Note:  Incorporate highlights about this release to give viewers a sufficient understanding about why they should care or what is now possible.
      New features in this release:

      < Insert list of new features here >

      MiNiFi—a subproject of Apache NiFi—is a complementary data collection approach that supplements the core tenets of NiFi in dataflow management, focusing on the collection of data at the source of its creation.

      Specific goals for the initial thrust of the MiNiFi effort comprise:

      • Small size and low resource consumption
      • Central management of agents
      • Generation of data provenance (full chain of custody of information)
      • Integration with NiFi for follow-on dataflow management

...

...

...

Release Supporting and Helper Resources

Sample NiFi and MiNiFi Configuration to transmit data from MiNiFi to NiFi via Site to Site

The following archive contains a flow.xml.gz to configure a flow with a known input port UUID to communicate with a configuration for MiNiFI C++ with the included flow.yml. 

minifi-sample-config.tgz

Hashes

  • sha1sum: 5b3797924eee1a59421ff216c542cb28c3564bfe

  • sha256sum: 9323165a2086053f8d1ad5478e2b7cc97f01fbc38ba133afe77badba1a446833

Signature

Signed with the key at http://people.apache.org/keys/committer/aldrin.asc

...