Versions Compared

Old Version 17

changes.mady.by.user Aaron Mihalik

Saved on

compared with

New Version Current

changes.mady.by.user Adina Crainiceanu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: removed references to incubating, updated repo names

Currently, this is just a collection area for my rough notes and what-not.  It will soon be a much cleaner guide.

 

Speaking of guides, here Here are a bunch of good onesguides:

http://accumulo.apache.org/releasing.html
http://accumulo.apache.org/verifying_releases.html

http://yetus.apache.org/contribute/releases/

How to Release Sqoop2 

How to Improve this Page

The purpose of this page is to document the mechanics of a release.  I took this phrase from the NiFi Release Guide (http://nifi.apache.org/release-guide.html) because I really like their outline and guide.  Eventually, we should incorporate other aspects into this guide including:

...

The flow of a release (an outline) 

(Feel free to toss in any additional sections here)

The mechanics of the release  

Release Manager

...

First-Time Preparation

Before a Release Manager begins cutting a release of Apache Rya, there are a handful of Dependencies and one-time Prerequisites they need to satisfy:

  1. They need to be a Rya Commiter

    As a Rya Commiter, the Release Manager will have access to several really important tools used throughout the release process.   This includes write access to the official Apache Rya Git Repo, the Apache Jenkins Server, and the Apache Nexus Server.

  2. Checkout and Build Rya

    The Release Manager should verify that they can checkout, build, and test Apache Rya on their machine.

    1. Checkout code from Git

      Code Block
      git clone https://git-wip-usgitbox.apache.org/repos/asf/incubator-rya.git rya
cd incubator-rya-apache


    2. Build and Test Apache Rya

      Code Block
      mvn clean install


  3. Setting up a PGP Key

    The Release manager needs a PGP Key and this key needs to be public.  The first two items listed here need to be done (i.e. get some software to create a key, and create a key).  I'm not sure about the last three.
    1. Getting software
      1. Get GPG.  (PGP should work as well, but this guide uses GPG).  It is probably already available in your linux distro.

      2. Get Use gpg-agent, and be sure to increase the gpg-agent cache timeout (via .gnupg/gpg-agent.conf) to ensure that the agent doesn’t require re-authentication mid-build, as it will cause things to fail. For example, you can add default-cache-ttl 6000 to increase the timeout from the default of 10 minutes to over an hour. If you do not have a GPG key, reference the very thorough ASF release signing documentation.

      3. Here is a working example of ~/.gnupg/gpg-agent.conf

        default-cache-ttl 28800
        # 8 hours
        pinentry-program /usr/bin/pinentry-curses
        allow-loopback-pinentry

      4. make sure you reset gpg-agent:

        echo RELOADAGENT | gpg-connect-agent


      5. Using Fedora, might need to use gpg2.  These two must match:

        1. gpg --version

        2. gpg-agent --version
          If they don't, check for version 2:
        1. gpg2 --version
        2. I was able to copy/link the commands in /usr/bin/:  mv gpg  gpg1 and ln -s gpg2  gpg

    2. Creating a key If you do not have a GPG key, reference the very thorough ASF release signing documentation available at https://www.apache.org/dev/release-signing.html.
    3. Publishing a key to a public server
    4. Adding your key to id.apache.org
    5. Adding your key to project key list (we currently don't have this)
  4. Setting up Maven to push to apache repo
    1. Username/pass in maven settings
  5. Add Key to KEYS file

    1. See it here:

      https://dist.apache.org/repos/dist/release/rya/KEYS


    2. Do this with your name where it says "David":

      svn checkout https://dist.apache.org/repos/dist/release/rya
      (gpg --list-sigs "David"    && gpg --armor --export "David") >> rya/KEYS 
      cd rya
      svn commit -m "Adding David to KEYS"


  6. Testing settings
    1. Testing locally: 
      1. mvn clean install -Papache-release

    2. Testing by pushing to apache snapshot repo

      1. mvn clean deploy -Papache-release

Staging a release

note: I followed these instructions: http://www.apache.org/dev/publishing-maven-artifacts.html#prepare-poms

...

    1. Testing private key use and gpg-agent prompting.  This is a way to preload the key before the release build below.  Do this with your name (Unique initial characters of the full name are accepted) where it says "David":
      1. eval $(gpg-agent --daemon) ### Start the gpg-agent if not already.

      2. echo "hello" >> text.txt

      3. gpg -e -u "Aaron" -r "David" text.txt  ### These names are actually in the Rya keys file.

      4. gpg -d text.txt.gpg   ### this will prompt for David's key decrypt password and cache it.


Staging a Release

  1. Communicate: Give notice of the intent and timing of the release to the developer's list:  dev@rya.apache.org
    Later, request descriptions for the release notes.  A day before cutting a release candidate, do a last call for Pull requests.

  2. Make sure the year in the NOTICE file is correct. If not, update the NOTICE file with the correct year.
  3. Checkout out code 
    git clone https://

...

  1. gitbox.apache.org/repos/asf/

...

  1. rya.git

...

  1. git checkout -b 3.2.10-RC1 master

...

  1. mvn clean deploy -Papache-release

Do some manual checks (note I think that the mvn release plugin does all of this)

  1.  rya
    cd rya

  2. Create release candidate branch from master.  Name the branch <releaseVersion>-RC<N>


Code Block
git checkout -b 4.0.1-RC2 master


5. Build Rya, prepare and perform release.  This takes about 50 minutes.  At this point you must have the gpg-agent running and probably have the password cached.  It has failed without asking.  See the preparation section for details on testing private key use (6c).  It will ask several things, release names quoted below, and Apache committer username and password a few times, currently 3 times.


Code Block
mvn clean release:prepare -Pgeoindexing -Pbenchmark -Pgiraph

...

Make sure there are no dependencies on snapshots in the POMs to be released

Check that your POMs will not lose content when they are rewritten during the release process

...

-Darguments="-DskipTests"

...

Diff the original file pom.xml with the one called pom.xml.tag to see if the license or any other info has been removed



Code Block
What is the release version for "Apache Rya Project"?

...

(org.apache.rya:rya-project)

...

 4.

...

0.

...

1: :

...

 
What is SCM release tag or label for "Apache Rya Project"? (org.apache.rya:rya-project)

...

 v4.0.1: : rya-4.0.1-rc1
What is the new development version for "Apache Rya Project"? (org.apache.rya:rya-project)

...

 4.

...

1.

...

0-SNAPSHOT: :


For the next step, create file: settings.xml containing your Apache Nexus username and password.  Make sure you remove the password after your done.  You can checking your password by logging in at: 

...

https://repository.apache.org/

Code Block
languagexml
titleCreate file: ~/.m2/settings.xml
<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
   <servers>
     <server>
       <id>apache.releases.https</id>
       <username>PUT_USERID_HERE</username>
       <password>PUTPASSWORDHERE</password>
     </server>
   </servers>
   <pluginGroups></pluginGroups>
   <proxies></proxies>
   <mirrors></mirrors>
   <profiles></profiles>
</settings>


Then perform the release for the release candidate.  If this command fails with "Return code is: 401, ReasonPhrase: Unauthorized." then your user/password is wrong, or you already have a file with the same name.  See this for troubleshooting: https://stackoverflow.com/questions/24830610/why-am-i-getting-a-401-unauthorized-error-in-maven

Code Block

...

Prepare release

  1. mvn release:clean

  2. mvn release:prepare

...

mvn release:perform

...

 

 

 -Darguments="-DskipTests"


6. Add the artifacts to dist

Code Block
cd ..
svn checkout

super short edition:

1.Checkout out code
git clone 
https://
git-wip-us
dist.apache.org/repos/
asf/incubator-rya.git incubator-rya-apache
2. Create release candidate branch

 

Code Block
git checkout -b 3.2.10-RC1 master
git push -u origin 3.2.10-RC1

 

3. Build Rya, prepare and perform release

 

Code Block
mvn clean package -Papache-release
Code Block
mvn release:prepare

 

dist/dev/rya
cd rya
mkdir rya-4.0.1-rc3
cd rya-4.0.1-rc3
 
cp ../../rya/target/checkout/target/rya-project-4.0.1-source-release.zip* .
openssl sha512 -r rya-project-4.0.1-source-release.zip > rya-project-4.0.1-source-release.zip.sha512
 
cd ..
 
svn add rya-4.0.1-rc3
svn commit -m "Adding rya-4.0.1-rc3"


7. Check that your keys are in the file: KEYS . It is here:  https://dist.apache.org/repos/dist/release/rya/KEYS
If not, see Release Manager Dependencies section above.


8. Once the release has been performed, the artifacts should be in a staging repository:
From "Publishing Maven Artifacts" page:
Now you must close the staging repository to indicate to Nexus that the build is done and to make the artifacts available. Follow the steps in Closing the Staged Repository to close your new repository, this will allow your community to VOTE on the staged artifacts.

9. Download the src artifact and performs some simple tests:

  • Download the sources and verify they compile cleanly.
  • Validate the hashes match.
  • Validate that the sources contain no unexpected binaries.
  • Validate the signature for the build and hashes.
  • Validate the LICENSE/NOTICE/Headers.

 Other release checklists:

http://wiki.apache.org/incubator/ReleaseChecklist

http://incubator.apache.org/guides/releasemanagement.html#check-list


how to verify hashes (note that md5 and sha1 are no longer considered useful; sha512 is recommended):

https://www.openoffice.org/download/checksums.html


Code Block
titleCommands to build Rya with a Fresh Maven Repo and Settings.xml
rm DEPENDENCIES
mkdir /tmp/new_m2
cp $M2_HOME/conf/settings.xml /tmp/new_m2/settings.xml
mvn --settings /tmp/new_m2settings.xml -Dmaven.repo.local=/tmp/new_m2 clean package



Code Block
titleCommand to Check for Binary Files
find . -type f | grep -v '\/target\/\|\/test\/\|\/site\/\|\.java\|\.xml\|\.xsl\|\.groovy\|\.properties\|\.sh\|\.bat\|\.md\|\.txt'



Code Block
titleFind large files
find . -type f | xargs du -sh | grep [GM]\\s


Send out a Vote email (make sure to include some sort of "RC-X" identifier in the vote subject so that we can differentiate them in the archives)

Code Block
To: dev@rya.apache.org
Subject: [VOTE] Release Rya version 4.0.1 RC2

I am pleased to be calling this vote for the source release of Apache Rya, version 4.0.1.
 
The source zip, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/rya/rya-4.0.1-rc2/

Ancillary artifacts such as poms, jars, wars, ect. can be found here:
https://repository.apache.org/content/repositories/orgapacherya-1002/org/apache/rya/rya-project/4.0.1/

The Git tag is rya-4.0.1-rc2
The Git commit ID is 9f0d63e6089df172eb3f41957d2956ec0035953a
https://gitbox.apache.org/repos/asf/rya.git;a=commit;h=9f0d63e6089df172eb3f41957d2956ec0035953a
 
Checksums of rya-project-4.0.1-source-release.zip:
SHA512: b6ae761aa42f80e1cb84bb645184e29e4436eaaaf66aa7249aa65b6adb38f3dfdd9a6f8b536eb0e2de88ac75cc2a0c8d97d85b4dd0ffef7f9407e05642110b3f
 
Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/mihalik.asc

KEYS file available here:
https://dist.apache.org/repos/dist/release/rya/KEYS
 
Issues that were closed/resolved for this release are here:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12334209&styleName=Html&projectId=12319020
 
Issues resolved between RC1 and RC2 are here:
https://issues.apache.org/jira/browse/RYA-184

The vote will be open for at least 72 hours and close after <Date Time and Timezone>.
Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build from source, and test.  Then please vote:
 
[ ] +1 Release this package as rya-project-4.0.1
[ ] +0 no opinion
[ ] -1 Do not release this package because because...

If the vote was unsuccessful, respond to the dev list with the "Unsuccessful" Vote Email

Code Block
To: dev@rya.apache.org
Subject: [RESULT] [VOTE] Release Rya version 4.0.1 RC1

Hello,

The vote to release Rya version 4.0.1 RC1 has failed.

+1 (binding):

(Members of PMC)

+1 (non binding):

(Everyone else)

(List of series of tasks that have been created or already exist that block the release)
The following Jira tasks have been created to capture the issues blocking a successful release:
 
RYA-XXX <Issue Title> [1]
RYA-XXX <Issue Title> [2]
RYA-XXX <Issue Title> [3]


(Create an issue that links to all of these issues)
The following Jira task has been created to capture all of the issues that will be resolved in the next RC:

RYA-XXX Perform 4.0.1-RC<N+1> Release [4]
 
These tasks should address the concerns raised during the voting process.

Since Rya is no longer incubating, Rya only has to pass the vote from the PMC.  

If vote passed, send Final Release Email

Code Block
To: dev@rya.apache.org
Subject: [RESULT] [VOTE] Release Rya version 4.0.1 RC1

Hello,

The vote to release Rya version 4.0.1 RC1 has passed.

+1 (binding):

(Members of PMC)



+1 (non binding):

(Everyone else)

Thank you all for voting!
I will promote the artifacts to the central repository.
 


Code Block
To: announce@apache.org, dev@rya.apache.org, general@incubator.apache.org
Subject: [ANNOUNCE] Apache Rya 4.0.1 released
 
The Apache Rya team is happy to announce the release of Apache Rya 4.0.1:

https://rya.apache.org/release/rya-4.0.1/

Rya (pronounced "ree-uh" /rēə/) is a cloud-based RDF triple store that supports SPARQL queries. Rya is a scalable RDF data management system built on top of Apache Accumulo®. Rya uses novel storage methods, indexing schemes, and query processing techniques that scale to billions of triples across multiple nodes. Rya provides fast and easy access to the data through SPARQL, a conventional query mechanism for RDF data.

[Include highlights of changes in this release]


Thanks,
The Apache Rya team

 

Code Block
What is the release version for "Apache Rya Project"? (org.apache.rya:rya-project) 3.2.10: : 
What is SCM release tag or label for "Apache Rya Project"? (org.apache.rya:rya-project) v3.2.10: : v3.2.10-RC1
What is the new development version for "Apache Rya Project"? (org.apache.rya:rya-project) 3.2.11-SNAPSHOT: :

 

 

Code Block
mvn release:perform

 

Once the release has been performed, the artifacts should be in a staging repository:
From "Publishing Maven Artifacts" page:
Now you must close the staging repository to indicate to Nexus that the build is done and to make the artifacts available. Follow the steps in Closing the Staged Repositoryto close your new repository, this will allow your community to VOTE on the staged atrifacts.


If there is something wrong with the staging repo:
delete the tag:

 
 Code Block
git tag -d v3v4.20.101-RC1
git push origin :refs/tags/v3v4.20.101-RC1
 
git branch -D 34.20.101-RC1
git push origin --delete 34.20.101-RC1
 

Rya uses Mini Accumulo Cluster for a number of it's tests and I've run into a number of gotchas:
  1. On windows, I've found that I need to update my hosts file to add my ip and my machine name (something like 10.aaa.bbb.ccc mymachinename)
  2. Close all VPNs
  3. Do not use strange DNS machines.  just use the default ones your network/isp provides
  4. McAfee Virus Scan may slow things down and throw off the test

When merging a pull request, it's important to verify whether or not new commits are pulling in any third party
dependencies that are incompatible with ASF.  To check whether the pull request contains invalid dependencies issue
the following command in your terminal:
 Code Block
>> mvn license:aggregate-add-third-party
This will generate a file THIRD-PARTY.txt in the directory target/generated-sources/license/ that lists the license for each java file in the jar.  You can now grep the directory for all licenses which are
not ASF approved licenses. 
The following command does a case insensitive search over all instances of THIRD-PARTY.txt in the project for licenses that are not in the list approved Apache Licenses .
 Code Block
egrep -iv "BSD|ASF|MIT|CDDL|EPL|Apache|Eclipse|Public Domain" target/generated-sources/license/THIRD-PARTY.txt

How to discover all the licenses for all the included libraries:  http://stackoverflow.com/questions/3500388/finding-out-the-licenses-of-jar-libraries
The Maven "Project Info Reports" plugin produces a Dependencies report that includes the Licenses for the dependencies.

After the release has been approved by PMC
  1. Release the Jars
    1. Go to https://repository.apache.org/ and release the staging repository
  2. Copy dist/dev artifacts to dist/release
     Code Block
    ### Starting from the folder where you svn checkout for the RC3:
mv rya ryadev  # first rename the RC3 dist/dev repo
svn checkout https://dist.apache.org/repos/dist/release/rya
mv rya ryarelease
cd ryarelease
mv ../ryadev/rya-4.0.1-rc3 ./
mv rya-4.0.1-rc3 rya-4.0.1
svn add rya-4.0.1 
svn commit -m "Adding rya-4.0.1"

  3. Create a rel/ tag in Git
    1. git checkout rya-4.0.1-rc3
    2. git tag -a rel/rya-4.0.1 -m "rya-4.0.1 Release"
    3. git push origin rel/rya-4.0.1
  4. Merge Release branch into Master and Delete Release Branch
    1. git checkout master
    2. git pull origin
    3. git merge 4.0.1-RC3
    4. git push origin master
  5. If everything looks great, delete the branches:
    git push origin --delete 4.0.1-RC3      
    git push origin --delete 4.0.1-RC2
    git push origin --delete 4.0.1-RC1   
  6. And tags:
    git push --delete origin rya-4.0.1-rc2
    git push --delete origin rya-4.0.1-rc1
    git tag --delete rya-4.0.1-rc1  ### local repo
    git tag --delete rya-4.0.1-rc2  ### local repo
  7. Update the website, see instructions on this wiki
  8. Update projects.apache.org: Fill out the add release form to update the projects website.
  9. Send out an announce email 
  10. Announce the release on Twitter




References:
http://www.apache.org/dev/publishing-maven-artifacts.html