Page History

This page is to document the release procedure for HCatalog. HCatalog is currently a Apache incubator project. Its release process is work in progress and is modeled from [Apache Pig Release|http://wiki.apache.org/pig/HowToRelease].

Note that only HCatalog committers can create a release. Copied verbatim from Pig wiki, will edit as I go along.


h1. Preparation

TODO:
# Define issue management process like assigning/removing issues from release.

h1. Creating Release Branch

We only branch for major (X.0.0) and minor(X.Y.0) releases but not for patches (X.Y.Z). Patch is and update to an existing branch created for X.Y.0.

1. Send email to _hcatalog-dev@incubator.apache.org_ to notify that you about to branch the tree and ask to hold off any commits till this is finished.

2. Edit _src/docs/src/documentation/content/xdocs/site.xml_.  In the external reference for api where the link contains the previous version number change this string to the correct version number.

3. Commit these changes to trunk:  

{code}
svn commit -m "Preparing for release X.Y.Z"
{code}
4. Create a branch for the release series:


{code}
svn copy https://svn.apache.org/repos/asf/incubator/hcatalog/trunk  \
https://svn.apache.org/repos/asf/incubator/hcatalog/branches/branch-X.Y -m "Branching for X.Y releases"
{code}
5. Update the default version in _build.xml_ on trunk to X.Y+1.0-dev.


6. Commit these changes to trunk:

{code}
svn commit -m "Preparing for X.Y+1.0 development"
{code}

h1. Updating Release Branch

The steps in this section are needed for all the releases (major, minor, and patches).

1. Check out the branch with:{{{
svn co [https://svn.apache.org/repos/asf/pig/branches/branch-X.Y]
}}}
2. Run rat report and make sure that all files that can have apache license agreement. Note that currently the rat report is part of output and needs to be manually extracted. TODO: write the report into a file.{{{
ant \-Djava5.home=<java5 jdk dir>  \-Dforrest.home=<forrest installation dir> releaseaudit >out
}}}
2. For patches, update CHANGES.txt to include the release version and date. See #2 from Create Release Branch section.
3. Update RELEASE_NOTES.txt for this release:
a. Make sure to change all of the version number references to the current version number.
a. Note highlights for this release.  CHANGES.txt is a great place to find these.
a. Note incompatibilities for this release.  These should be listed under INCOMPATIBLE CHANGES in CHANGES.txt.
4. Update the version number in `build.xml` to be X.Y.N-dev, where N is one greater than the release being made.
5. Commit these changes:{{{
svn commit \-m "Preparing for release X.Y.Z"
}}}
6. Tag the release candidate:{{{
svn copy [https://svn.apache.org/repos/asf/pig/branches/branch-X.Y] \[https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z-rcR\] \-m "Pig X.Y.Z-rcR release."
}}}

== Building ==

1. Build the release and run unit tests: {{{
ant clean
ant test
ant clean
ant jar
cd contrib/zebra
ant
cd ../..
cd contrib/piggybank/java
ant
cd ../../..
ant \-Dversion=X.Y.Z \-Dforrest.home=<Forrest installation dir>  tar
}}}
2. Test the tar file by unpacking the release and
* building pig.jar: {{{ant }}}
* building and running tutorial {{{
cd tutorial
ant
}}}
* build piggybank{{{
cd contrib/piggybank/java
ant
}}}
* running unit tests {{{ant test}}}
3. Generate the MD5 checksum of the release file: {{{
cd build
md5sum pig-X.Y.Z.tar.gz > pig-X.Y.Z.tar.gz.md5
}}}
4. If you do not have a gpg key pair, do the following steps:
a. Generating key pair using the following command. You can simply accept all default settings and give your name, email and Passphase. {{{
gpg \--gen-key
}}}
a. Export your public key. {{{
gpg \--armor \--output pubkey.txt \--export 'Your Name'
}}}
a. Open pubkey.txt, copy the full text and append it to the following files by pasting, then commit these changes: {{{[https://svn.apache.org/repos/asf/pig/branches/branch-X.Y.Z/KEYS][https://svn.apache.org/repos/asf/pig/trunk/KEYS]
}}}
a. Upload updated KEYS to Apache. {{{
scp KEYS people.apache.org:/www/www.apache.org/dist/pig/KEYS
}}}
a. Export your private key, keep it with you. {{{
gpg \--export-secret-key \-a "Your Name" > private.key
}}}
5. Sign the release (see \[[http://www.apache.org/dev/mirror-step-by-step.html?Step-By-Step|Step-By-Step Guide to Mirroring Releases]\] for more information). {{{
gpg \--armor \--output pig-X.Y.Z.tar.gz.asc \--detach-sig pig-X.Y.Z.tar.gz
}}}
6. Verify gpg signature. {{{
gpg \--import KEYS  (if necessarily)
gpg \--verify pig-X.Y.Z.tar.gz.asc pig-X.Y.Z.tar.gz
}}}
7. Copy release files to a public place (usually into public_html in your home directory):{{{
ssh people.apache.org mkdir public_html/pig-X.Y.Z-candidate-0
scp \-p pig-X.Y.Z.tar.gz\* people.apache.org:public_html/pig-X.Y.Z-candidate-0
cd ..
scp RELEASE_NOTES.txt people.apache.org:public_html/pig-X.Y.Z-candidate-0
}}}
7. Push proposed release to Maven staging area
a.  Create file \~/.m2/settings.xml with following contents (NOTE: It is highly recommended to use \[[http://maven.apache.org/guides/mini/guide-encryption.html|Maven's password encryption capabilities]\] for your passwords.): {{{
<settings>
<servers>
<server>
<id>apache.releases.https</id>
<username>your apache id</username>
<password>your apache password</password>
</server>
<server>
<id>apache.snapshots.https</id>
<username>your apache id</username>
<password>your apache password</password>
</server>
<server>
<id>apache.staging.https</id>
<username>your apache id</username>
<password>your apache password</password>
</server>
</servers>

<profiles>
<profile>
<id>apache-release</id>
<properties>
<gpg.passphrase>your GPG password</gpg.passphrase>
</properties>
</profile>
</profiles>
</settings>
}}}
a. Run ant command {{{ant mvn-deploy}}} to publish Pig artifacts to the apache snapshot repository.
a. Run ant command {{{ant –Drepo=staging –Dversion=X.Y.Z mvn-deploy}}} to publish Pig artifacts to the apache staging repository.
8. Call a release vote. The initial email should be sent to `dev@pig.apache.org`. Make sure to attache rat report to it. Here is a sample of email: {{{
From: Olga Natkovich [mailto:olgan@yahoo-inc.com]
Sent: Tuesday, November 25, 2008 3:59 PM
To: dev@pig.apache.org
Subject: [VOTE] Release Pig 0.1.1 (candidate 0)

Hi,

I have created a candidate build for Pig 0.1.1. This release is almost identical to Pig 0.1.0 with a couple of exceptions:

(1) It is integrated with hadoop 18
(2) It has one small bug fix (PIG-253)
(3) Several UDF were added to piggybank - pig's UDF repository

The rat report is attached.

Keys used to sign the release are available at [http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup].

Please download, test, and try it out:

[http://people.apache.org/~olga/pig-0.1.1-candidate-0]



Should we release this? Vote closes on Wednesday, December 3rd.

Olga
}}}

Forward the initial email to `private@pig.apache.org` for Pig PMC members to vote.

== Publish ==

Once \[[http://www.apache.org/foundation/voting.html#ReleaseVotes|three PMC members have voted for a release]\], it may be published.

1. Tag the release: {{{
svn move [https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z-rcR] \[https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z\] \-m "Pig X.Y.Z release."
}}}
2. Copy release files to the distribution directory and make them writable by the pig group.{{{
ssh people.apache.org
cp \-pr public_html/pig-X.Y.Z-candidate-0 /www/www.apache.org/dist/pig/pig-X.Y.Z
cd /www/www.apache.org/dist/pig
chgrp \-R pig pig-X.Y.Z
chmod \-R g+w pig-X.Y.Z
}}}
3. The release directory usually contains just two releases, the most recent from two branches, with a link named 'stable' to the most recent recommended version.{{{
ssh people.apache.org
cd /www/www.apache.org/dist/pig
rm \-rf pig-A.B.C; rm stable
ln \-s pig-A.B.D stable
}}}
3. Push Maven release from staging to production
a. Go to \[[https://repository.apache.org/index.html#view-repositories;staging~browsestorage]\]
a. Log in, using your Apache LDAP credentials.  The sign in link is in the upper right hand corner.
a. In the frame on the left side of the page, select "Staging Repositories", you should now see a list of artifacts in the main frame.
a. Select the appropriate artifacts for this release and click "Close" on the bar above the list of artifacts.
a. Select the appropriate artifacts for this release and click "Release" on the bar above the list of artifacts.
4. Wait 24 hours for release to propagate to mirrors.
5. Prepare to edit the website.{{{
svn co [https://svn.apache.org/repos/asf/pig/site]
}}}
6. Update the front page news in author/src/documentation/content/xdocs/index.xml.
7. Update the release news in author/src/documentation/content/xdocs/releases.xml.
8. Update the documentation links in author/src/documentation/content/xdocs/site.xml
9. Copy in the release specific documentation {{{
cd publish
mkdir docs/rX.Y.Z
cp \-pr <releasedir>/docs/\* publish/docs/rX.Y.Z/
svn add publish/docs/rX.Y.Z
}}}
10. Regenerate the site, review it and commit in HowToCommit.
11. Deploy your site changes.{{{
ssh people.apache.org
cd /www/pig.apache.org
svn up
}}}
12. Wait until you see your changes reflected on the Apache web site.
13. Send announcements to the user and developer lists as well as (`announce@haoop.apache.org`) once the site changes are visible. {{{
Pig  team is happy to announce Pig X.Y.Z release.

Apache Pig provides a high-level data-flow language and execution framework for parallel computation on Hadoop clusters.
More details about Pig can be found at [http://pig.apache.org/].

The highlights of this release are ... The details of the release can be found at [http://pig.apache.org/releases.html].
}}}
14. In JIRA, mark the release as released.
a. Goto JIRA and click on Administration tab.
a. Select the Pig project.
a. Select Manage versions.
a. Select Release for the version you have released.
a. If a description has not yet been added for the version you are releasing, select Edit Details and give a brief description of the release.
a. If the next version does not exist (that is, if you are releasing version 0.x, if version 0.x+1 does not yet exist) create it using the Add Version box at the top of the page.

15. In JIRA, mark the issues resolved in this release as closed.
a. Goto JIRA and click on the "Search for Issues" on "Issues" menu.
a. In the left hand Edit section, set Project to Pig.
a. In Status select "Resolved"
a. In Resolutions select "Fixed"
a. Click "Search" button
a. In the next screen, further select fix For select the version you are releasing.
a. Click on the "Search" button
a. Select "Tools->Bulk change all XX issues" (near the top right)
a. Select all the issues and click on "Next"
a. Select "Transition Issues" radio button and click on "Next"
a. Select "Close Issue" radio button and click on "Next"
a. Uncheck the box near the bottom at says "Send mail for this update" lest you spam every Pig developer with a message for every bug resolved in this release.  Click "Next".
a. Click "Confirm".  Don't worry if it gives you a HTTP 500 error, it still does the transitions.

16. Update jdiff for next release (step 16 to 19). {{{
svn co [https://svn.apache.org/repos/asf/pig/trunk]
}}}
17. Open build.xml. Change this line: {{{
<property name="jdiff.stable" value="X.Y-1.Z"/>
To
<property name="jdiff.stable" value="X.Y.Z"/>
}}}
18. Copy jdiff comparison base to trunk {{{
cp {releasedir}/lib/jdiff/pig_X.Y.Z.xml lib/jdiff
}}}
19. {{{
svn add lib/jdiff/pig_X.Y.Z.xml
svn remove lib/jdiff/pig_X.Y-1.Z.xml
svn commit \-m "Jdiff change for X.Y.Z"
}}}
[TODO Need to integrate javadoc into this.]