This The release directory usually contains just three releasesThis page is to document the release procedure for Pig. Pig is a fairly young Apache project. Its release process is work in progress and is modeled from Hadoop Release Procedure.
...
- Send email to
dev@pig.apache.org
to notify that you about to branch the tree and ask to hold off any commits till this is finished. Update
CHANGES.txt
to include the release version and date (useUnreleased
for the date if it is unknown) and removeTrunk (unreleased changes)
. Below is the example of the top of theCHANGES.txt
file after the update:Code Block Pig Change Log Release 0.1.1 - Unreleased INCOMPATIBLE CHANGES NEW FEATURES IMPROVEMENTS PIG-253: integration with hadoop-18 BUG FIXES PIG-342: Fix DistinctDataBag to recalculate size after it has spilled. (bdimcheff via gates)
- Edit
src/docs/src/documentation/content/xdocs/site.xml
. In the external reference for api where the link contains the previous version number change this string to the correct version number. Commit these changes to trunk:
Code Block svn commit -m "Preparing for release X.Y.Z"
Create a branch for the release series:
Code Block svn copy https://svn.apache.org/repos/asf/pig/trunk \ https://svn.apache.org/repos/asf/pig/branches/branch-X.Y -m "Branching for X.Y releases"
Update CHANGES.txt to add back in Trunk (unreleased changes). Top of the
CHANGES.txt
should look like this now:Code Block Pig Change Log Trunk (unreleased changes) INCOMPATIBLE CHANGES IMPROVEMENTS OPTIMIZATIONS BUG FIXES Release 0.1.1 - Unreleased ....
- Update the default version in
build.xml
on trunk to X.(Y+1).0-dev. Commit these changes to trunk:
Code Block svn commit -m "Preparing for X.(Y+1).0 development"
Updating Release Branch
The steps in this section are needed for all the releases (major, minor, and patches).
Check out the branch with:
Code Block svn co https://svn.apache.org/repos/asf/pig/branches/branch-X.Y
Run rat report and make sure that all files that can have apache license agreement. Save this rat report.
Code Block ant releaseaudit
- For patches, update CHANGES.txt to include the release version and date with the new version. See #2 from Create Creating Release Branch section.
- Update RELEASE_NOTES.txt for this release:
- Make sure to change all of the version number references to the current version number.
- Note highlights for this release. CHANGES.txt is a great place to find these.
- Note incompatibilities for this release. These should be listed under INCOMPATIBLE CHANGES in CHANGES.txt.
- Update the version number in in
build.xml
to to be X.Y.N-SNAPSHOT, where N is one greater than the release being made. - Update pig.version in libraries.properties to X.Y.Z
Commit this
Commit thesechanges:
Code Block svn commit -m "Preparing for release X.Y.Z"
Tag the release candidate:
Code Block svn copy https://svn.apache.org/repos/asf/pig/branches/branch-X.Y \ https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z-rcR -m "Pig X.Y.Z-rcR release."
Building
Build and run unit tests:
Code Block ant clean ant jar test
Check that contrib and tutorial directories compile and tests pass:
Code Block cd contrib/zebrapiggybank/java ant ant test cd ../.. cd contrib/piggybank/java/../tutorial ant ant test cd ../../../tutorial ant
Build the source release and save the src.tar.gz artifact:
Code Block ant -Dversion=X.Y.Z src-release
- Test the source tar file by unpacking the release and
- building pig.jar:
ant
building and running tutorial
Code Block cd tutorial ant cd ..
build piggybank
Code Block cd contrib/piggybank/java ant ant test cd ../../..
- running unit tests
ant test-commit
- building pig.jar:
Build the convenience artifacts:
Code Block ant clean ant test ant clean ant jar cd contrib/piggybank/zebrajava ant cd ../../.. cd contrib/piggybank/java ant cd ../../.. ant -Dversionant -Dversion=X.Y.Z -Dforrest.home=<Forrest installation dir> tar ant -Dversion=-h12
Generate the MD5 checksum of the release artifact and convenience binaries:
Code Block cd build md5sum pig-X.Y.Z -Dforrest.home=<Forrest installation dir> rpm (on a Fedora/Redhat system) ant -Dversion=src.tar.gz > pig-X.Y.Z-src.tar.gz.md5 md5sum pig-X.Y.Z.tar.gz > -Dforrest.home=<Forrest installation dir> deb (on a Ubuntu/Debian system)
- Generate the MD5 checksum of the release artifact and convenience binaries:
Code Block cd build md5sum pig-X.Y.Z-src.tar.gz > pig-X.Y.Z-src.tar.gz.md5 md5sum pig-X.Y.Z.tar.gz > pig-X.Y.Z.tar.gz.md5 md5sum pig-X.Y.Z-1.i386.rpm > pig-X.Y.Z-1.i386.rpm.md5 md5sum pig-X.Y.Z-1.i386.deb > md5sum pig-X.Y.Z-1.i386.deb.md5
- If you do not have a gpg key pair, do the following steps:
- Generating key pair using the following command. You can simply accept all default settings and give your name, email and Passphase.
Code Block gpg --gen-key
- Export your public key.
Code Block gpg --armor --output pubkey.txt --export 'Your Name'
- Open pubkey.txt, copy the full text and append it to the following files by pasting, then commit these changes:
Code Block https://svn.apache.org/repos/asf/pig/branches/branch-X.Y.Z/KEYS https://svn.apache.org/repos/asf/pig/trunk/KEYS
- Upload updated KEYS to Apache.
Code Block scp KEYS people.apache.org:/www/www.apache.org/dist/pig/KEYS
- Export your private key, keep it with you.
Code Block gpg --export-secret-key -a "Your Name" > private.key
- Generating key pair using the following command. You can simply accept all default settings and give your name, email and Passphase.
pig-X.Y.Z.tar.gz.md5
- If you do not have a gpg key pair, do the following steps:
Generating key pair using the following command. You can simply accept all default settings and give your name, email and Passphase.
Code Block gpg --gen-key
Export your public key.
Code Block gpg --armor --output pubkey.txt --export 'Your Name'
Open pubkey.txt, copy the full text and append it to the following files by pasting, then commit these changes:
Code Block https://svn.apache.org/repos/asf/pig/branches/branch-X.Y.Z/KEYS https://svn.apache.org/repos/asf/pig/trunk/KEYS
Upload updated KEYS to Apache.
Code Block scp KEYS people.apache.org:/www/www.apache.org/dist/pig/KEYS
Export your private key, keep it with you.
Code Block gpg --export-secret-key -a "Your Name" > private.key
Sign the release artifact only (see Step-By-Step Guide to Mirroring Releases for more information).
Code Block gpg --armor --output pig-X.Y.Z-src.tar.gz.asc --detach-sig pig-X.Y.Z-src.tar.gz
Verify gpg signature.
Code Block gpg --import KEYS (if necessarily) gpg --verify pig-X.Y.Z-src.tar.gz.asc pig-X.Y.Z-src.tar.gz
Copy release artifact, convenience binaries, release notes and the rat report to a public place (usually into public_html in your home directory):
Sign the release artifact only (see Step-By-Step Guide to Mirroring Releases for more information).Code Block ssh people.apache.org mkdir public_html/pig-X.Y.Z-rcR scp -p pig-X.Y.Z* people.apache.org:public_html/
Verify gpg signature.Code Block gpg --armor --output pig-X.Y.Z-srcrcR cd .tar.gz.asc --detach-sig . scp RELEASE_NOTES.txt people.apache.org:public_html/pig-X.Y.Z-rcR scp <rat_report> people.apache.org:public_html/pig-X.Y.Z-src.tar.gzrcR
- Push proposed release to Maven staging area
Create file ~/.m2/settings.xml with following contents (NOTE: It is highly recommended to use Maven's password encryption capabilities for your passwords.):
Code Block <settings>
<servers>
- Copy release artifact, convenience binaries, release notes and the rat report to a public place (usually into public_html in your home directory):
Code Block ssh people.apache.org mkdir public_html/pig-X.Y.Z-candidate-0 scp -p pig-X.Y.Z* people.apache.org:public_html/pig-X.Y.Z-candidate-0 cd .. scp RELEASE_NOTES.txt people.apache.org:public_html/pig-X.Y.Z-candidate-0 scp <rat_report> people.apache.org:public_html/pig-X.Y.Z-candidate-0
- Push proposed release to Maven staging area
- Create file ~/.m2/settings.xml with following contents (NOTE: It is highly recommended to use Maven's password encryption capabilities for your passwords.):
Code Block <settings> <servers> <server> <id>apache.<server> <id>apache.releases.https</id> <username>your apache id</username> <password>your apache password</password> </server> <server> <id>apache.snapshots.https</id> <username>your apache id</username> <password>your apache password</password> </server> <server> <id>apache.staging.https</id> <username>your apache id</username> <password>your apache password</password> </server> </servers> <profiles> <profile> <id>apache-release</id> <properties> <gpg.passphrase>your GPG password</gpg.passphrase> </properties> </profile> </profiles> </settings>
- Run ant command
ant mvn-deploy
to publish Pig artifacts to the apache snapshot repository. - Run ant command
ant –Drepo=staging –Dversion=X.Y.Z mvn-deploy
to publish Pig artifacts to the apache staging repository.
- Create file ~/.m2/settings.xml with following contents (NOTE: It is highly recommended to use Maven's password encryption capabilities for your passwords.):
Call a release vote. The initial email should be sent to
dev@pig.apache.org
. Here is a sample of email:Code Block From: Olga Natkovich [mailto:olgan@yahoo-inc.com] Sent: Tuesday, November 25, 2008 3:59 PM To: dev@pig.apache.org Subject: [VOTE] Release Pig 0.1.1 (candidate 0) Hi, I have created a candidate build for Pig 0.1.1. This release is almost identical to Pig 0.1.0 with a couple of exceptions: (1) It is integrated with hadoop 18 (2) It has one small bug fix (PIG-253) (3) Several UDF were added to piggybank - pig's UDF repository Keys used to sign the release are available at are available at http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup. Please download, test, and try it out: http://svnpeople.apache.org/viewvc/pig/trunk/KEYS?view=markup. Please download, test, and try it out: http://people.apache.org/~olga/pig-0.1.1-candidate-0 Release notes and the rat report are available from the same location. Should we release this? Vote closes on Wednesday, December 3rd. Olga
Forward the initial email to private@pig.apache.org
for Pig PMC members to vote.
Publish
Once three PMC members have voted for a release and a majority vote to release and the required 3 days have passed, it may be published.
~olga/pig-0.1.1-rc0 Release notes and the rat report are available from the same location. Should we release this? Vote closes on Wednesday, December 3rd. Olga
Forward the initial email to private@pig.apache.org
for Pig PMC members to vote.
Publish
Once three PMC members have voted for a release and a majority vote to release and the required 3 days have passed, it may be published.
Tag the release:
Code Block svn move https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z-rcR \ https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z -m "Pig X.Y.Z release."
Checkout Pig distribution svn.
Code Block svn co https://dist.apache.org/repos/dist/release/pig pig-dist
Copy release files to the distribution directory add them to svn.
Code Block cd pig-dist cp -pr ~/release/pig-X.Y.Z-candidate-0 pig-X.Y.Z svn add pig-X.Y.Z
The release directory should only contain the current releases, usually one or two (one is previous maintenance release, one is the most recent version release), with a link named 'latest' to the most recent version.
Code Block svn rm pig-A.B.C svn rm latest link -s
- Tag the release:
Code Block svn move https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z-rcR \ https://svn.apache.org/repos/asf/pig/tags/release-X.Y.Z -m "Pig X.Y.Z release."
- Copy release files to the distribution directory and make them writable by the pig group.
The release directory usually contains just two releases, the most recentCode Block ssh people.apache.org cp -pr public_html/pig-X.Y.Z-candidate-0 /www/www.apache.org/dist/pig/pig-X.Y.Z cd /www/www.apache.org/dist/pig chgrp -R pig pig-X.Y.Z chmod -R g+w pig-X.Y.Z
latest
Commit .the distribution artifacts changes
Code Block svn commit --message "Pig X.Y.Z Release"
Check if svn changes propagates to the staging directory
from two branches, with a link named 'stable' to the most recent recommended version.Code Block Code Block ssh people.apache.org cd /www/www.apache.org/dist/pig rm -rf pig-A.B.C; rm stable ln -s pig-A.B.D stable
After several minutes, you should see your distribution changes here.
- Push Maven release from staging to production
- Go to https://repository.apache.org/index.html#view-repositories;staging~browsestorage
- Log in, using your Apache LDAP credentials. The sign in link is in the upper right hand corner.
- In the frame on the left side of the page, select "Staging Repositories", you should now see a list of artifacts in the main frame.
- Select the appropriate artifacts for this release and click "Close" on the bar above the list of artifacts.
- Select the appropriate artifacts for this release and click "Release" on the bar above the list of artifacts.
- Wait 24 hours for release to propagate to mirrors.
Prepare to edit the website.
Code Block svn co https://svn.apache.org/repos/asf/pig/site
- Update the front page news in author/src/documentation/content/xdocs/index.xml.
- Update the release news in author/src/documentation/content/xdocs/releases.xml.
- Update the documentation links in author/src/documentation/content/xdocs/site.xml
Copy in the release specific documentation
Code Block cdmkdir publish mkdir /docs/rX.Y.Z cp -pr <releasedir>/docs/* publish/docs/rX.Y.Z/ svn add publish/docs/rX.Y.Z
- Regenerate the site, review it and commit in HowToCommit.
rX.Y.Z
- Regenerate the site, review it and commit in HowToDocument#UpdatingthePigSiteDocumentation.Deploy your site changes.
ssh people.apache.org cd /www/pig.apache.org umask 002 svn upCode Block - Wait until you see your changes reflected on the Apache web site. This might take a few minutes.
Send announcements to the user and developer lists as well as (dev@pig.apache.org; user@pig.apache.org; announce@apache.org;
announce@haoopgeneral@hadoop.apache.org) once the site changes are visible. Note that emails sent to
announce@apache.org
must be sent from yourapache.org
email address and you must be subscribed to each of the lists.Code Block [ANNOUNCE] Apache Pig X.Y.Z released The Pig team is happy to announce the Pig X.Y.Z release. Apache Pig provides a high-level data-flow language and execution framework for parallel computation on Hadoop clusters. More details about Pig can be found at http://pig.apache.org/. The highlights of this release are ... The details of the release can be found at http://pig.apache.org/releases.html.
- In JIRA, mark the release as released.
- Goto JIRA and click on Administration tab.
- Select the Pig project.
- Select Manage versionsVersions.
- Select Release for the version you have released.
- If a description has not yet been added for the version you are releasing, select Edit Details Add description and give a brief description of the release.
- If the next version does not exist (that is, if you are releasing version 0.x, if version 0.x+1 does not yet exist) create it using the Add Version box at the top of the page.
- In JIRA, mark the issues resolved in this release as closed.
- Goto JIRA and click on the "Search for Issues" on "Issues" menu.
- In the left hand Edit section, set Project to Pig.
- In Status select "Resolved"
- In Resolutions select "Fixed"
- Click "Search" button
- In the next screen, further select fix For select the version you are releasing.
- Click on the "Search" button
- Select "Tools->Bulk change all XX issues" (near the top right)
- Select all the issues and click on "Next"
- Select "Transition Issues" radio button and click on "Next"
- Select "Close Issue" radio button and click on "Next"
- Uncheck the box near the bottom at says "Send mail for this update" lest you spam every Pig developer with a message for every bug resolved in this release. Click "Next".
- Click "Confirm". Don't worry if it gives you a HTTP 500 error, it still does the transitions.
Update jdiff for next release (step 16 to 19).
Code Block svn co https://svn.apache.org/repos/asf/pig/trunk
Open build.xml. Change this line:
Code Block <property name="jdiff.stable" value="X.Y-1.Z"/> To <property name="jdiff.stable" value="X.Y.Z"/>
Copy jdiff comparison base to trunk
Code Block cp {releasedir}/lib}/src/docs/jdiff/pig_X.Y.Z.xml src/docs/jdiff
Code Block svn add src/docs/jdiff/pig_X.Y.Z.xml lib/jdiff
Code Block svn addremove libsrc/docs/jdiff/pig_X.Y.Z.xml svn remove lib/jdiff/pig_X.Y-1.Z.xml -1.Z.xml svn commit -m "Jdiff change for X.Y.Z"
Post Release
- Update the version number in
build.xml
in branch to be X.Y.N-SNAPSHOT, where N is one greater than the release being made. - Commit these changes:
Code Block |
---|
svn commit -m " |
...
Preparing for X.Y. |
...
N release development" |
...