Page History

...

# Append to CHANGES.txt file required by Apache
# If doing a minor release, append to existing CHANGES.txt file in release branch
# If doing a major release, copy CHANGES.txt file from last major release
#  and append to it (shown below)
$ cat CHANGES.txt | tail -n +3 > OLD_CHANGES.txt
$ echo "Spark Change Log" > CHANGES.txt
$ echo "" >> CHANGES.txt
$ echo "Release 0.9.0-incubating" >> CHANGES.txt
$ echo "" >> CHANGES.txt
$ # below might be the shittiest code I’ve ever written. This will be much easier
$ # once all PR's use the new merge format.
$ git log v0.8.0-incubating..HEAD \
$>   --grep "pull request" \
$>   --pretty="QQ  %h %cd%nQQ  %s%nQQ  QQQ%b%nQQ"  \
$>   | grep QQ | sed s/QQ// | sed "s/^  QQQ\(.*\)$/  [\1]/" >> CHANGES.txt
$ cat OLD_CHANGES.txt >> CHANGES.txt
$ rm OLD_CHANGES.txt
$ git add CHANGES.txt && git commit -m "Change log for release 0.9.0-incubating"

...

• The process of creating releases has been automated via this release script
• Read and understand the script fully before you execute it. It will cut a Maven release, build binary releases and documentation, then copy the binary artifacts to a staging location on people.apache.org.
• NOTE: You must use git 1.7.X for this or else you'll hit this horrible bug

Rolling Back Release Candidates

• If a release candidate does not pass, it is necessary to roll back the commits which advanced Spark's versioning.

$ git fetch apache
$ git checkout apache/branch-0.8
$ git tag -d v0.8.1-incubating
$ git push origin :v0.8.1-incubating
$ git revert HEAD --no-edit    # revert dev version commit
$ git revert HEAD~2 --no-edit  # revert release commit
$ git push apache HEAD:branch-0.8

Auditing a Staged Release

...

• The release voting happens in two stages. First, a vote takes place on the Apache Spark developers list (the podling PMC or PPMC is voting), then one takes place on the general@i.a.o list (the IPMC). I used the same template for both votes. Look at past vote threads to see how this goes. Once the vote is finished you should also send out a summary e-mail with the totals (subject “[RESULT] [VOTE]...”).
• If possible, include attach a draft of the release notes with the e-mail
• Include Attach the CHANGES.txt file in the e-mail
• NOTE: This will change once we graduate and there will be a single vote
  

Cutting the Official Release

Performing the Final Release in Nexus

• Find the staging repository and click "Release" and confirm. 

Uploading Final Source and Binary Artifacts

# Create SVN folder and add the release artifacts there:
# https://dist.apache.org/repos/dist/dev/incubator/spark/spark-0.9.0-incubating-rc5
$ scp pwendell@people.apache.org:~/public_html/spark-0.9.0-incubating-rc5/* spark-0.9.0-incubating-rc5/
# Verify md5 sums
$ svn add spark-0.9.0-incubating
$ svn commit -m "Adding spark-0.8.1-incubating-rc1" 
$ svn mv https://dist.apache.org/repos/dist/dev/incubator/spark/spark-0.8.1-incubating-rc4 \  
>    https://dist.apache.org/repos/dist/release/incubator/spark/spark-0.8.1-incubating
# Look at http://www.apache.org/dist/incubator/spark/ to make sure it's there.
# This will be mirrored throughout the Apache network.

Packaging and Wrap-Up for the Release

• Update remaining version numbers in the release branch (see this example commit)
• Update the spark-ec2 scripts
  • Upload the binary packages to the spark-related-packages bucket in S3 and make them public
  • Alter the init scripts in amplab/spark-ec2 repository to pull new binaries (see this example commit)
  • You can audit the ec2 set-up by launching a cluster and running this audit script
• Update the Spark website
  • The website repo is at: https://svn.apache.org/repos/asf/incubator/spark
  • Copy new documentation to /site/docs and update the "latest" link
  • NOTE: For the below items, look at how previous releases are documented on the site
  • Create release notes
  • Update documentation page
  • Update downloads page
  • Update the main page with a news item
• Once everything is working (ec2, website docs, website changes) create an announcement on the website and then send an e-mail to the mailing list
• Enjoy an adult beverage of your choice, congrats on making a Spark release