Page History

...

Cutting a Release Candidate

Cutting a release candidate involves a two steps. First, we use the Maven release plug-in to create a release commit (a single commit where all of the version files have the correct number) and publish the code associated with that release to a staging repository in Maven. Second, we check out that release commit and package binary releases and documentation.

Create and Stage a Release Candidate

Create / update CHANGES.txt

CHANGES.txt captures all the patches that have made it into this release candidate. It can be generated using this script. 

• Checkout the Spark release version in a Spark git repository. 
• Download the script to a location within the repo.
• Updated the previous release tag, and other information in the script.

• Set SPARK_HOME environment variable and run the script. 

  Code Block
  language bash
  $ export SPARK_HOME="..." $ python -u generate-changelist.py

Update JIRA

 If this is not the first RC, then make sure that JIRA issues that have been solved since the last RC (that is, they are going to make it to this new RC) are marked as fixed in this release version.

• A possible protocol for this is to mark such JIRA issues as fixed in next maintenance release. E.g. if you are cutting RC for 1.0.2, mark such issues as 1.0.3. Or for RC of 1.1, mark 1.1.1 .
• When cutting new RC, find all the issues that are marked as fixed for next maintenance release, and change them to the current release. Also verify from git log whether they are actually making it in the new RC or not.

Cut it!

The process of creating releases has been automated via this create release script. 

• Configure the script by specifying the Apache username + password and the Apache GPG key passphrase. BE CAREFUL to not to accidentally check them in.
• This script can be run in any directory.
• Make sure you have JAVA_HOME set, otherwise generation of pre-built packages with make-distribution.sh will fail, and you will have to run the script manually again (run with the option --package-only to generate the binary packages / tarballs)
• Make sure you have password-less access to Apache webspace (people.apache.org) from the machine you are running the script on. Otherwise uploading of binary tarballs and docs will fail and you will have upload them manually.
• Read and understand the script fully before you execute it. It will cut a Maven release, build binary releases and documentation, then copy the binary artifacts to a staging location on people.apache.org.
• NOTE: You must use git 1.7.X for this or else you'll hit this horrible bug.

If this is not the first RC, then make sure that the JIRA issues that have been solved since the last RC are marked as FIXED in this release version.

• A possible protocol for this is to mark such issues as FIXED in next maintenance release. E.g. if you are cutting RC for 1.0.2, mark such issues as FIXED in 1.0.3.
• When cutting new RC, find all the issues that are marked as FIXED for next maintenance release, and change them to the current release.
• Verify from git log whether they are actually making it in the new RC or not.

The process of cutting a release candidate has been automated via this script found in the Spark repository. First, run the following preliminary steps:

# This step is important to avoid confusion later
# when the script clones Spark with the generated tag
$ mv spark release-spark

# The distributions are packaged with Java 6 while
# the docs are built with Java 7 for nicer formatting
$ export JAVA_HOME=<Java 6 home>
$ export JAVA_7_HOME=<Java 7 home>

# Verify that the version on each tool is up-to-date
$ sbt --version # 0.13.5+
$ mvn --version # 3.0.4+
$ jekyll --version # 1.4.3+
$ git --version # 1.7+
$ $JAVA_HOME/bin/java -version # 1.6.x
$ $JAVA_7_HOME/bin/java -version # 1.7.x

It is highly recommended that you understand the contents of the script before proceeding. This script uses the Maven release plugin and can be broken down into four steps. In the likely event that one of the steps fails, you may restart from the step that failed instead of running the whole script again.

1. Run mvn release:prepare. This updates all pom.xml versions and cuts a new tag (e.g. 1.1.1-rc1). If this step is successful, you will find the remote tag here. You will also find the following commit pushed in your name in the release branch: [maven-release-plugin] prepare release v1.1.1-rc1 (see this example commit).
2. Run mvn release:perform. This builds Spark from the tag cut in the previous step using the spark/release.properties produced. If this step is successful, you will find the following commit pushed in your name in the release branch, but NOT in the release tag: [maven-release-plugin] prepare for the next development iteration (see this example commit). You will also find that the release.properties file is now removed.
3. Package binary distributions. This runs the make-distribution.sh script for each distribution in parallel. If this step is successful, you will find the archive, signing key, and checksum information for each distribution in the directory in which the create-release.sh script is run. You should NOT find a sub-directory named after one of the distributions as these should be removed. In case of failure, use the binary-release-*.log files generated to determine the cause. In the re-run, you may skip the previous steps and re-make only the distributions that failed by commenting out part of the script.
4. Compile documentation. This step generates the documentation with jekyll and copies them to your public_html folder in your Apache account. If this step is successful, you should be able to browse the docs under http://people.apache.org/~<USER> (see this example link).

Finally, run the script after filling in the variables at the top of the script. The information here is highly sensitive, so BE CAREFUL TO NOT ACCIDENTALLY CHECK THESE CHANGES IN! The GPG passphrase is the one you used to generate the key with.

$ cd .. # just so we don’t clone Spark in Spark
$ vim release-spark/dev/create-release/create-release.sh
$ release-spark/dev/create-release/create-release.sh

On a c3.4xlarge machine in us-west-2, this process is expected to take 2 - 4 hours. After the script has completed, you must find the open staging repository in Apache Nexus to which the artifacts were uploaded, and close the staging repository. Wait a few minutes for the closing to succeed. Now all staged artifacts are public!

 (Optional) In the event that you need to roll back the entire process and start again, you will need to run the following steps. This is necessary if, for instance, you used a faulty GPG key, new blockers arise, or the vote failed.

$ git tag -d <the new tag> # e.g. v1.1.1-rc1
$ git push origin :<the new tag>
$ git revert <perform release commit hash> # see this commit
$ git revert <prepare release commit hash> # see this commit
$ git push origin <release branch> # e.g. branch-1.1

Audit the Release Candidate

The process of auditing release has been automated via this script found in the Spark repository. First, find the staging repository in Apache Nexus to which the artifacts were uploaded (see this example repository). Configure the script by filling in the required variables at the top. This must be run from the directory that hosts the script.

# The script must be run from the audit-release directory
$ cd release-spark/dev/audit-release
$ vim audit-release.py
$ ./audit-release.py

 The release auditor will test example builds against the staged artifacts, verify signatures, and check for common mistakes made when cutting a release. This is expected to finish in less than an hour.

Note that it is entirely possible for the dependency requirements of the applications to be outdated. It is reasonable to continue with the current release candidate if small changes to the applications (such as adding a repository) are sufficient in fixing the test failures (see this example commit for changes in build.sbt files). Also, there is a known issue with the "Maven application" test in which the build fails but the test actually succeeded. This has been failing since 1.1.0.

 After script has completed, find the open staging repository in Apache Nexus to which the artifacts were uploaded to. Close the staging repository. Wait for the closing to succeed. Now all the staged artifacts are public!

Audit a Staged Release Candidate

The process of auditing release has been automated via this release audit script.

...