Page History

This document covers the process for managing Spark releasesdetails the steps required in cutting a Spark release. This was last updated on 11/12/14 for the 1.1.1 release.

Prerequisites

...

Pre-prerequisites

• Must have an Apache account.
• Make sure you can access your Apache web space. Try sshApache Account. Try SSH-ing into <USER_NAME>@people<USER>@people.apache.org.
• Set up password-less access by uploading your public key.
• You must have passwordless SSH access from the release machine to your account. To enable this, upload your public key here: http://id.apache.org. If there is not already a public_html folder in your home directory, be sure to create one.
  
• Access to Apache Nexus. You will need this for publishing artifacts. Try logging into http://repository.apache.org with your Apache username and password. If you do not have access, create an INFRA ticket to request it.

• Git Push Access. You will need push access to https://git-wip-us.apache.org/repos/asf/spark.git. Additionally, make sure your git username and email are set on the machine you plan to run the release on.

  Code Block
  language bash
  $ git config --global user.name <your name> $ git config --global user.email <your email>

  
  

• EC2 Instance (Highly Recommended). The process of cutting a release requires a number of tools to be locally installed (maven, jekyll, SBT etc). It may be most convenient to use a EC2 instance based on the ami-e9eda8d9 (available is US-West). This has all the necessary tools pre-installed. Consider using compute-optimized instances (e.g. c3.4xlarge). Mac users are especially recommended to use a EC2 instance instead of attempting to install all the necessary tools. If you want to prepare your own EC2 instance, follow the steps given in the Miscellaneous section (see at the end of this document).
• Up-to-date Tools. Ensure that your release machine is using at least the following versions of the required tools: Maven 3.0.4, Java 6 AND Java 7, Jekyll 1.4.3, SBT 0.13.5, Git 1.8.3. If you are using the provided AMI, the correct versions should already be installed, with the possible exception of SBT. Note that it is particularly important to use the correct version of SBT to avoid inconsistent behavior. It is likely that apt-get will install an outdated version, so it’s recommended to get it directly from http://www.scala-sbt.org/0.13/tutorial/Installing-sbt-on-Linux.htmlCreate a folder "public_html" under your home directory on people.apache.org.

Create a GPG Key (https://www.apache.org/dev/release-signing)

# ---- Install GPG ----
# For Ubuntu, install through apt-get
$ sudo apt-get install gnupg
# For Mac OSX, install GPG Suite from http://gpgtools.org

# ---- Generate key ----
$ gpg --gen-key                   # Create new key, make sure it is RSA and 4096 bits (see https://www.apache.org/dev/openpgp.html#generate-key)
$ gpg --output <KEY_ID>.asc --export -a <KEY_ID>  # Generate public key file for distribution to Apache infrastructure

# ---- Distribute key ----
$ gpg --send-key <KEY_ID>         # Distribute public key to a key server, <KEY_ID> is the 8 HEX characters in the output of the previous command "pub  4096R/<KEY_ID> "
$ gpg --fingerprint               # Get key digest
# Open http://id.apache.org , login with Apache account and upload the key digest
$ scp <KEY_ID>.asc <USER_NAME>@people.apache.org:~/   # Copy generated <KEY_ID>.asc to Apache web space
# Create an FOAF file and add it via svn (see http://people.apache.org/foaf/ )
#   - should include key fingerprint
# Eventually key will show up on apache people page (e.g. https://people.apache.org/keys/committer/pwendell.asc )

...