Versions Compared

Old Version 142

changes.mady.by.user Kengo Seki

Saved on

compared with

New Version 143

changes.mady.by.user Kengo Seki

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Create one for each of our Distro. Following are examples for YUM and APT:
noformat
Code Block
languagebash
VERSION=1.4.0
RC_NUMBER=0
DIR=bigtop-${VERSION}-RC${RC_NUMBER}/repos
mkdir -p ${DIR}
cd ${DIR}

# YUM
for OS in centos-7 fedora-26 opensuse-42.3; do
  cat > bigtop.repo <<__EOT__
[bigtop]
name=Bigtop
enabled=1
gpgcheck=1
baseurl=http://repos.bigtop.apache.org/releases/${VERSION}/${OS/-/\/}/\$basearch
gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
__EOT__
  gpg --detach-sign --armor bigtop.repo
  mkdir "${OS//\/}"
  mv bigtop.repo* "${OS//\/}"
done

# APT
for OS in debian-9 ubuntu-16.04; do
  cat > bigtop.list <<__EOT__
deb http://repos.bigtop.apache.org/releases/${VERSION}/${OS/-/\/}/\$(ARCH) bigtop contrib
__EOT__
  gpg --detach-sign --armor bigtop.list
  mkdir "${OS//\/}"
  mv bigtop.list* "${OS//\/}"
done

Add your signed armored GPG key to repos directory to ease the key import for the users

Code Blocknoformat
languagebash
gpg --armor --export <your name> >> GPG-KEY-bigtop
gpg --detach-sign --armor GPG-KEY-bigtop

Result looks like below:

No Formatcode
GPG-KEY-bigtop
GPG-KEY-bigtop.asc
centos7/
debian9/
fedora26/
opensuse42.3/
ubuntu16.04/

...

First we need to prepare a build environment. Mac OS X is unlikely supported because we'll run through some tests that depends on the OS. Here I'm running on Ubuntu-18.04 (There's an issue for running on 16.04 traced by BIGTOP-2830).

noformat
Code Block
language
bash
cd ~/
docker run -ti -u jenkins -v $PWD:/tmp bigtop/slaves:trunk-ubuntu-18.04 bash -l
# The subsequence commands should be executed inside the docker container
cp -r /tmp/.gnupg ~/

Create a maven settings file ~/.m2/settings.xml with the following content:

No Formatcode
<settings>
   <servers>
      <server>
         <id>apache.snapshots.https</id>
         <username>APACHE-ID</username>
         <password>APACHE-PASSWORD</password>
      </server>
      <server>
         <id>apache.staging.https</id>
         <username>APACHE-ID</username>
         <password>APACHE-PASSWORD</password>
      </server>
      <server>
         <id>apache.releases.https</id>
         <username>APACHE-ID</username>
         <password>APACHE-PASSWORD</password>
      </server>
   </servers>
   <profiles>
      <profile>
         <id>gpg</id>
         <properties>
            <gpg.executable>gpg</gpg.executable>
            <gpg.passphrase>GPG-PASSWORD</gpg.passphrase>
         </properties>
      </profile>
   </profiles>
   <activeProfiles>
      <activeProfile>gpg</activeProfile>
   </activeProfiles>
</settings>

Build the artifacts:

noformat
Code Block
languagebash
mvn clean site
mvn -Prelease package assembly:single

The following command deploys the binary release artifacts for iTest, tests and other helper files, checksums, and signatures (you will need to enter a GPG passphrase. If you got a "gpg: signing failed: Inappropriate ioctl for device" error, try `export GPG_TTY=$(tty)`) to the Apache Staging repo.

noformat
Code Block
language
bash
mvn deploy -Prelease -f pom.xml
mvn deploy -Prelease -f bigtop-test-framework/pom.xml
mvn deploy -Prelease -f bigtop-tests/test-artifacts/pom.xml
mvn deploy -Prelease -f bigtop-tests/test-execution/pom.xml -DskipITs

...

If you have multiple keys, the build process seems to pick up the first one w/o asking. Make sure you're using the CODE SIGNING KEY by explicitly specifying it. For example:

Code Blocknoformat
languagebash
mvn deploy -Prelease -f pom.xml -Dgpg.keyname="Evans Ye (CODE SIGNING KEY) <evansye@apache.org>"
mvn deploy -Prelease -f bigtop-test-framework/pom.xml -Dgpg.keyname="Evans Ye (CODE SIGNING KEY) <evansye@apache.org>"
mvn deploy -Prelease -f bigtop-tests/test-artifacts/pom.xml -Dgpg.keyname="Evans Ye (CODE SIGNING KEY) <evansye@apache.org>"
mvn deploy -Prelease -f bigtop-tests/test-execution/pom.xml -Dgpg.keyname="Evans Ye (CODE SIGNING KEY) <evansye@apache.org>"

...

The artifacts that end up in the distribution directory are the source distributions (along with their checksums and signatures), so they need to be copied from the Maven repo to a release candidate directory on apache dist, so the vote can begin:

Code Blocknoformat
languagebash
VERSION=X.Y.Z
RC_NUMBER=n
REPOSITORY_ID=xxxx # Should be a number, for example 1013. Find the number in NEXUS staging repository: orgapachebigtop-1013

svn checkout https://dist.apache.org/repos/dist/dev/bigtop bigtop-dist-dev
cd bigtop-dist-dev

mkdir bigtop-${VERSION}-RC${RC_NUMBER}
cd bigtop-${VERSION}-RC${RC_NUMBER}
# md5 and sha1 are out-of-date and should not be used, see: http://www.apache.org/dev/release-distribution#sigs-and-sums
wget --no-check-certificate https://repository.apache.org/content/repositories/orgapachebigtop-${REPOSITORY_ID}/org/apache/bigtop/bigtop/${VERSION}/bigtop-${VERSION}-project.tar.gz
wget --no-check-certificate https://repository.apache.org/content/repositories/orgapachebigtop-${REPOSITORY_ID}/org/apache/bigtop/bigtop/${VERSION}/bigtop-${VERSION}-project.tar.gz.asc
# manually generate sha256 and sha512
sha256sum bigtop-${VERSION}-project.tar.gz > bigtop-${VERSION}-project.tar.gz.sha256
sha512sum bigtop-${VERSION}-project.tar.gz > bigtop-${VERSION}-project.tar.gz.sha512

cd ..
svn add bigtop-${VERSION}-RC${RC_NUMBER}
svn ci -m "Apache Bigtop ${VERSION}-RC${RC_NUMBER}"

8. Sanity Check

noformat
Code Block
language
bash
VERSION=X.Y.Z
RC_NUMBER=n

cd /tmp
git clone https://gitbox.apache.org/repos/asf/bigtop.git -b release-${VERSION} --depth 1
wget --no-parent --recursive --reject='index.html*' https://dist.apache.org/repos/dist/dev/bigtop/bigtop-${VERSION}-RC${RC_NUMBER}/

# make sure that the source code tree and the release tarball are identical except for .git/CI/site releated files
tar xf dist.apache.org/repos/dist/dev/bigtop/bigtop-${VERSION}-RC${RC_NUMBER}/bigtop-${VERSION}-project.tar.gz
diff -r bigtop bigtop-${VERSION}

# make sure that the signature and checksums are correct
cd dist.apache.org/repos/dist/dev/bigtop/bigtop-${VERSION}-RC${RC_NUMBER}
gpg --verify bigtop-${VERSION}-project.tar.gz.asc bigtop-${VERSION}-project.tar.gz
diff <(sha256sum bigtop-${VERSION}-project.tar.gz) bigtop-${VERSION}-project.tar.gz.sha256 
diff <(sha512sum bigtop-${VERSION}-project.tar.gz) bigtop-${VERSION}-project.tar.gz.sha512 

# make sure that all files exist and their signatures are correct
for i in $(find repos -not -name '*.asc' -type f); do echo ${i}; gpg --verify ${i}.asc ${i}; done

...

This step makes the artifacts available on the mirrors.

noformat
Code Block
languagebash
VERSION=X.Y.Z #Example: 1.3.0
CANDIDATE=C #Example: RC2
svn co https://dist.apache.org/repos/dist/release/bigtop
cd bigtop
svn mv https://dist.apache.org/repos/dist/dev/bigtop/bigtop-$VERSION-$CANDIDATE https://dist.apache.org/repos/dist/release/bigtop/bigtop-$VERSION
rm stable
ln -s bigtop-$VERSION stable
svn commit

...

Create permanent release tag under rel/

Code Block
languagebash
git checkout release-x.y.z
git tag rel/x.y.z -u <signing key ID>
git push --tags

...

As said in 

Jira
serverASF JIRA
serverId5aa69414-a9e9-3523-82ec-879b028fb15b
keyBIGTOP-1162
, remove old releases to reduce the loading for apache mirror. The older releases are available in Apache archive server(older versions are archived automatically, see http://archive.apache.org/dist/bigtop/ and http://www.apache.org/dev/mirrors.html).

noformat
Code Block
language
bash
VERSION=X.Y.Z #Example: 1.2.1
svn co https://dist.apache.org/repos/dist/release/bigtop
cd bigtop
svn delete bigtop-$VERSION
svn commit

...