...

Update CHANGES.txt in master to replace Release X.Y.0 (unreleased) with Release X.Y.0 - (YYYY-MM-DD). Commit:
noformat
Code Block
language bash
git add CHANGES.txt git commit -m "Preparing for release X.Y.0"
Create a branch for the release series (ensure origin/master is up-to-date with the ASF repo before running it):
noformat
Code Block
language
bash
git checkout -b branch-X.Y origin/master
Switch back to master and add A.B.C-SNAPSHOT (unreleased changes) to CHANGES.txt.
noformat
Code Block
language
bash
git checkout - (edit CHANGES.txt)
Bump the version number in the master branch:
noformat
Code Block
language
bash
./gradlew setversion -Pnextversion="A.B.C-SNAPSHOT"

Commit these changes to the master and push.

noformat

Code Block

language	bash

git add CHANGES.txt (and other updated files)
git commit -m "Bumping up the version to A.B.C-SNAPSHOT"
git push upstream master (assuming that ASF repo is referred to as 'upstream')

Checkout the release branch
noformat
Code Block
language
bash
git checkout branch-X.Y
Update the release branch's version information: the version number in the release branch ends in -SNAPSHOT, but we need to remove this for the release. For example, 0.8.0-SNAPSHOT needs to be changed to 0.8.0.
noformat
Code Block
language
bash
./gradlew setversion -Pnextversion="X.Y.Z"

Rename docker images and repo URLs:

noformat

Code Block

language

bash

grep -R --color :trunk- * | grep yaml
grep -R --color "http://repos.bigtop.apache.org/releases/X.Y.Z" *
grep -R --color "http://repos.bigtop.apache.org/releases/x.y.z" * (where "x.y.z" is the previous version of "X.Y.Z". Just in case)
(replace the occurrences found by the above commands)

Commit these changes to the release branch and push

noformat

Code Block

language

bash

git add (updated files)
git commit -m "Changing version to X.Y.Z"
git push upstream branch-X.Y (assuming that ASF repo is referred to as 'upstream')

Also, update the default version of Bigtop (and your code signing key ID, if needed) defined in hieradata on master and the release branch. For example:

noformat

Code Block

language

bash

$ git checkout master

(edit bigtop-deploy/puppet/hieradata/bigtop/repo.yaml)

$ git diff
diff --git a/bigtop-deploy/puppet/hieradata/bigtop/repo.yaml b/bigtop-deploy/puppet/hieradata/bigtop/repo.yaml
index 48c8a4ae..0d6a9126 100644
--- a/bigtop-deploy/puppet/hieradata/bigtop/repo.yaml
+++ b/bigtop-deploy/puppet/hieradata/bigtop/repo.yaml
@@ -1,5 +1,5 @@
 bigtop::bigtop_repo_gpg_check: true
-bigtop::bigtop_repo_apt_key: "BB95B97B18226C73CB2838D1DBBF9D42B7B4BD70"
+bigtop::bigtop_repo_apt_key: "01621A73025BDCA30F4159C62922A48261524827"
 bigtop::bigtop_repo_yum_key_url: "https://downloads.apache.org/bigtop/KEYS"
-bigtop::bigtop_repo_default_version: "1.4.0"
+bigtop::bigtop_repo_default_version: "1.5.0"

$ git add bigtop-deploy/puppet/hieradata/bigtop/repo.yaml 
$ git commit -m "Update default values in hieradata for release 1.5.0"
[master 9715ca7a] Update default values in hieradata for release 1.5.0
 1 file changed, 2 insertions(+), 2 deletions(-)
$ git push upstream master

$ git checkout branch-1.5 
$ git cherry-pick -x 9715ca7a
$ git push upstream branch-1.5

Shall you need to commit additional fixes into ongoing release, the commits should go to the release branch and only then be merged into master. Doing this other way around forces git cherry-pick which leads to discrepancies in the commit hash-codes and makes the branch look untidy and hard to follow.

3. Generate the Release Notes

For a patch release, just do the following four items on the branch-X.Y, instead of doing all of the items above:

Update the release branch's version information
Rename docker images and repo URLs
Update the default version of Bigtop
Commit these changes to the release branch and push

3. Generate the Release Notes

JIRA has the ability to generate release notes automatically (this is why it is so important to keep the fix version number accurate).

...

Manually check this list for accuracy! , since JIRA can be updated after step 1, even if it's unintentional. Also, I've repeatedly seen closed bugs that were not fixed (i.e., duplicate) marked with a fix version, so that they incorrectly show up in this list. Find those, edit them to remove the fix release (only actually fixed bugs should have a fix release) and re-run the report. A better way to deal with it is to run

...

Tag, and push the changes and the tag to git:

noformat

Code Block

language

bash

git tag release-x.y.z-RC0 -m "Bigtop X.Y.Z release."
git push upstream release--tags x.y.z-RC0

5. Build and run Package and Smoke Tests

...

Create a release specific job to build images such as Docker-Puppet-x.y.z, Docker-Toolchain-x.y.z and Docker-Toolchain-x.y.z-pull by copying existing jobs:

https://ci.bigtop.apache.org/view/Docker/job/Docker-

...

Puppet-1.4.0/

Make sure all the built images are uploaded to Dockerhub

https://

...

ci.bigtop.

...

apache.

...

org/

...

view/

...

Docker/

...

5.2. Build RPM/DEB packages

Make sure there is enough free space on the disk of Jenkins worker nodes. Remove workspaces and Docker images for old releases if you need to reclaim the free space.

Make sure all the built images are uploaded to Dockerhub after running the job.Create a release specific job to build packages:

Make sure all the built images are downloaded on docker-slave-06 and docker-slave-07 (by Docker-Toolchain-x.y.z-pull).

5.2. Build RPM/DEB packages

Create a release specific job to build packages:

https://ci.bigtop.apache.org/view/Releases/job/Bigtop-1.4.0/

Download all built packages ci.bigtop.apache.org/view/Releases/job/Bigtop-1.4.0/Download all built packages via archive download feature provided by Jenkins on a machine that you want to proceed the signing:

Code Block

language	bash

PLATFORM=amd64
#PLATFORM=aarch64  # forenable allwhen OSrunning andon archthe supportedARM in Bigtop
for i in centos-7 fedora-26 opensuse-42.3 debian-9 ubuntu-16.04; do \
  for j in amd64-slave aarch64-slave ppc64le-slave; do \
    wget https://ci.bigtop.apache.org/view/Releases/job/Bigtop-1.4.0/DISTRO=${i},PLATFORM=${j}/lastSuccessfulBuild/artifact/*zip*/archive.zip
  done
done

5.3. Sign RPM packages and yum repos

Ref:

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	BIGTOP-2736

Startup a docker images that is RPM based system:

No Format
cd ~/ # Change the image for AARCH64 or PPC64LE docker run -ti --rm -v $PWD:/tmp bigtop/puppet:1.4.0-centos-7 bash

Prepare the environment for signing:

No Format
gpg --import YOUR_CODE_SIGNING_SECRET_KEY echo "%_gpg_name YOUR_CODE_SIGNING_KEY_ID" > ~/.rpmmacros yum install -y rpm-sign createrepo

Signing:

No Format

cd /tmp
OS=centos-7
 
# Sign all RPM packages (This step required to input passphrase, so don't copy and paste the entire script here)
rpm --addsign `find ${OS} -name \*rpm`
 
# Recreate the metadata for repository
createrepo ${OS}
 
# Armor the metadata
gpg --detach-sign --armor ${OS}/repodata/repomd.xml

[OPENSUSE ONLY]

No Format

gpg --armor --export evansye@apache.org > opensuse-42.3/repodata/repomd.xml.key
for i in `find opensuse-42.3/repodata -name *.xml.gz` opensuse-42.3/repodata/repomd.xml.key ; do gpg --detach-sign --armor $i ; done

5.4. Sign DEB packages and apt repos

Ref:

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	BIGTOP-2736

, https://manpages.debian.org/jessie/dpkg-sig/dpkg-sig.1.en.html

Startup a docker images that is DEB based system:

No Format
cd ~/ docker run -ti --rm -v $PWD:/tmp bigtop/puppet:1.4.0-debian-9 bash

Prepare the environment for signing:

No Format
apt-get update apt-get install -y gpg apt-get install -y libterm-readkey-perl apt-get install -y dpkg-sig apt-get install -y reprepro gpg --import YOUR_CODE_SIGNING_SECRET_KEY

Signing:

platform
#PLATFORM=ppc64le  # enable when running on the PowerPC platform

# Download artifacts for all OS supported in Bigtop.
# Note that you should run this on each platform (x86_64, ARM, and PowerPC), because
# you will sign the artifacts and create repository in Docker container later,
# which are dependent on the platform.
for DISTRO in centos-7 fedora-26 opensuse-42.3 debian-9 ubuntu-16.04; do
  curl -L https://ci.bigtop.apache.org/view/Releases/job/Bigtop-1.4.0/DISTRO=${DISTRO},PLATFORM=${PLATFORM}-slave/lastSuccessfulBuild/artifact/*zip*/archive.zip -o ${DISTRO}.zip
done

# Extract the downloaded archive. If you come across the 
# "invalid zip file with overlapped components (possible zip bomb)"
# error, use other tools than unzip, e.g., `jar xf "${i}.zip"`
for i in centos-7 fedora-26 opensuse-42.3 debian-9 ubuntu-16.04; do
  unzip "${i}.zip"
  mkdir "${i}"
  mv archive/output/* "${i}"
  rm -rf archive
done

5.3. Sign RPM packages and yum repos

Ref:

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	BIGTOP-2736

Startup a docker images that is RPM based system:

Code Block

language	bash

cd ~/
# Change the image for AARCH64 or PPC64LE
docker run -ti --rm -v $PWD:/work bigtop/puppet:1.4.0-centos-7 bash

Prepare the environment for signing:

Code Block

language	bash

yum install -y rpm-sign createrepo pinentry
gpg --import YOUR_CODE_SIGNING_SECRET_KEY
echo "%_gpg_name YOUR_CODE_SIGNING_KEY_ID" > ~/.rpmmacros

Signing:

Code Block

language	bash

cd /work
OS=centos-7
 
# Sign all RPM packages (This step required to input passphrase, so don't copy and paste the entire script here)
rpm --addsign `find ${OS} -name \*rpm`
 
# Recreate the metadata for repository
createrepo ${OS}
 
# Armor the metadata
gpg --detach-sign --armor ${OS}/repodata/repomd.xml

[OPENSUSE ONLY]

Code Block

language	bash

gpg --armor --export evansye@apache.org > opensuse-42.3/repodata/repomd.xml.key
for i in `find opensuse-42.3/repodata -name *.xml.gz` opensuse-42.3/repodata/repomd.xml.key ; do gpg --detach-sign --armor $i ; done

5.4. Sign DEB packages and apt repos

Ref:

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	BIGTOP-2736

, https://manpages.debian.org/jessie/dpkg-sig/dpkg-sig.1.en.html

Startup a docker images that is DEB based system:

Code Block

language	bash

cd ~/
# Change the image for AARCH64 or PPC64LE
docker run -ti --rm -v $PWD:/work bigtop/puppet:1.4.0-debian-9 bash

Prepare the environment for signing:

Code Block

language	bash

apt-get update
apt-get install -y gpg libterm-readkey-perl dpkg-sig reprepro
gpg --import YOUR_CODE_SIGNING_SECRET_KEY

Signing:

Code Block

language	bash

cd /work
VERSION=1.4.0
OS=debian-9
ARCH=amd64
#ARCH=arm64
#ARCH=ppc64el
SIGN_KEY=B7B4BD70
export GPG_TTY=$(tty)
 
# Sign DEB packages (This step required to input passphrase, so don't copy and paste the entire script here)
dpkg-sig --cache-passphrase --sign builder --sign-changes force_full

No Format

cd /tmp
VERSION=1.4.0
OS=debian-9
ARCH=amd64
SIGN_KEY=B7B4BD70
export GPG_TTY=$(tty)
 
# Sign DEB packages (This step required to input passphrase, so don't copy and paste the entire script here)
dpkg-sig --cache-passphrase --sign builder  `find ${OS}/ -name \*deb`
 
# Build signed apt repository
mkdir -p conf
cat > conf/distributions <<__EOT__
Origin: Bigtop
Label: Bigtop
Suite: stable
Codename: bigtop
Version: ${VERSION}
Architectures: ${ARCH} source
Components: contrib
Description: Apache Bigtop
SignWith: ${SIGN_KEY}
__EOT__
 
cat > conf/options <<__EOT__
verbose
ask-passphrase
__EOT__

rm -rf ${OS}/apt
reprepro --ask-passphrase -Vb . includedeb bigtop `find ${OS}/ -name \*deb`

5.5. Upload to S3

The easiest way to upload artifacts to S3 is via your own AWS account. Add your account(email) to bigtop's bucket in the section "Access for other AWS accounts":

Image Removed

Once permission granted, you're able to use your account's access key and secret key with aws s3 sync command for upload:

No Format
aws s3 sync --acl public-read ./ubuntu-16.04/ s3://repos.bigtop.apache.org/releases/1.4.0/ubuntu/16.04/amd64/

The directory layouts on S3 bucket looked like the following:


 
# Build signed apt repository
mkdir -p conf
cat > conf/distributions <<__EOT__
Origin: Bigtop
Label: Bigtop
Suite: stable
Codename: bigtop
Version: ${VERSION}
Architectures: ${ARCH} source
Components: contrib
Description: Apache Bigtop
SignWith: ${SIGN_KEY}
__EOT__
 
cat > conf/options <<__EOT__
verbose
ask-passphrase
__EOT__

rm -rf ${OS}/apt
# Note that this command creates the apt repository (db, dists, pool)
# in the current directory, so you can't ran it for multiple distros in parallel.
reprepro --ask-passphrase -Vb . includedeb bigtop `find ${OS}/ -name \*deb`
rm -rf ./${OS}/*
mv conf db dists pool ${OS}

5.5. Upload to S3

The easiest way to upload artifacts to S3 is via your own AWS account. Add your account(email) to bigtop's bucket in the section "Access for other AWS accounts":

Image Added

Once permission granted, you're able to use your account's access key and secret key with aws s3 sync command for upload, like:

Code Block

language	bash

# In case of CentOS 7
aws s3 sync --acl public-read ./centos-7/ s3://repos.bigtop.apache.org/releases/1.4.0/centos/7/x86_64/   # for x86_64/amd64
aws s3 sync --acl public-read ./centos-7/ s3://repos.bigtop.apache.org/releases/1.4.0/centos/7/aarch64/  # for aarch64/arm64
aws s3 sync --acl public-read ./centos-7/ s3://repos.bigtop.apache.org/releases/1.4.0/centos/7/ppc64le/  # for ppc64le/ppc64el

# In case of Ubuntu 16.04
aws s3 sync --acl public-read ./ubuntu-16.04/ s3://repos.bigtop.apache.org/releases/1.4.0/ubuntu/16.04/amd64/    # for x86_64/amd64
aws s3 sync --acl public-read ./ubuntu-16.04/ s3://repos.bigtop.apache.org/releases/1.4.0/ubuntu/16.04/arm64/    # for aarch64/arm64
aws s3 sync --acl public-read ./ubuntu-16.04/ s3://repos.bigtop.apache.org/releases/1.4.0/ubuntu/16.04/ppc64el/  # for ppc64le/ppc64el

The directory layouts on S3 bucket looked like the following:

Code Block

repos.bigtop.apache.org/releases/1.4.0/centos/7/x86_64
repos.bigtop.apache.org/releases/1.4.0/fedora/26/x86_64
repos.bigtop.apache.org/releases/1.4.0/opensuse/42.3/x86_64
repos.bigtop.apache.org/

No Format

repos.bigtop.apache.org/releases/1.4.0/centos/7/x86_64
repos.bigtop.apache.org/releases/1.4.0/fedora/26/x86_64
repos.bigtop.apache.org/releases/1.4.0/opensuse/42.3/x86_64
repos.bigtop.apache.org/releases/1.4.0/debian/9/amd64
repos.bigtop.apache.org/releases/1.4.0/ubuntu/16.04/amd64

For YUM repos, upload files /tmpwork/centos-7/* into into repos.bigtop.apache.org/releases/1.4.0/centos/7/x86_64/. For example:

No Formatcode
repos.bigtop.apache.org/releases/1.4.0/centos/7/x86_64/alluxio repos.bigtop.apache.org/releases/1.4.0/centos/7/x86_64/ambari ... repos.bigtop.apache.org/releases/1.4.0/centos/7/x86_64/repodata ...

For APT repos, upload files /tmpwork/debian-9/* into into repos.bigtop.apache.org/releases/1.4.0/debian/9/amd64/. For example:

No Formatcode

repos.bigtop.apache.org/releases/1.4.0/debian/9/amd64/conf
repos.bigtop.apache.org/releases/1.4.0/debian/9/amd64/db
repos.bigtop.apache.org/releases/1.4.0/debian/9/amd64/dists
repos.bigtop.apache.org/releases/1.4.0/debian/9/amd64/pool

5.6. Create repo files

Create one for each of our Distro. Following are examples for YUM and APT: one for each of our Distro. Following are examples for YUM and APT:

Code Block

language	bash

VERSION=1.4.0
RC_NUMBER=0
DIR=bigtop-${VERSION}-RC${RC_NUMBER}/repos
mkdir -p ${DIR}
cd ${DIR}

# YUM
for OS in centos-7 fedora-26 opensuse-42.3; do

No Format

# YUM
cat > bigtop.repo <<__EOT__
[bigtop]
name=Bigtop
enabled=1
gpgcheck=1
baseurl=http://repos.bigtop.apache.org/releases/1.4.0/centos/7/x86_64/${VERSION}/${OS/-/\/}/\$basearch
gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
__EOT__
 
  gpg --detach-sign --armor-sign --armor bigtop.repo
  mkdir "${OS//\/}"
  mv bigtop.repo
 * "${OS//\/}"
done

# APT
for OS in debian-9 ubuntu-16.04; do
  cat > bigtop.list <<__EOT__
deb http://repos.bigtop.apache.org/releases/1.4.0/debian/9/amd64	/${VERSION}/${OS/-/\/}/\$(ARCH) bigtop contrib
__EOT__
 
  gpg --detach-sign --armorarmor bigtop.list
  mkdir "${OS//\/}"
  mv bigtop.list* "${OS//\/}"
done

Add your signed armored GPG key to repos directory to ease the key import for the users

noformat

Code Block

language

bash

gpg --armor --export <your name> >> GPG-KEY-bigtop
gpg --detach-sign --armor GPG-KEY-bigtop

Result looks like below:

No Formatcode
GPG-KEY-bigtop GPG-KEY-bigtop.asc centos7/ debian9/ fedora26/ opensuse42.3/ ubuntu16.04/

...

First we need to prepare a build environment. Mac OS X is unlikely supported because we'll run through some tests that depends on the OS. Here I'm running on Ubuntu-18.04 (There's an issue for running on 16.04 traced by BIGTOP-2830).

noformat

Code Block

language	bash

cd ~/
docker run -ti -u jenkins -v $PWD:/tmpwork bigtop/slaves:trunk-ubuntu-18.04 bash -l
# The subsequence commands should be executed inside the docker container
cp -r /tmpwork/.gnupg ~/

Create a maven settings file ~/.m2/settings.xml with the following content:

No Formatcode

<settings>
   <servers>
      <server>
         <id>apache.snapshots.https</id>
         <username>APACHE-ID</username>
         <password>APACHE-PASSWORD</password>
      </server>
      <server>
         <id>apache.staging.https</id>
         <username>APACHE-ID</username>
         <password>APACHE-PASSWORD</password>
      </server>
      <server>
         <id>apache.releases.https</id>
         <username>APACHE-ID</username>
         <password>APACHE-PASSWORD</password>
      </server>
   </servers>
   <profiles>
      <profile>
         <id>gpg</id>
         <properties>
            <gpg.executable>gpg</gpg.executable>
            <gpg.passphrase>GPG-PASSWORD</gpg.passphrase>
         </properties>
      </profile>
   </profiles>
   <activeProfiles>
      <activeProfile>gpg</activeProfile>
   </activeProfiles>
</settings>

Build the artifacts:

noformat

Code Block

language

bash

mvn clean site
mvn -Prelease package assembly:single

The following command deploys the binary release artifacts for iTest, tests and other helper files, checksums, and signatures (you will need to enter a GPG passphrase. If you got a "gpg: signing failed: Inappropriate ioctl for device" error, try `export GPG_TTY=$(tty)`) to the Apache Staging repo.

noformat

Code Block

language

bash

mvn deploy -Prelease -f pom.xml
mvn deploy -Prelease -f bigtop-test-framework/pom.xml
mvn deploy -Prelease -f bigtop-tests/test-artifacts/pom.xml
HADOOP_CONF_DIR=/etc/hadoop/conf HADOOP_HOME=/usr/lib/hadoop mvn deploy -Prelease -f bigtop-tests/test-execution/pom.xml -DskipITs

...

If you have multiple keys, the build process seems to pick up the first one w/o asking. Make sure you're using the CODE SIGNING KEY by explicitly specifying it. For example:

noformat

Code Block

language

bash

mvn deploy -Prelease -f pom.xml -Dgpg.keyname="Evans Ye (CODE SIGNING KEY) <evansye@apache.org>"
mvn deploy -Prelease -f bigtop-test-framework/pom.xml -Dgpg.keyname="Evans Ye (CODE SIGNING KEY) <evansye@apache.org>"
mvn deploy -Prelease -f bigtop-tests/test-artifacts/pom.xml -Dgpg.keyname="Evans Ye (CODE SIGNING KEY) <evansye@apache.org>"
mvn deploy -Prelease -f bigtop-tests/test-execution/pom.xml -Dgpg.keyname="Evans Ye (CODE SIGNING KEY) <evansye@apache.org>"

...

The artifacts that end up in the distribution directory are the source distributions (along with their checksums and signatures), so they need to be copied from the Maven repo to a release candidate directory on apache dist, so the vote can begin:

noformat

Code Block

language

bash

VERSION=X.Y.Z
RC_NUMBER=n
REPOSITORY_ID=xxxx # Should be a number, for example 1013. Find the number in NEXUS staging repository: orgapachebigtop-1013

svn checkout https://dist.apache.org/repos/dist/dev/bigtop bigtop-dist-dev
cd bigtop-dist-dev

mkdir bigtop-${VERSION}-RC${RC_NUMBER}
cd bigtop-${VERSION}-RC${RC_NUMBER}
# md5 and sha1 are out-of-date and should not be used, see: http://www.apache.org/dev/release-distribution#sigs-and-sums
wget --no-check-certificate https://repository.apache.org/content/repositories/orgapachebigtop-${REPOSITORY_ID}/org/apache/bigtop/bigtop/${VERSION}/bigtop-${VERSION}-project.tar.gz
wget --no-check-certificate https://repository.apache.org/content/repositories/orgapachebigtop-${REPOSITORY_ID}/org/apache/bigtop/bigtop/${VERSION}/bigtop-${VERSION}-project.tar.gz.asc
# manually generate sha256 and sha512
sha256sum bigtop-${VERSION}-project.tar.gz > bigtop-${VERSION}-project.tar.gz.sha256
sha512sum bigtop-${VERSION}-project.tar.gz > bigtop-${VERSION}-project.tar.gz.sha512

cd ..
svn add bigtop-${VERSION}-RC${RC_NUMBER}
svn ci -m "Apache Bigtop ${VERSION}-RC${RC_NUMBER}"

8. Sanity Check

noformat

Code Block

language

bash

VERSION=X.Y.Z
RC_NUMBER=n

cd /tmpwork
git clone https://gitbox.apache.org/repos/asf/bigtop.git -b release-${VERSION}-$RC${VERSIONRC_NUMBER} --depth 1
wget --no-parent --recursive --reject='index.html*' https://dist.apache.org/repos/dist/dev/bigtop/bigtop-${VERSION}-RC${RC_NUMBER}/

# make sure that the source code tree and the release tarball are identical except for .git/CI/site releated files
tar xf dist.apache.org/repos/dist/dev/bigtop/bigtop-${VERSION}-RC${RC_NUMBER}/bigtop-${VERSION}-project.tar.gz
diff -r bigtop bigtop-${VERSION}

# make sure that the signature and checksums are correct
cd dist.apache.org/repos/dist/dev/bigtop/bigtop-${VERSION}-RC${RC_NUMBER}
gpg --verify bigtop-${VERSION}-project.tar.gz.asc bigtop-${VERSION}-project.tar.gz
diff <(sha256sum bigtop-${VERSION}-project.tar.gz) bigtop-${VERSION}-project.tar.gz.sha256 
diff <(sha512sum bigtop-${VERSION}-project.tar.gz) bigtop-${VERSION}-project.tar.gz.sha512 

# make sure that all files exist and their signatures are correct
for i in $(find repos -not -name '*.asc' -type f); do echo ${i}; gpg --verify ${i}.asc ${i}; done

...

This step makes the artifacts available on the mirrors.

noformat

Code Block

language	bash

VERSION=X.Y.Z #Example: 1.3.0
CANDIDATE=C #Example: RC2
svn co https://dist.apache.org/repos/dist/release/bigtop
cd bigtop
svn mv https://dist.apache.org/repos/dist/dev/bigtop/bigtop-$VERSION-$CANDIDATE https://dist.apache.org/repos/dist/release/bigtop/bigtop-$VERSION
rm stable
ln -s bigtop-$VERSION stable
svn commit

...

Create permanent release tag under rel/

Code Block

language	bash

git checkout release-x.y.z
git tag rel/x.y.z -u <signing key ID>
git push --tags

Wait 24 Hours

...

Wait 24 Hours

~~It takes up to 24 hours for all the mirrors to sync, so don't announce the new release just yet.~~

We had to wait 24 hours before, but it's not true now since ASF adopted CDN instead of mirror sites. According to the FAQs of the ASF infra:

Apache uses a global content distribution network (CDN) which collects new releases almost as soon as you post them.
The files therefore become available for download almost immediately. You probably don't need to wait more than fifteen minutes before announcing a release.

Build and Deploy Site

Full version of this procedure is described and maintained in Deploying Bigtop's Apache Website

No Format
mvn site-deploy

This will deploy the generated website to /www/bigtop.apache.org/ on people.apache.org. You can SSH there to check that things look OK.Once the deployment is done you need to publish new website via https://cms.apache.org/bigtop/: login and click "Publish bigtop site" link. Otherwise the content won't get propagated.

It will take an hour or so for the website to become live, but you can inspect the website by temporarily setting your browser proxy to see the new, unmirrored content (I used FoxyProxy in Firefox) as described in http://www.apache.org/dev/project-site.html.

Update files under src/site by referring to past commits such as https://github.com/apache/bigtop/commit/678375eb75f7367f9114a3ad8087ba40f21444b6 or https://github.com/apache/bigtop/commit/d5c45a2879f8e3a8d29a36386ad8904a4e71a274. Commit into master and push them.

Login to the Jenkins server provided by ASF Infra. Open our website publishing pipeline and run it from the "Build Now" menu. It checks out the master branch with your update from our git repository, runs mvn site, copies generated webpages into the asf-site branch, and commits them. It automatically updates our website thanks to .asf.yaml.

(If the above process doesn't seem to work correctly just like

Jira

server	ASF JIRA
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	BIGTOP-3467

, you can generate a website locally and push it to the asf-site branch directly as a workaround.)Remember to update download page accordingly.

As said in

Jira

server	ASF JIRA
serverId	5aa69414-a9e9-3523-82ec-879b028fb15b
key	BIGTOP-1162

, remove old releases to reduce the loading for apache mirror. The older releases are available in Apache archive server(older versions are archived automatically, see http://archive.apache.org/dist/bigtop/ and http://www.apache.org/dev/mirrors.html).

noformat

Code Block

language

bash

VERSION=X.Y.Z #Example: 1.2.1
svn co https://dist.apache.org/repos/dist/release/bigtop
cd bigtop
svn delete bigtop-$VERSION
svn commit

...

Page tree

Versions Compared

Old Version 140

New Version Current

Key

3. Generate the Release Notes

3. Generate the Release Notes

5. Build and run Package and Smoke Tests

5.2. Build RPM/DEB packages

5.2. Build RPM/DEB packages

5.3. Sign RPM packages and yum repos

5.4. Sign DEB packages and apt repos

5.3. Sign RPM packages and yum repos

5.4. Sign DEB packages and apt repos

5.5. Upload to S3

5.5. Upload to S3

5.6. Create repo files

8. Sanity Check

Wait 24 Hours

Wait 24 Hours

Build and Deploy Site

Page tree

Page History

Versions Compared

Old Version 140

New Version Current

Key

3. Generate the Release Notes

3. Generate the Release Notes

5. Build and run Package and Smoke Tests

5.2. Build RPM/DEB packages

5.2. Build RPM/DEB packages

5.3. Sign RPM packages and yum repos

5.4. Sign DEB packages and apt repos

5.3. Sign RPM packages and yum repos

5.4. Sign DEB packages and apt repos

5.5. Upload to S3

5.5. Upload to S3

5.6. Create repo files

8. Sanity Check

Wait 24 Hours

Wait 24 Hours

Build and Deploy Site