...
This
...
document
...
describes
...
how
...
to
...
make
...
a
...
release
...
of
...
Flume.
...
It
...
is
...
a
...
work
...
in
...
progress
...
and
...
should
...
be
...
refined
...
by
...
the
...
...
...
(RM)
...
as
...
they
...
come
...
across
...
aspects
...
of
...
the
...
release
...
process
...
not
...
yet
...
documented
...
here.
...
NOTE:
...
For
...
the
...
purpose
...
of
...
illustration,
...
this
...
document
...
assumes
...
that
...
the
...
version
...
being
...
released
...
is
...
1.0.0,
...
and
...
the
...
following
...
development
...
version
...
will
...
become
...
1.1.0.
Table of Contents |
---|
Prerequisites
Policy documents
The policy on releasing artifacts from an incubating Apache project is stated in the Guide to Release Management During Incubation. While Flume is now a top-level project, that guide is still a good resource for new release managers since it contains what are considered to be best practices.
The Release Manager (RM) must go through the policy document to understand all the tasks and responsibilities of running a release.
Give a heads up
The RM should first create an umbrella issue and then setup a timeline for release branch point. The time for the day the umbrella issue is created to the release branch point must be at least two weeks in order to give the community a chance to prioritize and commit any last minute features and issues they would like to see in the upcoming release.
The RM should then send the pointer to the umbrella issue along with the tentative timeline for branch point to the user and developer lists. Any work identified as release related that needs to be completed should be added as a subtask of the umbrella issue to allow users to see the overall release progress in one place.
TODO: Add an email template to send out
JIRA cleanup
1. Before a release is done, make sure that any issues that are fixed have their fixVersion setup correctly. Run the following JIRA query to see which resolved issues do not have their fix version set up correctly:
JIRA: project = Flume and Resolution = Fixed and fixVersion is EMPTY
The result of the above query should be empty. If some issues do show up in this query that have been fixed since the last release, please bulk-edit them to set the fix version to the version being released.
2. Since the JIRA release note tool will list all JIRAs associated with a particular fixVersion regardless of their Resolution, you may want to remove fixVersion on issues declared as "Duplicate", "Won't Fix", "Works As Expected", etc. You can use this query to find those issues, then just bulk edit them and leave the fixVersion field blank when editing them. (Note that you will need to replace "v1.4.0" below with the version you are trying to release.)
JIRA: project = Flume AND (Status != Resolved OR Resolution != Fixed) AND fixVersion = v1.6.0
3. Finally, check out the output of the JIRA release note tool to see which JIRAs are included in the release, in order to do a sanity check.
Monitor active issues
It is important that between the time that the umbrella issue is filed to the time when the release branch is created, no experimental or potentially destabilizing work is checked into the trunk. While it is acceptable to introduce major changes, they must be thoroughly reviewed and have good test coverage to ensure that the release branch does not start of being unstable.
If necessary the RM can discuss if certain issues should be fixed on the trunk in this time, and if so what is the gating criteria for accepting them.
Creating Release Artifacts
Communicate with the community
1. Send an email to dev@flume.apache.org to
Notify that you are about to branch.
Ask to hold off any commits until this is finished.
2. Send another email after branching is done.
Update the LICENSE file
The release manager is responsible for updating the LICENSE file to provide accurate license information for all binary artifacts contained in the codebase. This is a tedious and painstaking process, and must be performed for each release that includes a binary artifact.
Prepare branches and create tag
In this section, the release is X.Y.Z (e.g. 1.3.0)
1. Checkout the trunk.
No Format |
---|
{toc} h2. Prerequisites h3. Policy documents The policy on releasing artifacts from an incubating Apache project is stated in the [Guide to Release Management During Incubation|http://incubator.apache.org/guides/releasemanagement.html#glossary-release-manager]. While Flume is now a top-level project, that guide is still a good resource for new release managers since it contains what are considered to be best practices. The Release Manager (RM) must go through the policy document to understand all the tasks and responsibilities of running a release. h3. Give a heads up The RM should first create an umbrella issue and then setup a timeline for release branch point. The time for the day the umbrella issue is created to the release branch point must be at least two weeks in order to give the community a chance to prioritize and commit any last minute features and issues they would like to see in the upcoming release. The RM should then send the pointer to the umbrella issue along with the tentative timeline for branch point to the user and developer lists. Any work identified as release related that needs to be completed should be added as a subtask of the umbrella issue to allow users to see the overall release progress in one place. TODO: Add an email template to send out h3. JIRA cleanup Before a release is done, make sure that any issues that are fixed have their fixVersion setup correctly. Run the following JIRA query to see which resolved issues do not have their fix version set up correctly: {noformat} project = Flume and Resolution = Fixed and fixVersion is EMPTY {noformat} The result of the above query should be empty. If some issues do show up in this query that have been fixed since the last release, please bulk-edit them to set the fix version to the version being released. Since the JIRA release note tool will list all JIRAs associated with a particular fixVersion regardless of their Resolution, you may want to remove fixVersion on issues declared as "Duplicate", "Won't Fix", "Works As Expected", etc. You can use this query to find those issues, then just bulk edit them and leave the fixVersion field blank when editing them. (Note that you will need to replace "v1.0.0" below with the version you are trying to release.) {noformat} project = Flume and Resolution != Unresolved AND Resolution != Fixed AND fixVersion = "v1.0.0" {noformat} Finally, check out the output of the [JIRA release note tool|https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311321] to see which JIRAs are included in the release, in order to do a sanity check. h3. Monitor active issues It is important that between the time that the umbrella issue is filed to the time when the release branch is created, no experimental or potentially destabilizing work is checked into the trunk. While it is acceptable to introduce major changes, they must be thoroughly reviewed and have good test coverage to ensure that the release branch does not start of being unstable. If necessary the RM can discuss if certain issues should be fixed on the trunk in this time, and if so what is the gating criteria for accepting them. h2. Creating Release Artifacts h3. Communicate with the community 1. Send an email to dev@flume.apache.org to Notify that you are about to branch. Ask to hold off any commits until this is finished. 2. Send another email after branching is done. h3. Update the LICENSE file The release manager is responsible for updating the LICENSE file to provide accurate license information for all binary artifacts contained in the codebase. This is a tedious and painstaking process, and must be performed for each release that includes a binary artifact. h3. Prepare branches and create tag In this section, the release is X.Y.Z (e.g. 1.3.0) 1. Checkout the trunk. {code} git clone http://git-wip-us.apache.org/repos/asf/flume.git flume {code} |
2.
...
Update
...
CHANGELOG
...
in
...
the
...
trunk
...
to
...
indicate
...
the
...
changes
...
going
...
into
...
the
...
new
...
version.
...
The
...
change
...
list
...
can
...
be
...
swiped
...
from
...
the
...
...
...
...
...
(choose the "text"
...
format
...
for
...
the
...
change
...
log).
...
See
...
JIRA
...
Cleanup
...
above
...
to
...
ensure
...
that
...
the
...
release
...
notes
...
generated
...
by
...
this
...
tool
...
are
...
what
...
you
...
are
...
expecting.
...
Additionally,
...
in
...
the
...
last
...
two
...
releases
...
we
...
have
...
re-ordered
...
the
...
issues
...
to
...
be
...
New
...
Features,
...
Improvements,
...
Bugs,
...
etc
...
as
...
by
...
default
...
the
...
least
...
important
...
items
...
are
...
at
...
the
...
top.
...
3.
...
Update
...
the
...
"version"
...
value in RELEASE-NOTES
...
in
...
trunk
...
to
...
X.(Y+1).0:
No Format |
---|
vim RELEASE-NOTES {code} git add RELEASE-NOTES git checkout trunk find .commit RELEASE-NOTES -name pom.xml | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.$m "Updating Flume version in RELEASE-NOTES for release X.Y.Z" |
4. Create a branch for the X.(Y+1) release series
E.g. flume-1.5 should be branched off of trunk
No Format |
---|
git checkout trunk git checkout -b flume-X..0-SNAPSHOT/" find flume-ng-doc/ -name "*.rst" | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.$(Y+1) |
4. Update the "version" value of all pom.xml and documentation files in trunk to X.(Y+1).0-SNAPSHOT
No Format |
---|
git checkout trunk
mvn versions:set -DnewVersion=X.Y.(Z+1)-SNAPSHOT -DgenerateBackupPoms=false
cd build-support
mvn versions:set -DnewVersion=X.Y.(Z+1)-SNAPSHOT -DgenerateBackupPoms=false
cd ..
git add .
git commit -m "Updating trunk version to X.$(Y+1).0-SNAPSHOT for Flume X.Y.Z release"
|
5. Checkout the release branch and remove -SNAPSHOT from the release branch poms and docs and commit:
No Format |
---|
0-SNAPSHOT/" vim RELEASE-NOTES git add . git commit -m "FLUME-XXXX: Preparing for the X.Y release" {code} 4. Find hash for use later {code} git log {code} 5. Create a branch for the X.(Y+1) release series E.g. flume-1.5 should be branched off of flume-1.4 {code} git checkout flume-X.Y gitmvn checkoutversions:set -b flume-XDnewVersion=X.Y.(YZ+1) git cherry-pick <hash from the previous step> {code} 6. Push the changes up stream {code} git push -u origin trunk:trunk git push -u origin flume-X.Y:flume-X.Y {code} 7. Checkout the release branch: {code} git checkout flume-X.Y {code} 8. Remove -SNAPSHOT from the release branch and commit {code} find . -name pom.xml | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.Y.0/" find flume-ng-doc/ -name "*.rst" | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.Y.0/" git add . git commit -m "FLUME-XXXX: Removing -SNAPSHOT from X.Y branch" {code} 9. Ensure RELEASE-NOTES has the appropriate version and description of the release. 10. Tag a candidate: {code} git tag -a release-X.Y.Z -m "Apache Flume X.Y.Z release." git push origin release-X.Y.Z {code} If an rc1, rc2, etc is needed, delete that tag before creating a new one: {code} git tag -d release-X.Y.Z git push origin :refs/tags/release-X.Y.Z {code} h3. Performing sanity check 1. Check out the candidate {code} git checkout release-X.Y.Z {code} 2. Generate a tarball {code} mvn clean install -DskipTests {code} 3. Unpack the source tarball {code} cd flume-ng-dist/target tar xzvf apache-flume-X.Y.Z-src.tar.gz {code} 4. Do another full build inside the source tarball. This time, allow all unit tests & integration tests to run {code} cd apache-flume-X.Y.Z mvn clean install {code} h3. Signatures and Checksums All artifacts must be signed and checksummed. In order to sign a release you will need a PGP key. You should get your key signed by a few other people. You will also need to recv their keys from a public key server. See the [Apache release signing|https://www.apache.org/dev/release-signing] page for more details. 1. Add your key to the [KEYS|https://dist.apache.org/repos/dist/release/flume/KEYS] file: {code} (gpg --list-sigs <your-email> && gpg --armor --export <your-email>) >> KEYS {code} And commit the changes. 2. Create and sign the artifacts From the flume root directory {code} mvn clean deploy -Psign -DskipTests {code} This will sign, hash, and upload each artifact to Nexus. Note: the checksum files will not be mirrored; They should be downloaded from the main apache dist site. 3. Publish Staging repository Login to https://repository.apache.org and select Staging Repositories on the left under Build Promotion. Select org.apache.flume from the list of repositories, verify it looks OK, and then click Close using "Apache Flume X.Y.Z" as the description to allow others to see the repository. 4. Copy artifacts to people.apache.org Copy the apache-flume-X.Y.Z-{bin,src}.tar.gz{,.asc,.md5,.sha1} files to people.apache.org. h2. Running the vote h3. Call for dev list votes Send an email to dev@flume.apache.org list. For example, {noformat} To: dev@flume.apache.org Subject: [VOTE] Release Apache Flume version X.Y.Z rc0 This is the first release for Apache Flume as a top-level project, version 1.2.0. We are voting on release candidate rc0. *** Please cast your vote within the next 72 hours *** The list of fixed issues: <Link-to-CHANGELOG-in-the-tag> The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) for the source and binary artifacts can be found at: https://people.apache.org/~mpercy/flume/apache-flume-X.Y.Z-rc0/ Nexus Staging Repository: https://repository.apache.org/content/repositories/orgapacheflume-XXXXX/ The tag to be voted on: https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=<commit-hash-of-the-tag> The KEYS file can be found here: https://svn.apache.org/repos/asf/flume/dist/KEYS {noformat} You need 3 \+1 votes from Flume PMC members for a release. h2. Rolling out the Release h3. Upload the artifacts h4. Source and convenience artifacts {code} svn checkout https://dist.apache.org/repos/dist/release/flume dist-flume cd dist-flume mkdir 1.0.0 cp <all release artifacts including asc/checksum files> 1.0.0/ svn rm stable # remove older version link ln -s 1.0.0 stable # update stable version link svn add stable 1.0.0 svn commit {code} It may take up to 24 hours for all mirrors to sync up. h4. Deploy Maven artifacts General instructions on how to deploy the poms and jars to Maven Central can be found at the [Apache page on publishing maven artifacts|https://www.apache.org/dev/publishing-maven-artifacts.html]. *Flume-specific instructions:* After you get your {{settings.xml}} file configured correctly (see above link) then you can run: {{mvn clean deploy \-DskipTests \-Papache-release}} to push the artifacts to the staging repo. You will need to go to the Apache Maven Repository @ [-SNAPSHOT -DgenerateBackupPoms=false cd build-support mvn versions:set -DnewVersion=X.Y.(Z+1)-SNAPSHOT -DgenerateBackupPoms=false cd .. git add . git commit -m "FLUME-XXXX: Removing -SNAPSHOT from X.Y branch" |
6. Ensure RELEASE-NOTES has the appropriate version and description of the release.
7. Push the branching changes upstream
No Format |
---|
git push -u origin trunk:trunk
git push -u origin flume-X.Y:flume-X.Y |
8. Tag a release candidate (in the example below, RC1):
No Format |
---|
git tag -a release-X.Y.Z-rc1 -m "Apache Flume X.Y.Z RC1"
git push origin release-X.Y.Z-rc1
|
If an rc2, rc3 etc is needed, simply create a new rc tag:
No Format |
---|
git tag -d release-X.Y.Z-rc2
git push origin release-X.Y.Z-rc2
|
Performing sanity check
1. Check out the candidate tag
No Format |
---|
git checkout release-X.Y.Z-rc1
|
2. Generate a tarball
No Format |
---|
mvn clean install -DskipTests
|
3. Unpack the source tarball
No Format |
---|
cd flume-ng-dist/target
rm -rf ./apache-flume-X.Y.Z-src/
tar xzvf apache-flume-X.Y.Z-src.tar.gz
|
4. Do another full build inside the source tarball. This time, allow all unit tests & integration tests to run and also include the docs
No Format |
---|
cd apache-flume-X.Y.Z-src
export LC_ALL=C.UTF-8 # Required to build the javadocs on some platforms and in some locales
mvn clean install -Psite -DskipTests
|
5. Verify that the HTML docs that should have been generated inside the binary artifact under /docs are there and do not have rendering errors.
Signatures and Checksums
All artifacts must be signed and checksummed. In order to sign a release you will need a PGP key. You should get your key signed by a few other people. You will also need to recv their keys from a public key server. See the Apache release signing page for more details.
1. Add your key to the KEYS file:
No Format |
---|
(gpg --list-sigs <your-email> && gpg --armor --export <your-email>) >> KEYS
|
And commit the changes.
2. Create and sign the artifacts, including site docs. This pushes the signed artifacts to the ASF staging repository.
In order to do this, you will need a settings.xml file with your username and password for the ASF staging repository. Typically this is placed in ~/.m2/settings.xml and might look something like this:
No Format |
---|
<settings>
<servers>
<server>
<id>apache.releases.https</id>
<username>your_user_id</username>
<password>your_password</password>
</server>
</servers>
</settings>
|
Once your settings.xml file is correct, you run the following from the flume root directory to generate and deploy the artifacts:
No Format |
---|
mvn clean deploy -Psite -Psign -DskipTests
|
This will sign, hash, and upload each artifact to Nexus.
Note: the checksum files will not be mirrored; They should be downloaded from the main apache dist site.
Do the same for javadoc and source artifacts:
No Format |
---|
mvn javadoc:jar gpg:sign deploy:deploy
mvn source:jar gpg:sign deploy:deploy
|
3. Publish Staging repository
Login to https://repository.apache.org and select Staging Repositories on the left under Build Promotion.
Select org.apache.flume from the list of repositories, verify it looks OK, and then click Close using "Apache Flume X.Y.Z" as the description to allow others to see the repository. Note that the staging repository will have a numeric id associated with it that will be used later
4. Copy artifacts to the distribution directories
Copy the apache-flume-X.Y.Z-{bin,src}.tar.gz{,.{asc,sha512}} files to the distribution directory.
Copy the distribution archive files from flume-ng-dist/target to where https://dist.apache.org/repos/dist/dev/flume is checked out on your machine.
- Use "svn delete" to remove any files from prior releases in the directory.
- Use "svn commit" to update the new files.
Update the staging website
- Clone https://github.com/apache/flume-site.
- Checkout the asf-staging branch.
- Add a page to the
source/sphinx/releases
directory for the new release, with the Changelog and links to the documentation (refer previous release pages for details. The documentation should simply use the same paths as the previous releases with correct versions - the documentation will be checked in directly to the production website as mentioned below).edit the new release page using a regex. The search string should be
Code Block ^(.*)\[FLUME\-(\d+)\](.*)$
and the replacement string should be
Code Block $1\[`FLUME-$2 <https://issues.apache.org/jira/browse/FLUME-$2>`__\]$3
- Update
content/sphinx/releases/index.rst
to update the pointer to the latest release. - Update
content/sphinx/download.rst
to also point to the latest release. - Update
content/sphinx/index.rst
as necessary to add a News item to the home page. - Copy the release version of the FlumeUserGuide.rst and FlumeDeveloperGuide.rst to the Documentation directory.
- Commit the changes and push them.
Run mvn package and verify the site at target/site/index.html.
- Run mvn pre-site install.
- Verify the site at https://flume.staged.apache.org.
Running the vote
Call for dev list votes
Send an email to dev@flume.apache.org list. For example,
No Format |
---|
To: dev@flume.apache.org
Subject: [VOTE] Release Apache Flume version X.Y.Z RC1
This is the XXXXX release for Apache Flume as a top-level project,
version X.Y.Z. We are voting on release candidate RC1.
It fixes the following issues:
<Link-to-CHANGELOG-in-the-tag>
*** Please cast your vote within the next 72 hours ***
The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1)
for the source and binary artifacts can be found here:
https://people.apache.org/~mpercy/flume/apache-flume-X.Y.Z-RC1/
Maven staging repo:
https://repository.apache.org/content/repositories/orgapacheflume-XXXXX/
The tag to be voted on:
https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=<commit-hash-of-the-tag>
Flume's KEYS file containing PGP keys we use to sign the release:
https://svn.apache.org/repos/asf/flume/dist/KEYS
|
You need 3 +1 votes from Flume PMC members for a release.
Rolling out the Release
Upload the artifacts
Source and convenience artifacts
No Format |
---|
svn checkout https://dist.apache.org/repos/dist/release/flume dist-flume
cd dist-flume
mkdir {x.y.z}
cp <all release artifacts including asc/checksum files> {x.y.z}/
svn rm {x.?.?} # remove prior release(s)
svn commit -m "Release Flume {x.y.z}"
|
It may take up to 24 hours for all mirrors to sync up.
Deploy Maven artifacts
General instructions on how to deploy the poms and jars to Maven Central can be found at the Apache page on publishing maven artifacts.
Flume-specific instructions:
After you get your settings.xml
file configured correctly (see above link) then you can run: mvn clean deploy -Psite -DskipTests -Papache-release
to push the artifacts to the staging repo.
You will need to go to the Apache Maven Repository @ http://repository.apache.org/
...
and
...
log
...
in
...
with
...
your
...
Apache
...
LDAP
...
credentials
...
Once
...
you
...
log
...
in
...
you
...
will
...
see
...
Build
...
Promotion
...
>
...
Staging
...
Repositories
...
on
...
the
...
left
...
hand
...
side
...
You
...
will
...
want
...
to
...
edit
...
the
...
Flume
...
artifacts
...
that
...
you
...
don't
...
want
...
to
...
push
...
to
...
Maven
...
(you
...
can
...
delete
...
stuff
...
like
...
the
...
release
...
tarball)
...
Click
...
Close
...
to
...
make
...
the
...
atrifacts
...
available
...
on
...
the
...
Staging
...
repository.
...
To
...
push
...
to
...
Central,
...
you
...
click
...
Release.
...
Announce
...
the
...
release
...
Send
...
an
...
...
to
...
announce@apache.org
...
(the
...
from:
...
address
...
must
...
be
...
@apache.org).
...
For
...
example,
No Format |
---|
{noformat} To: announce@apache.org, user@flume.apache.org, dev@flume.apache.org Subject: [ANNOUNCE] Apache Flume 1.2.0 released The Apache Flume team is pleased to announce the release of Flume version 1.2.0. Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. This release can be downloaded from the Flume download page at: http://flume.apache.org/download.html The change log and documentation are available on the 1.2.0 release page: http://flume.apache.org/releases/1.2.0.html Your help and feedback is more than welcome. For more information on how to report problems and to get involved, visit the project website at http://flume.apache.org/ The Apache Flume Team {noformat}: http://flume.apache.org/download.html The change log and documentation are available on the 1.2.0 release page: http://flume.apache.org/releases/1.2.0.html Your help and feedback is more than welcome. For more information on how to report problems and to get involved, visit the project website at http://flume.apache.org/ The Apache Flume Team |
Update the website
- Checkout https://svn.apache.org/repos/asf/flume/site/trunk
- Add a page to the
content/sphinx/releases
directory for the new release, with the Changelog and links to the documentation (refer previous release pages for details. The documentation should simply use the same paths as the previous releases with correct versions - the documentation will be checked in directly to the production website as mentioned below). - Update
content/sphinx/releases/index.rst
to update the pointer to the latest release. - Update
content/sphinx/download.rst
to also point to the latest release. - Update
content/sphinx/index.rst
as necessary to add a News item to the home page. - Copy the release version of the FlumeUserGuide.rst and FlumeDeveloperGuide.rst to the Documentation directory.
- Commit the changes to svn. Go to https://cms.apache.org/flume/ - Stage and publish the changes.
- Checkout https://svn.apache.org/repos/infra/websites/production/flume.
- Create a directory with the current release's directory name under content/releases/content (e.g., content/releases/content/1.3.1)
- Copy both the HTML and PDF versions of the user guide and developer guide (manually generate the PDF from the HTML files), and the javadocs (apidocs directory) into this directory.
- Commit the changes to svn. Done!