Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

1. Preparing the Release Candidate:

Step 1.1. Prepare Release Notes

Draft release notes which should include the list of new features and bug fixes, along with limitations, known issues, etc. Seek help from committers on @dev to achieve good coverage of all important fixes/API changes in the release. 

...

Note: Check if this release has to go out on a blog on Medium (Sukwon) beforehand. If so, keep time to work on the blog post as this may take a while longer. This blog post needs to go through multiple reviews. 

Step 1.2. Inform the Community of timing

Recommendation: While we have added email content in this wiki, it would be best to refer to emails that were sent for the most recent release. You can find them in the apache archive (or your inbox!):

...

Info
iconfalse
titleCommunity announcement email template


To: dev@mxnet.apache.org
Subject: [Announce] Upcoming Apache MXNet (incubating) $release release

Dear MXNet community,

I will be the release manager for the upcoming $release release. (Optional: $CO-RM will be co-managing the release and providing help from the committers side.)
A release candidate will be cut on $utc_date and voting will commence $utc_date_vote0 - $utc_date_vote1. Release notes have been drafted here [1]. If you have any additional features in progress and would like to include it in this release, please assure they have been merged by $utc_date-72.

Feel free to add any other comments/suggestions.

Thanks,
$RM

[1] $release-notes-link

Step 1.3. Make sure all files have correct License headers and the top level LICENSE file has been updated

  • Any new files added in this release must have a valid license header. 
  • Make sure any new submodules or dependencies that have been added in this release have been mentioned in the top level LICENSE file - There is no test to cover this (and is a frequent cause of a -1)
  • Make sure the Apache RAT test is passing (contact mbaijal@ in case of issues). Run manually on the final commit as mentioned below (will be part of a nightly test soon)
    Link - https://github.com/apache/incubator-mxnet/pull/10390
  • DISCLAIMER is correct, filenames include “incubating”
  • More Details: The top level LICENSE and NOTICE files are correct and dependency licenses are acceptable
    • LICENSE and NOTICE files at the root of the artifact directory must only reflect the contents of the artifact in which they are contained.
    • Any new dependency with a non-ASF license must be added to the top level LICENSE file as directed in the links below.
    • All source files have license headers where appropriate.
    • The provenance of all source files is clear (ASF or software grants)
  • Some More details: Refer to the following Links 

 

...

 Step 1.4. Make code changes with necessary version updates

At least 3 days prior to cutting the first release candidate, bump up the version number and address other changes that the release depends on.

    1. Bump up the version number. An example can be followed here: https://github.com/apache/incubator-mxnet/pull/6462, update https://github.com/apache/incubator-mxnet/blob/master/setup-utils/install-mxnet-osx-python.sh#L36
      More recent example - https://github.com/apache/incubator-mxnet/pull/8567
      make sure no broken/non-existing links are added to the master branch.
    2. Check with the website lead and docs lead that all necessary updates have made it into the release branch
      1. Most updates happen post-release.
      2. Broken link updates in tutorials and faqs would be an exception.
    3. Create a RSA GPG key of length 4096, upload it to the public server, and add it to the KEYS file (do this process once for each release lead): https://github.com/apache/incubator-mxnet/blob/master/KEYS & https://dist.apache.org/repos/dist/dev/incubator/mxnet/KEYS See more detailed instructions on creating the key here: https://www.apache.org/dev/openpgp.html#generate-key. Instructions for updating the KEYS file can be found here: https://github.com/apache/incubator-mxnet/blob/master/KEYS#L9-L13. Note that you must use apache email instead of personal email for the key. 

      Code Block
      languagebash
      themeMidnight
      titleAdding Keys
      # checkout the apache mxnet repo
      svn co https://dist.apache.org/repos/dist/dev/incubator/mxnet apache-mxnet
      cd apache-mxnet
      # update the KEYS file ...
      # commit the update
      svn commit -m “update keys file for xxx ” --username your_username --password your_passwd


Step 1.5. Cut the Release Branch

Note: From version 1.3 all branches are created with a name v1.3.x to make patch releases easier. For patch releases cutting a new branch is not necessary. Fixes for patch releases should be merged to v#.#.x branch and latest commit should be tagged with an exact version number, ie v1.3.1.

...

        1. The build_all_versions.sh must also be updated with the correct tag
        2. This script must be built locally to test docs (confirm procedure with Santhosh, Yao) and automate after 0.12.0.

Step 1.6 Test the Release Branch

Once all the PRs have been merged the final commit hash on the release branch must be tested. (You should begin testing as soon as the release branch is created, but make sure it is run atleast once on the final commit)

  1. Nightly Tests
    Following jobs are run as a part of the nightly tests:
    1. NightlyTests
    2. NightlyTestsForBinaries
      These jobs should already contain needed branches if they are cut. If not, check the jobs configuration. These are triggered by DailyTrigger. Either wait until the jobs have been triggered or trigger them manually.
    3. Backward Compatibility checker (currently restricted).
    4. Some jobs might take up to 4-5 hours, so plan accordingly.
    5. In case of a failure of any job, check the console logs, the newly added PRs and try and resolve issues or contact community for a fix (on @dev list, for example). 
  2. Jenkins CI Tests
    1. The release branch must pass the tests on the CI. For example, here - http://jenkins.mxnet-ci.amazon-ml.com/job/incubator-mxnet/job/v1.1.0/
  3. RAT Check - make sure the RAT check passes (it's already part of the verification pipeline as a stage, no special actions needed)

Step 1.7.

...

 Tag release candidate (To be done by a committer)

    1. Go to the GitHub repo’s “releases” tab
    2. Click “Draft a new release”
        • Provide the release tag in the form of “1.0.0.rc0” where 0 means it’s the first release candidate
        • Select the commit by clicking Target: branch > Recent commits > $commit_hash 
        • Copy and paste NEWS.md change into the description box from here
        • Select “This is a pre-release"

Step 1.8. This step has not been done in the past (question)

Tag all the dependent submodules for every MXNet release. If a code-change (e.g. bug-fix) is required to a dependent sub-module then we take a branch from the tag and apply the code-change only to the branch for the sub-module so that the change is minimal. This should allow the MXNet release process to have better convergence towards a stable release.

Step 1.9. Create artifacts for the release and push to the dist folder

a) The src tar - apache-mxnet-src-#.#.#.rc0-incubating.tar.gx:

...

Code Block
languagebash
themeMidnight
titleUploading to dist
# Checkout the Apache directory in Apache distribution SVN "dev" repo
$ svn co https://dist.apache.org/repos/dist/dev/incubator/mxnet
# select p
# Make directory for this RC in the above directory
$ cd mxnet
$ mkdir #.#.#.rc0
# Move artifacts into this folder
$ cp ../apache-mxnet-src-#.#.#.rc0-incubating.tar.gz* #.#.#.rc0
$ svn add #.#.#.rc0
$ svn commit -m "Add mxnet-#.#.#.rc0" --username "lxn2"

Step 1.10. Validate release package (Committer or Contributor)

As per the Apache documentation, verify that the release candidate artifacts satisfy the following:

...

Code Block
languagebash
themeMidnight
titlevalidate the signatures
linenumberstrue
gpg --verify ./apache-mxnet-src-0.11.0.rc3-incubating.tar.gz.asc
gsha512sum --check  ./apache-mxnet-src-0.11.0.rc1-incubating.tar.gz.sha512 

Step 1.11. Test the final release package (Committer or Contributor)

    • Download the package that was just uploaded to svn as above. 
    • Test that a build is successful - spinning up a deep learning Ubuntu AMI will give you an environment with all dependencies to build. For example:

      Code Block
      languagebash
      themeMidnight
      ci/build.py -p ubuntu_gpu


    • if Possible, run a simple (mnist) test to verify the src.

Step 1.12. Upload the release artifacts to Github release tag

    • Edit the release tag from Step 1.7, and upload the release artifacts from Step 1.9 (which were verified in 1.10-1.11).

...

After creating RC, Give 2 days to build scala/ R packages if needed before starting vote on dev@.

Step 1.13. Build Scala Packages 

Follow the release process to build Scala packages and include the stage repo links in the voting email.

...

2. Running the voting process for a release:

The full voting process consists of two phases: (1) vote on dev@mxnet.apache.org (3 days) (2) vote on general@incubator.apache.org (3 days)

Step 2.1. Vote on dev@ 

For the dev@ vote, there must be at least 3 binding +1 votes and more +1 votes than -1 votes. Once that quorum has been reached, the vote then moves onto the general@ vote.

To find out if the person is a committer/IPMC member, you can check apache phonebookhas a binding vote visithttp://homeincubator.apache.org/phonebookprojects/mxnet.html?ctte=incubator

Use the email template below to start the vote:

Info
iconfalse
titleOpening vote email template


To: dev@mxnet.apache.org
Subject: [VOTE] Release Apache MXNet (incubating) version $release.rc0

Dear MXNet community,

This is the vote to release Apache MXNet (incubating) version $release. Voting will start $start-date-time and close on $end-date-time, $time-zone.

Link to release notes:
https://cwiki.apache.org/confluence/<tiny-link>

Link to release candidate:
https://github.com/apache/incubator-mxnet/releases/tag/$release.rc0

Link to source and signatures on apache dist server:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/$release.rc0/

Link to scala packages on the staging repo:

* CPU
https://repository.apache.org/content/repositories/snapshots/org/apache/mxnet/<link>

* GPU
https://repository.apache.org/content/repositories/snapshots/org/apache/mxnet/<link>

Please remember to TEST first before voting accordingly:
+1 = approve
+0 = no opinion
-1 = disapprove (provide reason)


Best regards,
$RM

Step 2.2. Send out the

...

vote results on dev@

...

Info
iconfalse
titleVote results email template

EMAIL TEMPLATE: VOTING RESULTS

...


To:

...

dev@mxnet.apache.org
Subject:

...

[RESULTS]

...

[VOTE]

...

Release

...

Apache MXNet

...

 

(incubating) version $release.rc0

Dear MXNet community,

I'm happy to announce the results of the vote.

This vote passes with <num> +1 votes (<num> binding) and no 0 or -1

...

votes.

+1

...

votes

PMC Members:
* $Name
* $Name 
* $Name
 
Committers:
* $Name
* $Name
 
Community:
* $Name
* $Name
 
0 votes
* No votes
 
-1 votes
* No votes
 
Vote thread:
https://lists.apache.org/list.html?commits@mxnet.apache.org/
I'll continue with the release process and the release announcement will follow in the next few days.

* <name> / binding
* <name> / binding
* <name>
* <name>

0 votes
* No votes

-1 votes
* No votes

Vote thread can be found here [1]. The list of members can be found here [2].

I'll continue with the release process and the release announcement will follow in the next few days.


Best regards,
$RM

[1] https://lists.apache.org/thread.html/<link>
[2] http://incubator.apache.org/projects/mxnet.html

Step 2

...

.3. Triage issues and revote if necessary 

Should any vote fail to reach quorum or release lead determines that another RC needs to go out, the release lead should triage the issues, create GitHub issues, and move to fix the issues. Once fixed, make changes to NEWS & README.md in case it specifically lists the old RC# number. Start the process for another release candidate. In the new voting email, detail what has changed since the last release candidate. Repeat until the vote passes.

...

Info
iconfalse
titleCancel vote template

 

...


To:

...

dev@mxnet.apache.org
Subject:

...

[CANCELLED]

...

[VOTE]

...

Release

...

Apache MXNet

...

(incubating) version $release.rc0

Dear MXNet community,

We are cancelling this vote because:

...

...

We

...

will

...

address the issues and send out rc#+1 for another vote.

Best regards,
$RM

...

Step 2.4. Start a vote on general@

For the vote on general@, anyone can contribute a vote, but only "Incubator PMC" (IPMC) votes are binding.
To pass, there must be 3 binding +1 votes and more +1 votes than -1 votes.

In ASF, votes are open "at least" 72hrs (3 days). If you don't get enough number of binding votes within that time, you cannot close the voting deadline. You need to extend it.

Send out a notice to vote with the template below.
To get the fingerprint of your keys, run `gpg2 --list-keys --fingerprint`

...

Info
iconfalse
titleVote on @general email template

Dear community,

This is a call for a releasing Apache MXNet (incubating)

EMAIL TEMPLATE: OPEN A VOTE TO GENERAL@

...

#.#.#,

...

release
candidate

...

#.

Apache

...

MXNet

...

(incubating)

...

community

...

has

...

voted

...

and

...

approved

...

the

...

release.

Vote

...

thread:
https://lists.apache.org/#######

Result

...

thread:
https://lists.apache.org/#########

The

...

source

...

tarball,

...

including

...

signatures,

...

digests,

...

etc.

...

can

...

be

...

found

...

at:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/########

The

...

tag

...

to

...

be

...

voted

...

upon

...

is #########:
<link>

The

...

release

...

hash

...

is

...

#########:
<link>

Release

...

artifacts

...

are

...

signed

...

with

...

the

...

following

...

key:
<link/fingerprint>

KEYS

...

file

...

available:
<link>

For

...

information

...

about

...

the

...

contents

...

of

...

this

...

release,

...

see:
<link

...

to

...

release

...

notes>

The

...

vote

...

will

...

be

...

open

...

for

...

72

...

hours.

[

...

]

...

+1

...

release this

...

package

...

as #########
[

...

]

...

+0

...

no

...

opinion
[

...

]

...

-1

...

do not

...

release

...

this

...

package

...

because...

Best regards,
$RM

Step 2.5: Send out the

...

vote results on general@

Send out an email on general@ in the same voting thread which you sent in previous step to summarize the result.

3. Finalizing and posting a release

...

3.1. Marking & Publishing the Official release

...