...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
#!/bin/bash # to get a list of commits since <date> git log --pretty=format:"%h - %cn, %cd : %s" --no-merges --since="2017-09-01" > commits.txt # To get a list of committers since <date> git shortlog -sne --since="10 Aug 2017" |
Alternatively, you can use the script here to create groups of commits based on commit comments.
Info | ||
---|---|---|
| ||
Check if this release has to go out on a blog on Medium (Sukwon) beforehand. If so, keep time to work on the blog post as this may take a while longer. This blog post needs to go through multiple reviews. |
...
- Any new files added in this release must have a valid license header.
- Make sure any new submodules or dependencies that have been added in this release have been mentioned in the top level LICENSE file - There is no test to cover this (and is a frequent cause of a -1)
- Make sure the Apache RAT test is passing (this is ensured automatically by the PR validation RAT check in CI)
- DISCLAIMER is correct, file names include “incubating”
- Check if the license issues listed in Github are all addressed or they're not the release blocker.
- Major modifications/additions to third-party and dual license should be dealt with on a case-by-case basis by the PMC.
More Details
The top level LICENSE and NOTICE files are correct and dependency licenses are acceptable
- LICENSE and NOTICE files at the root of the artifact directory must only reflect the contents of the artifact in which they are contained.
- Any new dependency with a non-ASF license must be added to the top level LICENSE file as directed in the links below.
- All source files have license headers where appropriate.
- The provenance of all source files is clear (ASF or software grants)
...
- license_header.py can be used to check the potential license issue and add license header if necessary.
Refer to the following Links
...
At least 3 days prior to cutting the first release candidate, bump up check the version number and address other changes that the release depends on.
Bump up Update the version number . An example can be followed here: if it's not reflecting the new release version. Here is a previous version update PR (https://github.com/apache/incubator-mxnet/pull/6462, update https://github.com/apache/incubator-mxnet/blob/master/setup-utils/install-mxnet-osx-python.sh#L36
More recent example - https://github.com/apache/incubator-mxnet/pull/856719196).
make sure no broken/non-existing links are added to the master/v1.x branch.
Check with the website lead and docs lead that all necessary updates have made it into the release branch
- Most updates happen post-release.
- Broken link updates in tutorials and faqs would be an exception.
...
Step 1.5. Add RSA GPG key of release manager/co-manager to repository
Create a RSA GPG key of length 4096, upload length 4096, upload it to the public server, and add it to the KEYS file (do this process once for each release lead):
...
...
See more detailed instructions on creating the key here: https://www.apache.org/dev/openpgp.html#generate-key. Instructions for updating the KEYS file can be found here: https://github.com/apache/incubator-mxnet/blob/master/KEYS#L9-L13at the beginning of the KEYS file. Note that you must use apache email instead of personal email for the key. The following instructions shows how to update the key using svn.
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
# checkout the apache mxnet repo svn co https://dist.apache.org/repos/dist/devrelease/incubator/mxnet apache-mxnet cd apache-mxnet # update the KEYS file ... # commit the update svn commit -m “update keys file for xxx ” --username your_username --password your_passwd |
Step 1.
...
6. Cut the Release Branch
Note: From version 1.3 all branches are created with a name v1.3.x to make patch releases easier. For patch releases cutting a new branch is not necessary. Fixes for patch releases should be merged to v#.#.x branch and latest commit should be tagged with an exact version number, ie v1.3.1.
- To create a new major or minor branch use (replace v1.4.x with actual branch name):
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
# checkout clean master branch
git clone https://github.com/apache/incubator-mxnet.git
cd incubator-mxnet
git branch v1.4.x
git push --set-upstream origin v1.4.x |
- The release candidate should contain all planned featuresThe release candidate should contain all planned features, bug fixes, and code changes above. The release candidate commit should have passed the merge build, pip builds
- Docs should also be manually built locally & checked for correctness. TODO: how to retrieve the commit hash.
- Checkout the commit Checkout the commit into a new branch "v#.#.x" and push it (It might be a good idea to do this in advance and freeze the branch)
- Update NEWS and README. An example Examples can be followed here: https://github.com/apache/incubator-mxnet/pull/6471. Note NEWS.md and README.md. Note that this assumes the existence of the tag of next release, which is not created yet. Therefore, the update only happens on the release branch (not the master branch) so broken links are not visible to users.
Manual Docs Build
- The build_all_versions.sh must also be updated with the correct tag
- This script must be built locally to test docs (confirm procedure with Santhosh, Yao) and automate after 0.12.0.
...
- Docs should also be manually built locally & checked for correctness.
- build and host the website locally to check the correctness.
- More reference here: Host a MXNet Website Preview. Contact MXNet engineering team for access if needed
Step 1.7. Test the release branch
Once all the PRs have been merged the final commit hash on the release branch must be tested. (You should begin testing as soon as the release branch is created, but make sure it is run atleast at least once on the final commit)
...
Following jobs are run as a part of the nightly tests:
These jobs should already contain needed branches if they are cut. If not, check the jobs configuration. These are triggered by DailyTrigger. Either wait until the jobs have been triggered or trigger them manually.
...
Steps to trigger the jobs manually
- Only Jenkins Admin can update the pipeline with the release branch & trigger build on the pipelines
- Verify if the release branch is scanned by the pipeline (for eg Jenkins → NightlyTests pipeline → Branches should reveal the release branch).
- If the branch doesn't exist, trigger a branch scan using "Scan Repository now" (this should scan all the branches that meet the criterion mentioned in Configure → Branch Sourches → Behavior)
- If branch is still not added, then manually add the release branch by updating the criterion
- Once logged in to Jenkins, Configure -> Branch Sources -> Behavior -> Append release branch E.g. (v1.7.x)
- Trigger a build manually (if needed) <Pipeline> -> 1.7.x -> Build Now
Note : Currently (master)|(^v.*) ensures all release branches starting with v should be scanned & included into the pipeline.
Backward Compatibility checker (can skip this step).
Some jobs might take up to 4-5 hours, so plan accordingly.
In case of a failure of any job, check the console logs, the newly added PRs and try and resolve issues or contact community for a fix (on @dev list, for example).
Jenkins CI Tests
Ensure that PR verification is enabled for the target release branch (This is automatically ensured by
Jira | ||||||
---|---|---|---|---|---|---|
|
Ensure that nightly tests are enabled for the target release branch, for example v.1.3.x NightlyTestsForBinaries, v1.3.x NightlyTests
Ensure that release branch is in the daily reports (contact Marco de Abreu )
RAT Check
make sure the RAT check passes (it's already part of the verification pipeline as a stage, no special actions needed)
Step 1.8. Tag release candidate (To be done by a committer)
Go to the GitHub repositories “releases” tab
Click “Draft a new release”
- Provide the release tag in the form of “1.0.0.rc0” where 0 means it’s the first release candidate
- Select the commit by clicking Target: branch > Recent commits > $commit_hash
- Copy and paste NEWS.md change into the description box from here
- Select “This is a pre-release"
Step 1.9. This step has not been done in the past
Tag all the dependent submodules for every MXNet release. If a code-change (e.g. bug-fix) is required to a dependent sub-module then we take a branch from the tag and apply the code-change only to the branch for the sub-module so that the change is minimal. This should allow the MXNet release process to have better convergence towards a stable release.
Step 1.10. Create artifacts for the release and push to the dist folder
The src tar - apache-mxnet-src-#.#.#.rc0-incubating.tar.gz:
Clone the Repo and checkout the release branch
Remove R-package until the licensing issue is resolved
Remove all the .git files
Remove all other files of the form .* (eg .travis, .DS_Store etc)
on MacOS use gnu-tar
Try to untar it in a different type of machine and try a classic build with `make`. Follow build instructions in the "Build from Source" section http://mxnet.incubator.apache.org/get_started/install.html
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
git clone git@github.com:apache/incubator-mxnet.git |
Jenkins CI Tests
- Ensure that PR verification is enabled for the target release branch, for example v.1.3.x job
- Ensure that nightly tests are enabled for the target release branch, for example v.1.3.x NightlyTestsForBinaries, v1.3.x NightlyTests
- Ensure that release branch is in the daily reports (contact Marco de Abreu )
RAT Check
- make sure the RAT check passes (it's already part of the verification pipeline as a stage, no special actions needed)
Step 1.7. Tag release candidate (To be done by a committer)
- Go to the GitHub repositories “releases” tab
- Click “Draft a new release”
- Provide the release tag in the form of “1.0.0.rc0” where 0 means it’s the first release candidate
- Select the commit by clicking Target: branch > Recent commits > $commit_hash
- Copy and paste NEWS.md change into the description box from here
- Select “This is a pre-release"
Step 1.8. This step has not been done in the past
Tag all the dependent submodules for every MXNet release. If a code-change (e.g. bug-fix) is required to a dependent sub-module then we take a branch from the tag and apply the code-change only to the branch for the sub-module so that the change is minimal. This should allow the MXNet release process to have better convergence towards a stable release.
Step 1.9. Create artefacts for the release and push to the dist folder
The src tar - apache-mxnet-src-#.#.#.rc0-incubating.tar.gz:
...
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
git clone git@github.com:apache/incubator-mxnet.git apache-mxnet-src-0.12.0.rc0-incubating git checkout v0.12.0 git submodule update --init --recursive git checkout 0.12.0.rc0 rm -rf R-package/* rm -rf R-package rm -rf .DS_Store rm -rf CODEOWNERS find . -name ".git*" -print0 | xargs -0 rm -rf brew install gnu-tar gtar -czvf apache-mxnet-src-0.12.0.rc0-incubating.tar.gz apache-mxnet-src- git checkout v0.12.0 git submodule update --init --recursive git checkout 0.12.0.rc0-incubating rm -rf R-package rm -rf .DS_Store rm -rf CODEOWNERS find . -name ".git*" -print0 | xargs -0 rm -rf brew install gnu-tar gtar -czvf apache-mxnet-src- |
...
0. |
...
12. |
...
0.rc0-incubating.tar.gz |
...
apache-mxnet-src-0.12.0.rc0-incubating |
apache-mxnet-src-#.#.#.rc0-incubating.tar.gz.asc:
Details here: http://www.apache.Details here: http://www.apache.org/dev/release-signing.html#openpgp-ascii-detach-sig
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
# Checkout the Apache directory in Apache distribution SVN "dev" repo $ svn co https://dist.apache.org/repos/dist/dev/incubator/mxnet # select p # Make directory for this RC in the above directory $ cd mxnet $ mkdir #.#.#.rc0 # Move artifacts into this folder $ cp ../apache-mxnet-src-#.#.#.rc0-incubating.tar.gz* #.#.#.rc0 $ svn add #.#.#.rc0 $ svn commit -m "Add mxnet-#.#.#.rc0" --username "lxn2" |
Step 1.
...
11. Validate release package (Committer or Contributor)
As per the Apache documentation, verify that the release candidate artifacts satisfy the following:
PGP signatures and SHA256 checksum verification
Code Block | ||||
---|---|---|---|---|
| ||||
svn co https://dist.apache.org/repos/dist/dev/incubator/mxnet && cd mxnet/#.#.#.rc0 brew install gpg coreutils wget https://downloads.apache.org/incubator/mxnet/KEYS gpg --import ../KEYS |
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
gpg --verify ./apache-mxnet-src-0.11.0.rc3-incubating.tar.gz.asc gsha512sum --check ./apache-mxnet-src-0.11.0.rc1-incubating.tar.gz.sha512 |
Step 1.
...
12. Test the final release package (Committer or Contributor)
Download the package that was just uploaded to svn as above.
Test that a build is successful - spinning up a deep learning Ubuntu AMI will give you an environment with all dependencies to build. You can build MXNet from source or from CI dockers. For example:
Code Block | ||||
---|---|---|---|---|
| ||||
pip install --user -r ci/requirements.txt
python ci/build.py -p ubuntu_gpu |
...
If possible, run a simple (MNIST) test to verify the src.
Step 1.
...
13. Upload the release artifacts to Github release tag
Edit the release tag from Step 1.7, and upload the release artifacts from Step 1.9 (which were verified in 1.10-1.11).
After creating RC, give 2 days to build scala/ R packages if needed before starting vote on dev@.
Step 1.
...
14. Language bindings
...
(optional)
The language bindings are released separately from the main MXNet release. To release Scala/Java Clojure and R bindings, follow the Follow the guides for the corresponding language bindings. Release manager can ask the help from dev@ to reach owners of different language bindings to do the validation.
Scala/Java
Follow the release process to build Scala/Java packages and include the stage repo links in the voting email:
...
Follow the release process to build Clojure packages and include the stage repo links in the voting email:
Closure Clojure Release Process
R
Follow the release process to build R packages and include the stage repo links in the voting email:
...
The full voting process consists of two phases: (1) vote on dev@mxnet.apache.org (3 days) (2) vote on general@incubator.apache.org (3 days). For full documentation refer to the Apache Voting Process.
Step 2.1. Vote on dev@
We should cc the incubation mentors in the vote threads to keep them informed.
Our mentors are: Bob Paulin (bob@apache.org), Henri Yandell (bayard@apache.org), Jason Dai (jasondai@apache.org), Markus Weimer (weimer@apache.org), Michael Wall (mjwall@apache.org).
Step 2.1. Vote on dev@
For the dev@ vote, there must be at least 3 binding For the dev@ vote, there must be at least 3 binding +1 votes and more +1 votes than -1 votes. Once that quorum has been reached, the vote then moves onto the general@ vote.
...
Info | ||||
---|---|---|---|---|
| ||||
Cc: mentors Dear MXNet community, This is the vote to release Apache MXNet (incubating) version $release. Voting will start $start-date-time and close on $end-date-time, $time-zone. Link to release notes: Link to release candidate: Link to source and signatures on apache dist server: Link to scala packages on the staging repo: * CPU * GPU Please remember to TEST first before voting accordingly:
|
Step 2.2.
...
icon | false |
---|---|
title | Vote results email template |
...
I'm happy to announce the results of the vote.
This vote passes with <num> +1 votes (<num> binding) and no 0 or -1 votes.
...
Testing Release Candidate (for Contributors)
There are 3 ways of getting the source code for the release candidate
Check out tag
Method 1Code Block git clone --recursive https://github.com/apache/incubator-mxnet.git git checkout tags/<tag_name> -b <branch_name>
Method 2Code Block git clone --recursive https://github.com/apache/incubator-mxnet.git --branch=<tag_name>
- Download the zip on Github Source page
https://github.com/apache/incubator-mxnet/tags - Download the distribution
https://dist.apache.org/repos/dist/dev/incubator/mxnet/
Once checked out, follow the steps to build mxnet from source [1] and then various tests can be run.
It is recommended that contributors test out parts of the MXNet codebase they have touched. This ensures greater coverage of the testing and reduces the risk of running into errors post release.
Note: CPU vs GPU, MKLDNN : ON/OFF, Linux vs Centos vs Windows vs Mac are various configurations possible.
Make sure to verify the build flags before running the tests on the release candidate.
[1] https://mxnet.apache.org/get_started/build_from_source
Step 2.3. Send out the vote results on dev@
Info | ||||
---|---|---|---|---|
| ||||
Cc: mentors |
I'll continue with the release process and the release announcement will follow in the next few days.
...
[1] https://lists.apache.org/thread.html/<link>
[2] http://incubator.apache.org/projects/mxnet.html
Step 2.3. Triage issues and re-vote if necessary
Should any vote fail to reach quorum or release lead determines that another RC needs to go out, the release lead should triage the issues, create GitHub issues, and move to fix the issues. Once fixed, make changes to NEWS & README.md in case it specifically lists the old RC# number. Start the process for another release candidate. In the new voting email, detail what has changed since the last release candidate. Repeat until the vote passes.
Info | ||||
---|---|---|---|---|
| ||||
To: dev@mxnet.apache.org We are cancelling this vote because: ... We will address the issues and send out rc#+1 for another vote. Best regards, |
Step 2.4. Start a vote on general@
For the vote on general@, anyone can contribute a vote, but only "Incubator PMC" (IPMC) votes are binding.
To pass, there must be 3 binding +1 votes and more +1 votes than -1 votes.
...
I'm happy to announce the results of the vote. This vote passes with <num> +1 votes (<num> binding) and no 0 or -1 votes. +1 votes I'll continue with the release process and the release announcement will follow in the next few days.
[1] https://lists.apache.org/thread.html/<link> |
Step 2.4. Triage issues and re-vote if necessary
Should any vote fail to reach quorum or release lead determines that another RC needs to go out, the release lead should triage the issues, create GitHub issues, and move to fix the issues. Once fixed, make changes to NEWS & README.md in case it specifically lists the old RC# number. Start the process for another release candidate. In the new voting email, detail what has changed since the last release candidate. Repeat until the vote passes.
Info | ||||
---|---|---|---|---|
| ||||
To: dev@mxnet.apache.org We are cancelling this vote because: ... We will address the issues and send out rc#+1 for another vote. Best regards, |
Step 2.5. Start a vote on general@
For the vote on general@, anyone can contribute a vote, but only "Incubator PMC" (IPMC) votes are binding.
To pass, there must be 3 binding +1 votes and more +1 votes than -1 votes.
In ASF, votes are open "at least" 72hrs (3 days). If you don't get enough number of binding votes within that time, you cannot close the voting deadline. You need to extend it.
Make sure the dev community is aware of any ASF votes and results so they can be active in the ongoing voting process.
Send out a notice to vote with the template below.
To get the fingerprint of your keys, run `gpg2 --list-keys --fingerprint`
Info | ||||
---|---|---|---|---|
| ||||
To: general@incubator.apache.org Dear community, This is a call for a releasing Apache MXNet (incubating) #.#.#, release Apache MXNet (incubating) community has voted and approved the release. Vote thread: Result thread: The source tarball, including signatures, digests, etc. can be found at: The tag to be voted upon is #########: The release hash is #########: Release artifacts are signed with the following key: KEYS file available: For information about the contents of this release, see: The vote will be open for 72 hours. [ ] +1 release this package as ######### Best regards, |
Step 2.6: Send out the vote results on general@
Send out an email on general@ in the same voting thread which you sent in previous step to summarize the result. Send out a notice with the vote results with the template below.
Info |
---|
To: general@incubator.apache.org |
3. Finalizing and posting a release
3.1. Marking & Publishing the Official release
Once the release candidate passes the vote, we draft the official release on GitHub and rename the release candidate as the official release.
Step 3.1.1: Draft official release on GitHub
Go to the GitHub repo’s “releases” tab
Click “Draft a new release”
Provide the release tag in the form of “<major>.<minor>.<patch>”
Select the commit by clicking Target: branch > the passing release candidate tag
Copy and paste NEWS change into the description box
Step 3.1.2: Rename and publish the source package
First upload to dev and then move to releases as follows:
#create a directory under releases - https://httpd.apache.org/dev/release.html
This process needs to change -> http://www.apache.org/dev/release-download-pages.html
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
# Step 1: Download and Untar
tar -zxvf apache-mxnet-src-0.11.0.rc3-incubating.tar.gz
# Step 2: rename (remove the rc) and tar again
mv apache-mxnet-src-0.11.0.rc3-incubating apache-mxnet-src-0.11.0-incubating
gtar -czvf apache-mxnet-src-0.11.0-incubating.tar.gz apache-mxnet-src-0.11.0-incubating
# Step 3: Resign now
# Step 4: First upload here -
svn mv https://dist.apache.org/repos/dist/dev/incubator/mxnet/0.11.0/
# Step 5: Move from dev to release
svn mv https://dist.apache.org/repos/dist/dev/incubator/mxnet/0.11.0/ https://dist.apache.org/repos/dist/release/incubator/mxnet/ --message "0.11.0 release moving from dev to release repo"
# Wait 24hours for the src mirroring. The final download page for the announce email is somewhere here (example 0.11.0) - https://www.apache.org/dyn/closer.cgi/incubator/mxnet/0.11.0-incubating/ |
Step 3.1.3: Build, test, and publish pip wheels and python docker images
For linux, we can trigger a release pipeline that builds, tests and publishes the pip wheels and python docker images for us with just a few steps:
- Go to https://jenkins.mxnet-ci.com/job/restricted-mxnet-cd/job/mxnet-stable-release-pipeline/ and make sure you have write access to the Jenkins system.
- Click "Configure" and update the "Branch specifier" to point to the GitHub branch/tag and click "Save" (The GitHub repository URL is set to apache/incubator-mxnet, if you are releasing using a fork, update this URL too)
- For branch, use
<branchName>
- For tag, use
refs/tags/<tagName>
- For branch, use
- Click "Build with Parameters" from the left pane, modify the following fields as needed and click on "Build"
- MXNET_VARIANTS : Make sure all supported cu* flavors are present
- RELEASE_BUILD" : Keep the box Unchecked
- CD_RELEASE_JOB_NAME : Do not modify this field
- VERSION : Mention the MXNet release tag here e.g. 1.8.0, 2.0.0 (This tag is only used in naming python docker images.)
- This triggers the release pipelines for different cpu and cu* flavors which can be found here https://jenkins.mxnet-ci.com/job/restricted-mxnet-cd/job/mxnet-stable-release-job/ . You can click on "Open Blue Ocean" from the left pane to check pipeline status. There are 3 pipelines that get trigerred:
- First pipeline builds the libmxnet binaries for different flavors. Only after this pipeline completes (takes around 2.5 hours), the other two pipelines are triggered, that run in parallel.
- Second pipeline "python/pypi" publishes the pip wheels to https://dist.mxnet.io/python/.
- Third pipeline "python/docker" publishes the python docker images to https://gallery.ecr.aws/w6z5f7h2/mxnet/python.
- If any job in any of the three pipelines fails, go through the corresponding logs and either
- Re-trigger the pipeline in case it's a flaky failure or
- Root cause and fix the issue and then re-trigger the pipeline.
- Re-trigger the pipeline in case it's a flaky failure or
- Once all pipelines are green, Python wheels and Python docker images for all variants are published to their respective targets. Now we need to move these released artifacts to their final locations so that they are available to users:
- Python wheels (for this release) from https://dist.mxnet.io/python/ go to PyPI.
- Identify the wheels by the version number, date created, and variant string
- Download the above wheels, give them another test if needed, and upload them to PyPI
- For more info regarding publishing pip wheels to Pypi, please contact zhasheng@apache.org
- Python docker images (for this release) from https://gallery.ecr.aws/w6z5f7h2/mxnet/python go to dockerhub.
- Identify the docker images by the version number, date created, and variant string
- Pull each docker image to local (docker pull public.ecr.aws/w6z5f7h2/mxnet/python:<tag>)
- Tag downloaded images (docker tag …)
- Login to dockerhub (docker login ...)
- Push images (docker push …)
- Python wheels (for this release) from https://dist.mxnet.io/python/ go to PyPI.
For mac, the following automated script can be used to build the wheel.(MacOS machine is required to generate wheels for mac)
#!/bin/bash
# Install brew
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# Install developement tools
brew install nasm
brew install automake
brew install libtool
brew install ninja
brew install pkg-config
python -m pip install contextvars
python -m pip install numpy
python -m pip install requests
# Clone MXNet
cd $HOME
git clone -b v1.x --single-branch https://github.com/apache/incubator-mxnet.git # Use release tag instead
cd $HOME/incubator-mxnet/
git submodule update --init --recursive
# Set ENV variables and build statically
export IS_RELEASE=True
export mxnet_variant=cpu
echo $(git rev-parse HEAD) >> python/mxnet/COMMIT_HASH
mkdir -p $HOME/pip_build/mxnet-build
cp -r * $HOME/pip_build/mxnet-build
export CODE_DIR=$(pwd)
cd $HOME/pip_build/mxnet-build
CMAKE_STATICBUILD=1 ./tools/staticbuild/build.sh $mxnet_variant pip
# Package wheel
cd $HOME/pip_build
cp -r mxnet-build/tools/pip/* .
python setup.py bdist_wheel
# Copy wheel to $HOME/pip_build
cp $HOME/pip_build/dist/mxnet* $HOME/pip_build
Wheels are publish manually to pypi.
Please note that the release tag is usually created on a branch that's forked from v1.x and master. In that case it may lack some of the latest ci/cd fixes on the v1.x and master branch. We might need to manually port back those fixes for the cd pipeline to work.
Step 3.1.4: Verify the resources are uploaded
Check that the resources are present in http://www.apache.org/dist/incubator/mxnet. It may take a while for them to be visible. This will be mirrored throughout the Apache network. There are a few remaining steps.
3.2. Update the MXNet website
Anchor | ||||
---|---|---|---|---|
|
Now that a new version has been drafted on GitHub with source and pip packages published, we need to update the MXNet website. This process includes 2 steps, the first one is to create website artifacts in release version, the second is to update website in master branch.
Step 3.2.1: Create website artifacts in release version
Follow these steps to update website in release branch and create website artifacts.
- Add the new installation version on the get_start page: Set the new version to be default, and add the dropdown option. See this example
- Add download links to the install archives. See this example. Make sure to use archive.apache.org for all except the current one.
- Update the pip package table in /python/cpu/pip.md and /python/gpu/pip.md. See this example
- Update and generate the image file for pip install table.
- Download pip-packages.sketch from dmlc/web-data repo
- Use Sketch or other diagram drawing tools to update the sketch file for the new version table (add rows/columns and checkmarks)
- Save the new pip table as a png file with name pip-packages-{NEW_RELEASE_VERSION}.png
- Upload the .sketch file and .png file to dmlc/web-data repo. See this example
- Update pip install table link in pip_snippet.md. See this example
- Add menu options for general version dropdown. Add the new release version number in _config.ym, _config_beta.tml and _config_prod.yml under '- master'.
- Update the dropdown versions for python doc. See this example
- Update the dropdown versions in header.html. See this example
- Prepare and deploy the release version website artifacts, this step requires access permissions to CI Jenkins server
- Log in to Jenkins pipeline restricted-website-update-versions
- Goto Configure to make sure the Git Repository URL points to https://github.com/apache/incubator-mxnet.git
- Click Build with Parameters on the left panel
- Specify COMMIT_ID to the release branch
- Specify FOLDER_NAME to the release version, e.g. 1.8.0 or 2.0.0
- Build and make sure the pipeline finish successfully
Step 3.2.2 Update website in master version
Ensure all the above processes in step 3.2.1 are all completed. Next task is to update master version website.
- Cherry-pick above changes (step 1-6 in 3.2.1) from release branch to master branch and v1.x branch. See this example
- Set default website version to new release version by updating the
RewriteRule
in /docs/static_site/src/.htaccess under comment "# Set default website version to current stable". For example: update the version 1.x to the new release version:- RewriteRule ^(.*)$ /versions/1.x/$1 [r=307,L] => RewriteRule ^(.*)$ /versions/{NEW_RELEASE_VERSION}/$1 [r=307,L]
- Publish website artifact to the main website
- Manually trigger pipeline restricted-website-build-master to publish the new website artifact. Make sure that the pipeline finishes successfully.
- The new website
...
icon | false |
---|---|
title | Vote on @general email template |
To: general@incubator.apache.org
Subject: [VOTE] Release Apache MXNet (incubating) version $release.rc0
Dear community,
This is a call for a releasing Apache MXNet (incubating) #.#.#, release
candidate #.
Apache MXNet (incubating) community has voted and approved the release.
Vote thread:
https://lists.apache.org/#######
Result thread:
https://lists.apache.org/#########
The source tarball, including signatures, digests, etc. can be found at:
https://dist.apache.org/repos/dist/dev/incubator/mxnet/########
The tag to be voted upon is #########:
<link>
The release hash is #########:
<link>
Release artifacts are signed with the following key:
<link/fingerprint>
KEYS file available:
<link>
For information about the contents of this release, see:
<link to release notes>
The vote will be open for 72 hours.
[ ] +1 release this package as #########
[ ] +0 no opinion
[ ] -1 do not release this package because...
Best regards,
$RM
Step 2.5: Send out the vote results on general@
Send out an email on general@ in the same voting thread which you sent in previous step to summarize the result.
3. Finalizing and posting a release
3.1. Marking & Publishing the Official release
Once the release candidate passes the vote, we draft the official release on GitHub and rename the release candidate as the official release.
Step 3.1.1: Draft official release on GitHub
- Go to the GitHub repo’s “releases” tab
- Click “Draft a new release”
- Provide the release tag in the form of “<major>.<minor>.<patch>”
- Select the commit by clicking Target: master > the passing release candidate tag
- Copy and paste NEWS change into the description box
Step 3.1.2: Rename and publish the source package
First upload to dev and then move to releases as follows:
#create a directory under releases - https://httpd.apache.org/dev/release.html
This process needs to change -> http://www.apache.org/dev/release-download-pages.html
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
# Step 1: Download and Untar
tar -zxvf apache-mxnet-src-0.11.0.rc3-incubating.tar.gz
# Step 2: rename (remove the rc) and tar again
mv apache-mxnet-src-0.11.0.rc3-incubating apache-mxnet-src-0.11.0-incubating
gtar -czvf apache-mxnet-src-0.11.0-incubating.tar.gz apache-mxnet-src-0.11.0-incubating
# Step 3: Resign now
# Step 4: First upload here -
svn mv https://dist.apache.org/repos/dist/dev/incubator/mxnet/0.11.0/
# Step 5: Move from dev to release
svn mv https://dist.apache.org/repos/dist/dev/incubator/mxnet/0.11.0/ https://dist.apache.org/repos/dist/release/incubator/mxnet/ --message "0.11.0 release moving from dev to release repo"
# Wait 24hours for the src mirroring. The final download page for the announce email is somewhere here (example 0.11.0) - https://www.apache.org/dyn/closer.cgi/incubator/mxnet/0.11.0-incubating/ |
Step 3.1.3: Download the nightly pip build, rename, and publish:(by Sheng)
Step 3.1.4: Build, test, and publish the Docker images: (by Meghna)
- This was done as follows in 0.11.0. In the future use the automated pipeline -
- Update the tag in the tool.sh file or select the correct tag in the Jenkins Job
- Select the correct Repo name = mxnet
- Trigger the Job with IS_PUBLISH=false
- Login to the slave and test each image for correct version by checking logs and/or Readme.
- Manually upload the 20 images that are built.
- Make sure to update the latest and previous docker images
Step 3.1.5: Verify the resources are uploaded
Check that the resources are present in http://www.apache.org/dist/incubator/mxnet. It may take a while for them to be visible. This will be mirrored throughout the Apache network. There are a few remaining steps.
Step 3.1.6: Test
Test multinode and multi gpu (kvstore=1)
...
Now that a new version has been drafted on GitHub with source and pip packages published, we need to update the MXNet website.
...
- Set the new version to be default. See this example: docs/_static/js/options.js.
- Add the option and all of the new selector classes. See this example: docs/install/index.md.
...
- Current: http://apache.org/dist/incubator/mxnet/
- Legacy: http://archive.apache.org/dist/incubator/mxnet/
...
- If this is a minor release, the branch should already be in the first parameter "tags_to_build".
- If this is a major release, you will need to add the branch to the list in the Default Value field. Note these are semicolon separated values.
...
- If this is a minor release, you will want to bump the version up for the branch you're updating. Example: change 1.2.0 to 1.2.1.
- If this is a major release, you will need to add the new version number to the list. Example: add 1.3.0. Note these are semicolon separated values.
...
- will be published to http://mxnet.incubator.apache.org/. There can be a delay due to some edge caching with Apache infra.
...
- After publishing, check the correctness of the main website
Not Sure Manually update the website’s download links:
...
3.3. Update NEWS.md & README.md on the MXNet master branch
Can happen after the announcement.
3.4. Review the announcement email content on private@mxnet.incubator.apache.org before sending the announcement.
...
the announcement.
Update the Table of Contents by deleting the one from the previous release, and generating a new one using the complete contents of news.md. You can use this markdown toc generator.
Also update NEWS.md and README.md on v1.x branch, If the release version is branched off from v1.x.
3.4. Check if this release has to go out on a blog on blogs.apache.org
...
In order to get write access to blogs.apache.org/mxnet, one need to first register on blogs.apache.org and ask Henri (bayard@apache.org) to grant the permission.
3.5. Review the announcement email content on dev@mxnet.incubator.apache.org before sending the announcement.
3.6. Announce the release
Once everything is working (website docs, website changes, 24 hours after upload) create an announcement on the website and then send an email to the mailing list.
...
Info |
---|
There must be a link in the announcement to the downloads page, not to the dyn/closer page. Downloads must be mirrored from the official Apache distribution site, not from github or other site. e.g. "A Link to the httphttps://mxnet.incubator.apache.org/install/versions/1.8.0/get_started/download.html" |
Send an email to announce@apache.org, general@incubator.apache.org and dev@mxnet.incubator.apache.org with the subject: [ANNOUNCE] Release Apache MXNet <release number> Release and (incubating) version $release and a body along the lines of:
Info | ||
---|---|---|
| ||
The Apache MXNet (incubating) community is happy to announce Apache MXNet (incubating) version ##.#.#! ( ## if applicable ## ) Release blog post: ( ## -- if applicable -- ## ) Apache MXNet (incubating) is a deep learning framework designed for both efficiency and flexibility. It allows you to mix symbolic and imperative programming to maximize efficiency and productivity. ( ## for patch releases ## ) 1.3.1 is a maintenance release incorporating important bug fixes and important performance improvements. ( ## -- for patch releases -- ## ) A full list of the changes in this release can be found in the release notes: A link to the download page can be found here: If you prefer to build from source and experiment with various compile-time configuration options, use this link to get the instructions: Or You you can download and play with MXNet easily using one of the options below: 1. The Pip packages can be found here: 2. The Docker Images can be found here: Links in Maven to the published Scala packages: https://repository.apache.org/content/repositories/releases/org/apache/mxnet/ and to the experimental Clojure packageshere:
The Pip package: The Release Tagrelease tag: MXNet Resources - Our discussion forum (https://discuss.mxnet.iogithub.com/apache/incubator-mxnet/discussions) Social Media - Apache MXNet website on Twitter (https://mxnet.incubator.apache.org/faq/twitter.com/apachemxnet) For more information on Apache MXNet (incubating), please see:
_______ DISCLAIMER: Apache MXNet (incubating) is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the name of Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. https://cwiki.apache.org/confluence/x/BINjB |
...
3.7. Bump up the versions on the master
Merge the PR you created for the release branch into the master
Add a new PR to bump up the version for next release
- Bump up the version number into v1.x branch if the new version is branched from v1.x. Bump up the version number into master branch if the new version is branched from master. Here is a previous version update PR (https://github.com/apache/incubator-mxnet/pull/19196). Please update this wiki with latest version update PR to stay current.
- make sure no broken/non-existing links are added to the master/v1.x branch. update the version to one up/or next release for pip-releases
3.8. Clean up older releases on ASF mirrors
Clean up older releases stored in https://dist.apache.org/repos/dist/release/incubator/mxnet/ and https://dist.apache.org/repos/dist/dev/incubator/mxnet/
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
# checkout apache release repo svn co https://dist.apache.org/repos/dist/release/incubator/mxnet apache-mxnet-release cd apache-mxnet-release # remove the previous release folder svn rm 1.0.0 svn commit -m 'commit message' # checkout apache dev repo svn co https://dist.apache.org/repos/dist/dev/incubator/mxnet apache-mxnet-dev cd apache-mxnet-dev # remove the previous release candidate folders svn rm 1.1.0.rc* svn commit -m 'commit message' |
...
Task # | Task | Owner | Relative Date | Absolute Date | ||||||
PreReqs for Release Start: | PM | |||||||||
1 | Finalize the Release Date | PM | T-15 | |||||||
2 | High Level List of Features and artifact specs | PM | T-15 | |||||||
3 | Prepare Release Notes - draft 1 | PM | T-15 | |||||||
4 | Inform the community | ReleaseLead | T-15 | |||||||
5 | PR to update the Website into master/RB | WebsiteLead | T-14 | |||||||
6 | PR to update the Docs into master/RB | DocsLead | T-14 | |||||||
7 | PR to update the versions into master/RB | ReleaseLead | T-14 | |||||||
8 | Validate License Headers and top Level LICENSE file | ReleaseLead | T-14 | |||||||
9 | Validate/update submodules | ReleaseLead | T-14 | |||||||
10 | Stabilize the code for CI to pass | ReleaseLead | T-14 | |||||||
11 | Merge all necessary PRs into master | Community | T-13 | |||||||
12 | Code Freeze and Release Start: Cut the Release Branch | ReleaseLead | T-10 | |||||||
13 | Finalize the Release Notes based on PRs that got in - final draft | PM | T-10 | |||||||
14 | PR to update the NEWS and README into the RB | ReleaseLead | T-9 | |||||||
15 | Validate the docs build locally by checking release-specific docs like pip install, etc. | DocsLead | T-9 | |||||||
16 | Test part 1: Run the unit Tests on the Release Branch | ReleaseLead | T-8 | |||||||
17 | Test part 2: Run the Nightly Tests | ReleaseLead | T-8 | |||||||
18 | Create the Github Tag for the rc0 | ReleaseLead | T-7 | |||||||
19 | Create the src tar and sign, Upload the src tar | ReleaseLead | T-7 | |||||||
20 | Validate the signatures | Release Lead | T-7 | |||||||
21 | Clone svn repo and do a manual test | Release Lead | T-7 | |||||||
22 | Update version number in Travis CI used for Scala Build | Release Lead | T-7 | |||||||
Begin Apache Voting | ReleaseLead | T-7 | ||||||||
23 | 22 | Start the vote on dev@ | ReleaseLead | |||||||
24 | 23 | Send out the results of the vote on dev@ | ReleaseLead | T-4 | ||||||
2425 | Revote if necessary (Steps 15-22) | ReleaseLead | Not Accounted For | |||||||
26 | 25 | Start the vote on general@ | ReleaseLead | T-4 | ||||||
27 | 26 | Send out the results of the vote on general@ | ReleaseLead | T-1 | ||||||
End Apache Voting | ||||||||||
28 | 27 | Create the final release tag on github | ReleaseLead | T-1 | ||||||
29 | 28 | Rename, resign and upload the src tar to final dir | ReleaseLead | T-1 | ||||||
2930 | Update the website using tag | Website Lead | T-1 | |||||||
3031 | Release the official pip package | pip Lead | T-1 | |||||||
32 | 31 | Release the official docker images | Docker Lead | T-1 | ||||||
33 | 32 | After 24 hrs, validate the packages are uploaded | Release Lead | T | ||||||
3334 | Draft the offical announce email and review | Release Lead | T-1 | |||||||
3435 | Send out the email on announce@ | Release Lead | T | |||||||
36 | 35 | Update the apache blog | Release Lead | T | ||||||
3637 | update the aws blog | PM | T | |||||||
3738 | send internal announcement | PM | T | |||||||
39 | 38 | Update the version on master | Release Lead | T |
Notes for reference
*NOTES FROM DOCS FOR REFERENCE:* http://incubator.apache.org/guides/releasemanagement.html - 3 +1 votes from IPMC members (these are the votes that count but we should open up to the whole podling community) - For podlings, 2 additional constraints: - Release artifacts must include “incubating” in final file name (ex: apache-mxnet-src-0.10.1-incubating.tar.gz) - Release artifacts must include disclaimer in the release artifacts - The Incubator PMC expects the source releases to be staged on https://dist.apache.org/repos/dist/dev/incubator/podlingName so that they can easily be moved to the release location via svn mv ( http://www.apache.org/dist/incubator/) - After graduating, RC’s go into https://dist.apache.org/repos/dist/dev/ and official releases go into https://dist.apache.org/repos/dist/release/ http://incubator.apache.org/guides/branding.html#disclaimers - Apache Press Team [http://www.apache.org/press/index.html#whoweare] must review and coordinate releases for branding - On website and in release DISCLAIMER file: - Apache Podling-Name is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the name of Apache TLP sponsor. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. - Website should include Apache Incubator logo: http://incubator.apache.org/guides/press-kit.html - Release should include: - DISCLAIMER - LICENSE - NOTICE - attribution notices http://www.apache.org/legal/release-policy.html - A release must contain source package which is cryptographically signed by Release lead with detached signature. It must be tested prior to voting for release. - Release must only contain appropriately licensed code - Please ensure you wait >=24 hours after uploading a release before making announcements so mirrors catch up - Releases of more than 1GB of artifacts require a heads-up to Infrastructure in advance. - Which directory for what build? http://www.apache.org/legal/release-policy.html#build-directories http://www.apache.org/dev/release-distribution.html - Artifacts MUST be accompanied by: - apache-mxnet-src-0.10.1-incubating.asc - contains OpenPGP compatible ASCII armored detached signature - apache-mxnet-src-0.10.1-incubating.sha - SHA checksum (SHOULD) - Publish KEYS file in distribution directory root - Signing keys MUST be published in KEYS file, SHOULD be available in global public keyserver http://www.apache.org/dev/release-signing#keyserver, SHOULD be linked into web of trust - Keys MUST be RSA & 4096 bits http://www.apache.org/dev/release-publishing.html - Apache RAT can assist in checking license compliance http://creadur.apache.org/rat/ - Eventually we should set up a build system to sign our releases with cryptographic signatures. For now we’ll do it manually. http://www.apache.org/dev/release-signing.html - Create a signature and sign releases as mentioned above
...
...