Page History

...

To: general@incubator.apache.org
Subject: [RESULTS][VOTE] Release Apache MXNet (incubating) version $release.rc#

Dear Community,

The voting for releasing Apache MXNet (incubating) $release.rc# has passed with # +1 votes (# binding) and no 0 or -1 votes.
+1 votes:
* ###### / binding
* ############

Vote thread can be found at:
https://lists.apache.org/thread.html/##############

Thanks everyone for taking the time to review and vote for the release!
We will continue the rest of release process and send out the announcement email in the coming days.

Thanks,
$RM

...

# Step 1: Download and Untar
tar -zxvf apache-mxnet-src-0.11.0.rc3-incubating.tar.gz

# Step 2: rename (remove the rc) and tar again
mv apache-mxnet-src-0.11.0.rc3-incubating apache-mxnet-src-0.11.0-incubating
gtar -czvf apache-mxnet-src-0.11.0-incubating.tar.gz apache-mxnet-src-0.11.0-incubating

# Step 3: Resign now
 
# Step 4: First upload here - 
svn mv https://dist.apache.org/repos/dist/dev/incubator/mxnet/0.11.0/
 
# Step 5: Move from dev to release
svn mv https://dist.apache.org/repos/dist/dev/incubator/mxnet/0.11.0/ https://dist.apache.org/repos/dist/release/incubator/mxnet/ --message "0.11.0 release moving from dev to release repo"

 
# Wait 24hours for the src mirroring. The final download page for the announce email is somewhere here (example 0.11.0) - https://www.apache.org/dyn/closer.cgi/incubator/mxnet/0.11.0-incubating/

Step 3.1.3: Build, test, and publish

...

pip

...

wheels and python docker images:

For linux, we can trigger a mxnet-cd-release job to let the existing pipelines build and test release pipeline that builds, tests and publishes the pip wheels and python docker images for us with just a few steps.:

1. Go to https://jenkins.mxnet-ci.amazon-ml.com/job/restricted-mxnet-cd/job/mxnet-stable-release-pipeline/ and make sure you have write access to the Jenkins system. You can contact Joe (joseph.evans@gmail.com) to be added.
2. We would like to trigger a mxnet-cd-pipeline job which in turn triggers a down-stream mxnet-cd-release job.
   1. The existing mxnet-cd-pipeline and mxnet-cd-pipeline-v1.x pipelines point to the master and v1.x branch of https://github.com/apache/incubator-mxnet respectively. So, what what we would want to do is duplicate one of them, depending on whether it's a 2.x or 1.x release, and point the pipeline to the correct release tag.
   2. Click "New Item" on the sidebar.
   3. Give the new pipeline a meaningful name, scroll down to the bottom of the page, and paste in "mxnet-cd-pipeline" or "mxnet-cd-pipeline-v1.x" to field "Copy from".
   4. Click "OK".
   5. Go into the new pipeline and click "Configure" on the sidebar.
   6. Under "Pipeline" find "Branches to Build." Use refs/tags/<tagName> as the branch specifier.
   7. Click "Save"
   8. Now also replicate the mxnet-cd-release-job or mxnet-cd-release-job-1.x pipeline and give it a meaningful name. Here we do not need to mess with any settings.
   9. Go back to configure the duplicated cd pipeline created in step b~g. Find the parameter "CD_RELEASE_JOB_NAME", change the default value to the release job pipeline created in step h.
   10. Click "Save"
   11. Now, click "Build with Parameters" on the side-bar. Choose the variants of MXNet to build and select the "RELEASE_BUILD" box and hit the "Build" Button.
3. Now the cd pipeline will be triggered and in turn it will spawn several jobs in the release job pipeline. Specifically, "mxnet_lib/static" will build the MXNet lib statically, and "python/pypi" will publish the pip wheels to https://dist.mxnet.io/python/.
4. If any variant of MXNet failed in either of the two jobs, the pip wheel of that variant will not be published. In this case we can try to trigger the cd pipeline again for that variant or root cause and fix the issue based on the pipeline log.
5. Click "Build with Parameters" from the left pane, modify the following fields as needed and click on "Build"
   1. MXNET_VARIANTS : Make sure all supported cu* flavors are present
   2. RELEASE_BUILD" : Keep the box Unchecked
   3. CD_RELEASE_JOB_NAME : Do not modify this field
   4. VERSION : Mention the MXNet release tag here e.g. 1.8.0, 2.0.0 (Make sure the tag exists in MXNet github repository)
6. This triggers the release pipelines for different cpu and cu* flavors which can be found here https://jenkins.mxnet-ci.amazon-ml.com/job/restricted-mxnet-cd/job/mxnet-stable-release-job/ . You can click on "Open Blue Ocean" from the left pane to check pipeline status. There are 3 pipelines that get trigerred:
   1. First pipeline builds the libmxnet binaries for different flavors. Only after this pipeline completes (takes around 2.5 hours), the other two pipelines are triggered, that run in parallel.
   2. Second pipeline "python/pypi" publishes the pip wheels to https://dist.mxnet.io/python/.
      
   3. Third pipeline "python/docker" publishes the python docker images to https://gallery.ecr.aws/w6z5f7h2/mxnet/python.
7. If any job in any of the three pipelines fails, go through the corresponding logs and either
   1. Re-trigger the pipeline in case it's a flaky failure or
      
   2. Root cause and fix the issue and then re-trigger the pipeline.
8. Once all pipelines are green, Python wheels and Python docker images for all variants are published to their respective targets. Now we need to move these released artifacts to their final locations so that they are available to users:
   1. Python wheels (for this release) from https://dist.mxnet.io/python/ go to PyPI.
      1. Identify the wheels by the version number, date created, and variant string
      2. Download the above wheels, give them another test if needed, and upload them to PyPI
      3. For more info regarding publishing pip wheels to Pypi, please contact zhasheng@apache.org
   2. Python docker images (for this release) from https://gallery.ecr.aws/w6z5f7h2/mxnet/python go to dockerhub.
      
      1. Identify the wheels by the
   Once all variants succeed to publish, we can go to https://dist.mxnet.io/python/ and identify the wheels we just created by the
   1. 1. version number, date created, and variant string
   . We can simply download those wheels
   1. 2. Pull the above docker images, give them another test if needed, and
   upload
   1. 1. push them to
   PyPI.
   
9. Now we can remove the two duplicated pipelines from https://jenkins.mxnet-ci.amazon-ml.com/job/restricted-mxnet-cd/.

For publishing pip wheels to Pypi please contact zhasheng@apache.org for more info.

1. 1. 1. dockerhub

For mac, the following automated script can be used to build the wheel.For mac, the following automated script can be used to build the wheel.(MacOS machine is required to generate wheels for mac)

#!/bin/bash

# Install brew
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Install developement tools
brew install nasm
brew install automake
brew install libtool
brew install ninja
brew install pkg-config
python -m pip install contextvars
python -m pip install numpy
python -m pip install requests

# Clone MXNet
cd $HOME
git clone -b v1.x --single-branch https://github.com/apache/incubator-mxnet.git # Use release tag instead
cd $HOME/incubator-mxnet/
git submodule update --init --recursive

# Set ENV variables and build statically
export IS_RELEASE=True
export mxnet_variant=cpu
echo $(git rev-parse HEAD) >> python/mxnet/COMMIT_HASH
mkdir -p $HOME/pip_build/mxnet-build
cp -r * $HOME/pip_build/mxnet-build
export CODE_DIR=$(pwd)
cd $HOME/pip_build/mxnet-build
CMAKE_STATICBUILD=1 ./tools/staticbuild/build.sh $mxnet_variant pip

# Package wheel
cd $HOME/pip_build
cp -r mxnet-build/tools/pip/* .
python setup.py bdist_wheel

# Copy wheel to $HOME/pip_build
cp $HOME/pip_build/dist/mxnet* $HOME/pip_build

Wheels are publish manually to pypi.

Please note that the release tag is usually created on a branch that's forked from v1.x and master. In that case it may lack some of the latest ci/cd fixes on the v1.x and master branch. We might need to manually port back those fixes for the cd pipeline to work.

Step 3.1.4: Build, test, and publish the Docker images: (by Meghna)

This was done as follows in 0.11.0. In the future use the automated pipeline - 

Update the tag in the tool.sh file or select the correct tag in the Jenkins Job

Select the correct Repo name  = mxnet

Trigger the Job with IS_PUBLISH=false

Login to the slave and test each image for correct version by checking logs and/or Readme. 

Manually upload the 20 images that are built. 

 $HOME/pip_build

Wheels are publish manually to pypi.

Please note that the release tag is usually created on a branch that's forked from v1.x and master. In that case it may lack some of the latest ci/cd fixes on the v1.x and master branch. We might need to manually port back those fixes for the cd pipeline to work.Make sure to update the latest and previous docker images

Step 3.1.

...

4: Verify the resources are uploaded

Check that the resources are present in http://www.apache.org/dist/incubator/mxnet. It may take a while for them to be visible. This will be mirrored throughout the Apache network. There are a few remaining steps.  

...