Page History

Important Links

...

1. Jenkinsfile_website_mxnet_build - builds the MXNet binaries; this runs on the ubuntu_cpu_lite Docker image; it listed first because this functionality is usually the first phase in all of the following Jenkinsfiles.
2. Jenkinsfile_website_c_docs - builds MXNet's binary, then runs the C docs on a minimal custom Docker container that has `doxygen`
3. Jenkinsfile_website_clojure_docs - builds MXNet's binary, then runs the Clojure docs on a minimal container that has the Scala and Clojure dependencies
4. Jenkinsfile_website_full - builds MXNet's binary, then runs all of the documentation sets and combines them into a single website archived artifact .tgz file that is used for website deployments. The last step calls a separate publish pipeline in Jenkins.
5. Jenkinsfile_website_full_pr - similar to the previous but does not archive
6. Currently set to test all docs sets, but only publish jekyll & python docs to S3 as part of a PR (the website is not functional on S3, so this is of limited utility)
7. Jenkinsfile_website_java_docs - builds MXNet's binary, then runs the java docs
8. Jenkinsfile_website_jekyll_docs - generates the jekyll website
9. Jenkinsfile_website_julia_docs - builds MXNet's binary, then generates the Julia website
10. Jenkinsfile_website_python_docs- builds MXNet's binary, then generates the Python website
11. Jenkinsfile_website_r_docs - builds MXNet's binary, then generates the R pdf
12. Jenkinsfile_website_scala_docs - builds MXNet's binary, then generates the Scala website

...

Prerequisites - You will need Docker.  Refer You will also need a Docker account from hub.docker.com. Refer to [ci/README.md](https://github.com/apache/incubator-mxnet/blob/master/ci/README.md) for setup instructions for Docker and docker-python. Be sure to run `docker login` before running the following script. These are required for running the `build.py` script.

Step 1 - You first need have build the MXNet binary (that matches what you have checked out). The following will generate the binaries that will be found in `/lib/` in your local MXNet repo folder. The different docs sets will be looking for the binary there.```bash

ci/build.py --docker-registry mxnetci --platform ubuntu_cpu_lite /work/runtime_functions.sh build_ubuntu_cpu_docs

```Step 2 -  Generate the doc set that you require. The following example generates Python docs in your local `docs/_build/` folder.```bash

ci/build.py --docker-registry mxnetci --platform ubuntu_cpu_python /work/runtime_functions.sh build_python_docs

```Troubleshooting - You can interact with the Docker container, make changes, and force a rebuild of the docs. 

```bash

docker run -v /home/ubuntu/incubator-mxnet:/work/mxnet -u 1000:1000 -ti mxnetci/build.ubuntu_cpu_python bash

```

You must adjust the folder mapping in the command to match wherever you're running the MXNet from. 

Once inside the container you can trigger another build by calling a function from runtime_functions.sh directly. For example:```bash#

ci/docker/runtime_functions.sh build_python_docs

...

```

or directly running

```

pushd docs/python_docs

eval "$(/work/miniconda/bin/conda shell.bash hook)"

conda env create -f environment.yml -p /work/conda_env

conda activate /work/conda_env

pip install themes/mx-theme

pip install -e /work/mxnet/python --user

pushd python

make clean

make html EVAL=0

```

To fully emulate the Jenkinsfile that generates the website, you may call each runtime_function for each docs package and for the Jekyll site then deploy the contents of your `docs/_build/` folder.

...

Automatically generated docs for each API are hosted their own folder with the following structure:
* /api/$lang - Example: api/python
* /api/$lang/docs/ - An overview.
* /api/$lang/docs/api/ - the automatically generated API reference
* /api/$lang/docs/guide/ - overview on how to use it and links to important information
* /api/$lang/docs/tutorials/ - overview on the list of tutorials

Publishing Versioned Artifacts

The artifacts are being posted on S3 in MXNet's public folder. For example, the Julia microsite can be found at: https://mxnet-public.s3.us-east-2.amazonaws.com/docs/v1.5.0/julia-artifacts.tgz

You must have write access to this bucket to publish new artifacts. You may request access from a committer. Anyone can read from the bucket.

Preview the `ci/publish/website/publish_artifacts.sh` script and verify the settings. You may want to change the version number, which will affect the bucket location.

TODO

1. CI for python docs must use 2-GPU nodes; jenkinsfile spec must be updated
2. Tidy up R env - probably doesn't need the ubuntu_docs.sh step in the Docker installs
3. Refactor BLC jobs as they're probably broken on the new setup
4. create pipelines that generate a "last successful build" artifact for each doc set; triggered upon a merge event

...