Background

Julia's package manager

There are a module from stdlib named Pkg, served as the package manager in Julia, like Python's `pip`.

The package registry and package source

The package manager will download the desired packages from a default registry.
In Python, pip will conduct two actions before installing packags:
       (1) Search the metadata (like pkg name, version, ...etc)
           from the CheeseShop (a.k.a PyPI) for users.
       (2) Fetch them from the CheeseShop if matched.

In Julia, there is a default registry to hold the packages metadata, 
hosted on GitHub: https://github.com/JuliaRegistries/General


And where to fetch the package? The aren't a center for download packages.
It's unlike Python's idea, we don't upload package to a center server in case of Julia.
The package manager will fetch stuffs from the URL listed in the metadata.
It should be a valid git URL for git-clone . e.g. For the package Distributions :

https://github.com/JuliaRegistries/General/tree/master/D/Distributions.

Most of the packages use GitHub for hosting code.

A valid package directory structure

In the process of package installation,
Python's pip will run `./setup.py` from unpacked package dir.
this is kind of protocol between package developer and package manager.

In Julia, the package manager run `./deps/build.jl` from the git-cloned dir.

Here comes an issue of mxnet's main repo.
The Julia package is collected under the subdir `./julia/`, so the setup script
in our case is `./julia/deps/build.jl`.
This breaks the protocol.

Also, in runtime, there is another protocol for phase of module loading.
The entry point of a module must be `./src/PackageName.jl`.
 In our repo, it's `./julia/src/MXNet.jl`.
 So, the directory structure of mxnet main repo is invalid for Julia package manager.
 Putting the url of main repo as package metadata will not work.

The proposed solution

To provide a valid dir structure, it needs a standalone git repo.
We can just treat this standalone repo as the place to "upload" package code,
just like uploading Python package to PyPI in every release.
We reuse the old repo: https://github.com/dmlc/MXNet.jl

  1. Split the julia  dir out via git-subtree 
    cd /path/to/mxnet/
git subtree split -P julia -b MXNet_j

  2. Push the branch MXNet_jl  to the standalone git repo MXNet.jl.
  3. Update the prebuilt binary version in julia/deps/build.jl in that new branch. e.g.:
    https://github.com/dmlc/MXNet.jl/blob/73e0fba911539bf074ee9de71d09541ec5368480/deps/build.jl#L26-L27
  4. Push a new tag for MXNet.jl. The version number should be same as the mxnet main repo.
  5. Trigger the registration bot on the tagged commit.
    @JuliaRegistrator register

    (See https://github.com/JuliaRegistries/Registrator.jl for more information)
  6. Make a release page via GitHub release button.
  7. Relax!

  • No labels