Versions Compared

Old Version 6

changes.mady.by.user Naveen Swamy

Saved on

compared with

New Version 7

changes.mady.by.user Naveen Swamy

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...


ModulePackage GeneratedDependenciesDescription
  1. assembly
  1. mxnet-full-parent_2.11
  2. mxnet-full_2.11-{platform}
  1. mxnet-core_2.11
  2. libmxnet-scala-{platform}

2. core

mxnet-core_2.11

  1. mxnet-init_2.11
  2. mxnet-macros_2.11

3. macros

mxnet-macros_2.11

  1. mxnet-init_2.11
  2. libmxnet-init-scala-{platform}

4. init

mxnet-init_2.11

 

5. init-native

  1. mxnet-scala-init-native-parent
  2. libmxnet-init-scala-osx-x86_64

  3. libmxnet-init-scala-linux-x86_64

mxnet-init_2.11


6. native

  1. mxnet-parent_2.11
  2. libmxnet-scala-{platform}

 mxnet-core_2.11


7. infer

mxnet-infer_2.11

  1. mxnet-core_2.11
  2. libmxnet-scala-{platform}

8. spark

mxnet-spark_2.11

  1. mxnet-core_2.11
  2. libmxnet-scala-{platform}
  3. spark-mllib_2.11

9. examples

mxnet-examples_2.11

  1. mxnet-core_2.11
  2. mxnet-infer_2.11


Publishing Scala Packages to Nexus/Maven

Currently 3 Packages from the above generated are published to Maven. `mxnet_full_2.11-{platform}`.

The following flags are used to build the MXNet Backend(derived from mxnet pip compiler flags)


FLAGSMXNet-Scala OSX

MXNet-Scala Linux CPU

MXNet-Scala Linux GPU
USE_BLASappleopenblas 
USE_LAPACK11 
USE_OPENCV11 (opencv 3.4.1) 
USE_S311 
USE_DIST_KVSTORE01 
USE_SIGNAL_HANDLER11 
USE_OPENMP 11

USE_CUDNN

 01
USE_NCCL 01


Build MXNet backend on OSX

  1. Clone MXNet repo and checkout the release branch
Code Block
languagebash
collapsetrue
https://github.com/apache/incubator-mxnet.git ~/$MXNET_LOC
git fetch all
git checkout -b $RELEASE_BRANCH
cp ~/$MXNET_LOC/make/config.mk ~/$MXNET_LOC


2. Install Lapack

Code Block
languagebash
brew install lapack


setup all the flags

Code Block
echo "ADD_CFLAGS  += -I/usr/local/opt/lapack/include" >> ~/$MXNET_LOC/config.mk
echo "ADD_LDFLAGS += -L/usr/local/opt/lapack/lib"     >> ~/$MXNET_LOC/config.mk
echo "USE_OPENMP = 0" >> ~/$MXNET_LOC/config.mk


Build MXNet backend on Linux-CPU

To build for linux-cpu platform, we use the AWS Deep Learning Base AMI that comes built with all the necessary 

libraries such as opencv/cudnn etc,.

Install LAPACK/BLASMaven on Ubuntu

Code Block
languagebash
sudo apt-get install -y libopenblas-devmaven

setup all the flags

Code Block
echo "USEADD_SIGNAL_HANDLERCFLAGS += 1-I/usr/local/include/opencv" >> ~/$MXNET_LOC./config.mk
echo "ADD_LDFLAGS += -L/usr/local/lib" >> ./config.mk
echo "USE_BLAS = openblas" >> ~/$MXNET_LOC./config.mk 
echo "USE_SIGNAL_HANDLER=1" >> ./config.mk
echo "USE_DIST_KVSTORE = 1" >> ~./$MXNET_LOC/config.mk
echo "USE_S3 = 1" >> ~/$MXNET_LOC./config.mk
echo "USE_OPENMP=1" >> ./config.mk

Note: compiling with OpenBlas throws a segmentation fault on both OSX and Linux platforms

Publishing to Maven


Code Block
languagebash
export GPG_TTY=$(tty)
make scalapkg
make scalarelease-dryrun


Set up your dev environment