Versions Compared

Old Version 10

changes.mady.by.user Meghna Baijal

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (zhasheng)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Want to make changes? Edit here -  - https://docs.google.com/document/d/1SdFwiTXlFBMmyVfEHpe7s3jteWxqfUzsxTgbagcZWzo/edit?usp=sharing

Performance

  • Enhanced the performance of `sparse.dot` operator

    .

  • MXNet now automatically set OpenMP to use all available CPU cores to maximize CPU utilization when `NUM_OMP_THREADS` is not set.

  • Unary and binary operators now avoid using OpenMP on small arrays if using OpenMP actually hurts performance due to multithreading overhead.

  • Significantly improved performance of `broadcast_add`, `broadcast_mul`, etc on CPU.

  • Added bulk execution to imperative mode. You can control segment size with `mxnet.engine.bulk`. As a result, the speed of Gluon in hybrid mode is improved, especially on small networks and multiple GPUs.

  • Improved speed for `ctypes` invocation from Python frontend.

    Performance regression automation (vikram)

New Features - Gradient Compression

...

[Experimental]

  • Speed up multi-GPU and distributed training by compressing communication of gradients. This is especially effective when training networks with large fully-connected layers. In Gluon this can be activated with `compression_params` in Trainer.

    Enabled users to train models faster by reducing communication bandwidth between compute nodes

New Features - Support of NVIDIA Collective Communication Library

...

(NCCL) [Experimental]

  • Use kvstore=’nccl’ for (in some cases) faster training on multiple GPUs.

  • Significantly faster than kvstore=’device’ when batch size is small.

  • It is recommended to set environment variable NCCL_LAUNCH_MODE to PARALLEL when using NCCL version 2.1 or newer.

  • Implemented multi-GPU and multi-node collective communication primitives that are performance optimized for NVIDIA GPUs
  • Enabled users to train models faster

New Features - Advanced Indexing

...

  • Enabled users to leverage the powerful array operations in MXNet (e.g. supports MXNet NDArray and Numpy ndarray as index

New Features - Caffe to MXNet translator

  • Enabled users to migrate Caffe code to MXNet using a new code translation tool

New Features - Gluon

  • Gluon - GA (vikram)
  • Gluon IO performance (sukwon, Mu)

Other Planned Features

[General Availability]

New Features - Gluon [General Availability]

  • Performance optimizations discussed above

  • Added support for loading data in parallel with multiple processes to gluon.data.DataLoader. The number of workers can be set with num_worker. Does not support windows yet.

  • Added Block.cast to support networks with different data types, e.g. float16.

  • Added Lambda block for wrapping a user defined function as a block.

  • Generalized `gluon.data.ArrayDataset` to support arbitrary number of arrays.

New Features - ARM / Raspberry Pi support [Experimental]

New Features - NVIDIA Jetson support [Experimental]

  • MXNet now compiles and runs on NVIDIA Jetson TX2 boards with GPU acceleration.

  • You can install the python MXNet package on a Jetson board by running - `$ pip install mxnet-jetson-tx2`

New Features - Sparse Tensor Support [General Availability]

  • Added more sparse operators: `contrib.SparseEmbedding`, `sparse.sum` and `sparse.mean`

  • Added `asscipy()` for easier conversion to scipy

  • Added `check_format()` for sparse ndarrays to check if the array format is valid

    Nvidia PR - https://github.com/apache/incubator-mxnet/pull/8294

Bug Fixes

  • Fixed a[-1] indexing doesn't work on

    • NDArray

  • `NDArray`

  • Fixed `expand_dims` if axis < 0

  • Fixed a bug that causes topk to produce incorrect result on large arrays.

  • Improved numerical precision of unary and binary operators for `float64` data.

  • Fixed derivatives of log2 and log10. They used to be the same with log.

  • Fixed a bug that causes MXNet to hang after fork. Note that you still cannot use GPU in child processes after fork due to limitations of CUDA.

  • Fixed a bug that causes `CustomOp` to fail when using auxiliary states.

  • Fixed a security bug that is causing MXNet to listen on all available interfaces when running training in distributed mode.

Doc Updates

  • Added a security best practices document under FAQ section.

  • Fixed License Headers including restoring copyright attributions.

  • Documentation updates

  • Links for viewing source

Known Issues

  • Attempt to run distributed training with FP16 lead to failure.

  • PR-8904: Fix multiprocessing too many open files

    • Problem: The data-loader has a file-descriptor leak when the number of files being opened exceeds the ulimit of the user session.
    • Workaround: Increase the ulimit of the user session or at the OS level using any of the following steps:
      • $ ulimit -n unlimited  (to make the fd limit as unlimited), OR
      • $ ulimit -n 60000  (to make the fd limit as 60,000 files assuming you have around but less than 60,000 files to be opened) OR
      • $ echo 65535 > /proc/sys/fs/file-max (to increase the system-wide file limit) along with the ulimit command.
      • Further details: https://
    github/apache/incubator-mxnet/issues/7749 (Meghna)Documentation updates (markhama)

How to build MXNet

Please follow the instructions at https://mxnet.incubator.apache.org/install/index.html 

List of submodules used by Apache MXNet (Incubating) and when they were updated last

Submodule:: Last updated by MXNet:: Last update in submodule

  1. cub@:: 31-Jul :: 28-Aug

  2. dlpack@: 08-Sep :: 06-Oct

  3. dmlc-core@: 08-Sep:: 06-Oct

  4. mshadow@: 03-Oct:: 09-Oct

  5. nnvm@: 10-Sep:: 10-Oct

  6. ps-lite@: 22-Nov:: 21-Nov