...

New Features - Sparse Tensor Support

• Added comprehensive support for sparse matrices. See help on `mxmx.sym.sparse` and `mxsparse and mx.nd.sparse` for sparse for more info.
  Limited support for fancy indexing. x[idx_arr0, idx_arr1, ..., idx_arrn] is now supported. Full support coming soon in next release. Checkout master to get a preview.
• Added limited cpu support for two sparse formats in Symbol and NDArray - CSRNDArray and RowSparseNDArray
• Added a sparse dot product operator and many element-wise sparse operators
• Added a data iterator for sparse data input - LibSVMIter
• Added three optimizers for sparse gradient updates: Ftrl, SGD and Adam
• Added push and row_sparse_pull with RowSparseNDArray in distributed kvstore
  

New Features - Autograd and Gluon

...

• Added new loss functions

...

• -

...

• SigmoidBinaryCrossEntropyLoss,

...

• CTCLoss,

...

• HuberLoss,

...

• HingeLoss,

...

• SquaredHingeLoss,

...

• LogisticLoss,

...

• TripletLoss

...

• gluon.

...

• Trainer now allows reading and setting learning rate with

...

• trainer.learning_

...

• rate property.

• Added

...

• mx.autograd.

...

• grad and experimental second order gradient support (though most operators don't support second order gradient yet)

• Added

...

• ConvLSTM etc to

...

• gluon.

...

• contrib

• Autograd now supports cross-device graphs. Use

...

• x.copyto(mx.gpu(i))

...

• and

...

• x.copyto(mx.cpu())

...

• to do computation on multiple devices

...

Other New Features

• Added limited support for fancy indexing, which allows you to very quickly access and modify complicated subsets of an array's values. x[idx_arr0, idx_arr1, ..., idx_arrn] is now supported. Full support coming soon in next release. Checkout master to get a preview.
  

• Random number generators in

...

• mx.nd.random.*

...

• and

...

• mx.sym.random.*

...

• now supports both CPU and GPU

...

• NDArray and

...

• Symbol now supports "fluent" methods. You can now use

...

• x.exp()

...

• etc instead of

...

• mx.nd.exp(x)

...

• or

...

• mx.sym.exp(x)

...

• Added

...

• mx.rtc.

...

• CudaModule for writing and running CUDA kernels from python

• Added

...

• multi_

...

• precision option to optimizer for easier float16 training

Performance

• Enabled JIT compilation. Autograd and Gluon hybridize now use less memory and has faster speed. Performance is almost the same with old symbolic style code.
  
• Full support for NVidia Volta GPU Architecture and Cuda 9. Training is up to 3.5x faster than Pascal when using float16.

API Changes

• Operators like

...

• mx.sym.linalg_*

...

•  and mx.sym.random_*

...

• are now moved to

...

• mx.sym.linalg.*

...

• and

...

• mx.sym.random.*

...

• . The old names are still available but deprecated.

...

• sample_*

...

•  and random_*

...

• are now merged as

...

• random.*

...

• , which supports both scalar

...

• and  NDArray distribution parameters.

Bug-fixes

• Fixed a bug that causes

...

• argsort operator to fail on large tensors

...

• Fixed numerical stability issues when summing large tensors

...