Performance

• Added full support for NVIDIA Volta GPU Architecture and CUDA 9. Training is up to 3.5x faster than Pascal when using float16.
• Enabled JIT compilation. Autograd and Gluon hybridize now use less memory and has faster speed. Performance is almost the same with old symbolic style code.

...

• Added limited support for fancy indexing, which allows you to very quickly access and modify complicated subsets of an array's values. x[idx_arr0, idx_arr1, ..., idx_arrn] is now supported. Full support coming soon in next release. Checkout master to get a preview.
  

• Added new loss functions - SigmoidBinaryCrossEntropyLoss, CTCLoss, HuberLoss, HingeLoss, SquaredHingeLoss, LogisticLoss, TripletLoss

• Random number generators in mx.nd.random.* and mx.sym.random.* now support both CPU and GPU

• NDArray and Symbol now supports "fluent" methods. You can now use x.exp() etc instead of mx.nd.exp(x) or mx.sym.exp(x)

• Added mx.rtc.CudaModule for writing and running CUDA kernels from python

• Added multi_precision option to optimizer for easier float16 training

New Features - Sparse Tensor Support

...

• Added support for sparse matrices. See help on mx.sym.sparse and mx.nd.sparse for more info
  
• Added limited cpu support for two sparse formats in Symbol and NDArray - CSRNDArray and RowSparseNDArray
• Added a sparse dot product operator and many element-wise sparse operators
• Added a data iterator for sparse data input - LibSVMIter
• Added three optimizers for sparse gradient updates: Ftrl, SGD and Adam
• Added push and row_sparse_pull with RowSparseNDArray in distributed kvstore

...