...

• Added comprehensive support for sparse matrices. See help on mx.sym.sparse and mx.nd.sparse for more info.
  
• Added limited cpu support for two sparse formats in Symbol and NDArray - CSRNDArray and RowSparseNDArray
• Added a sparse dot product operator and many element-wise sparse operators
• Added a data iterator for sparse data input - LibSVMIter
• Added three optimizers for sparse gradient updates: Ftrl, SGD and Adam
• Added push and row_sparse_pull with RowSparseNDArray in distributed kvstore
  

New Features - Gluon and Autograd

...

• Added

  new loss functions - SigmoidBinaryCrossEntropyLoss, CTCLoss, HuberLoss, HingeLoss, SquaredHingeLoss, LogisticLoss, TripletLoss

  Gluon package, a high-level interface designed to be easy to use while keeping most of the flexibility of low level API. Gluon supports both imperative and symbolic programming, making it easy to train complex models imperatively with minimal impact on performance. Neural networks (and other machine learning models) can be defined and trained with gluon.nn and gluon.rnn packages. 

• gluon.Trainer now allows reading and setting learning rate with with trainer.learning_rate property.

• Added autograd package, which enables automatic differentiation of NDArray operations. Enter code in a autograd.record() block to capture the exact path by which each variable was generated. Use *.backward() on any variable to backpropagate.

• Added Added mx.autograd.grad and  and experimental second order gradient support (though most operators don't support second order gradient yet)

• Added ConvLSTM etc to new loss functions - SigmoidBinaryCrossEntropyLoss, CTCLoss, HuberLoss, HingeLoss, SquaredHingeLoss, LogisticLoss, TripletLoss

• Added ConvLSTM to gluon.contrib

• Autograd now supports cross-device graphs. Use x.copyto(mx.gpu(i)) and x.copyto(mx.cpu()) to do computation on multiple devices

...