...

• split_v2 (#13687)
• Gradient multiplier (contrib) operator (#13632)
• Image normalize operator - GPU support, 3D/4D inputs (#13802)
• Image ToTensor operator - GPU support, 3D/4D inputs (#13837)
• Add Gluon Transformer Crop (#14259)
• GELU (#14449)
• AdamW operator (Fixing Weight Decay Regularization in Adam) (#13728)
• [MXNET-1382] Add the index_array operator (#14638)
• add an operator for computing the likelihood of a Hawkes self-exciting process (#14683)
• Add numpy linspace (#14927)

Feature Improvements

Operators

• make ROIAlign support position-sensitive pooling (#13088)
• Add erfinv operator for calculating inverse error function (#13811)
• Added optional parameters to BilinearResize2D to do relative scaling (#13985)
• MXNET-1295 Adding integer index support to Sequence* family of operators. (#13880)
• Export resize and support batch size (#14014)
• CUDNN dropout (#13896)
• Relaxing type requirements for slice_like op (#14097)
• Relaxing type requirements for reshape_like op (#14325)
• Parallelize CPU version and add GPU version of boolean_mask op (#14090)
• Add NHWC layout support to Pooling (cpu, gpu cuda, gpu cuDNN) (#13749)
• Multi-precision AdamW update op (#14171)
• [op] add back support for scalar type rescale_grad argument for adamw_update/mp_adamw_update (#14221)
• move choose_element_0index to operator (#14273)
• Optimize NMS (#14290)
• Optimize NMS part 2 (#14352)
• add background class in box_nms (#14058)
• Use cudnn for dropout by default (#14278)
• In-place updates for Nadam, Adadelta, Adamax and SGLD (#13960)
• Aggregate SGD (#13346)
• Add proper exception message for negative shape in array creation routines (#14362)
• Support multi-threading for Custom Operator (#14363)
• moveaxis operator now accepts negative indices and sequence of ints as well. (#14321)
• Support SyncBatchNorm5D (#14542)
• Add nd.power and sym.pow (#14606)
• Change RNN OP to stateful (#14476)
• Add imresize and copyMakeBorder to mx.image (#13357)
• add ctx for rand_ndarray and rand_sparse_ndarray (#14966)
• Add cpu implementation for Deformable PSROIPooling (#14886)
• Add warning for fp16 inputs with MXNET_SAFE_ACCUMULATION=0 (#15046)
• Safe LayerNorm (#15002)
• use MXNET_SAFE_ACCUMULATION for softmax accumulator (#15037)
• LayerNorm acceleration on GPU (#14935)
• Add matrix inversion operator in linalg (#14963)
• implementation for equivalence of tf.moments (#14842)
• Use env var to enforce safe accumulation in ReduceAxesCompute (#14830)
• [MXNet-1211] Factor and "Like" modes in BilinearResize2D operator (#13226)
• added extraction/generation of diagonal and triangonal matrices to linalg (#14501)
• [Mxnet-1397] Support symbolic api for requantize and dequantize (#14749)
• [MXNET-978] Support higher order gradient for log. (#14992)
• Add cpu implementation for Deformable Convolution (#14879)

MKLDNN

• Feature/mkldnn static (#13628)
• Feature/mkldnn static 2 (#13503)
• support mkl log when dtype is fp32 or fp64 (#13150)
• Add reshape op supported by MKL-DNN (#12980)
• Move the debug output message into MXNET_MKLDNN_DEBUG (#13662)
• Integrate MKLDNN Conv1d and support 3d layout (#13530)
• Making MKL-DNN default on MXNet master (#13681)
• Add mkldnn OP for slice (#13730)
• mkldnn s8 conv API change for master (#13903)
• [MKLDNN] Enable signed int8 support for convolution. (#13697)
• add mkldnn softmax_output (#13699)
• MKLDNN based Quantized FullyConnected Operator and its fusion (#14128)
• Fix entropy for uint8 (#14150)
• Update MKL-DNN to v0.18 release (was: fix the Dense layer issue) (#13668)
• [MKL-DNN] Enable s8 support for inner product and 3d input with flatten=false (#14466)
• Optimize transpose operator with MKL-DNN (#14545)
• [MKLDNN] Remove repeat parts in MKLDNN.md (#14995)
• [MKLDNN] Enable more convolution + activation fusion (#14819)
• Update MKL-DNN submodule to v0.19 (#14783)
• Add mkldnn_version.h to pip package (#14899)
• [MKLDNN] add quantized sum (#14614)
• [MKLDNN]Refactor requantize to speed up execution (#14608)
• [MKLDNN]Add quantized relu (#14604)
• Add MKLDNN headers to pip package (#14339)
• add symbolic link to mkldnn header files in include (#14300)
• disable default MKLDNN for cross compilation (#13893)
• Update MKLDNN_README.md (#13653)
• [Quantization] Support zero-size tensor input for quantization flow (#15031)
• Support 3D input for MKL-DNN softmax operator (#14818)
• Add primitive cache for MKL-DNN sum(elemwise_add operator (#14914)
• Fix reshape to add in-place back (#14903)
• [int8] Add MobileNetV2_1.0 & ResNet18 Quantization (#14823)
• [MKLDNN]Improve quantizeV2 and dequantize latency (#14641)
• added mkldnn dependency for plugin compile target (#14274)
• Support Quantized Fully Connected by INT8 GEMM (#12922)

ONNX

• ONNX export: Instance normalization, Shape (#12920)
• ONNX export: Logical operators (#12852)
• ONNX import/export: Size (#13112)
• ONNX export: Add Flatten before Gemm (#13356)
• ONNX import/export: Add missing tests, ONNX export: LogSoftMax (#13654)
• ONNX import: Hardmax (#13717)
• [MXNET-898] ONNX import/export: Sample_multinomial, ONNX export: GlobalLpPool, LpPool (#13500)
• ONNX ops: norm exported and lpnormalization imported (#13806)
• [MXNET-880] ONNX export: Random uniform, Random normal, MaxRoiPool (#13676)
• ONNX export: Add Crop, Deconvolution and fix the default stride of Pooling to 1 (#12399)
• onnx export ops (#13821)
• ONNX export: broadcast_to, tile ops (#13981)
• ONNX export: Support equal length splits (#14121)

TensorRT

• [MXNET-1252][1 of 2] Decouple NNVM to ONNX from NNVM to TenosrRT conversion (#13659)
• [MXNET-703] Update to TensorRT 5, ONNX IR 3. Fix inference bugs. (#13310)
• [MXNET-703] Minor refactor of TensorRT code (#13311)
• reformat trt to use subgraph API, add fp16 support (#14040)

FP16 Support

• Update mshadow to support batch_dot with fp16. (#13716)
• float32 → float16 cast consistency across implementations (#13857)
• modifying SyncBN doc for FP16 use case (#14041)
• support dot(vector, vector) for fp16 inputs on GPU (#14102)
• softmax for fp16 with fp32 accumulator (#14098)
• [MXNET-1327] Allow RNN Layers to be initialized to fp16 (#14219)
• fp16 safe norm operator (#14616)
• NAG Optimizer with multi-precision support (#14568)

Deep Graph Library(DGL) support

• Add graph_compact operator. (#13436)
• Accelerate DGL csr neighbor sampling (#13588)

Horovod Integration

• Add extra header file to export for error checking (#13795)
• whitelist symbols for using MXNet error handling externally (#13812)
• Use CPUPinned context in ImageRecordIOParser2 (#13980)
• Add pin_device_id option to Gluon DataLoader (#14136)

Dynamic Shape

• [MXNET-1315] Add checks for dynamic-shaped operators in CachedOp (#14018)
• [MXNET-1325] Make InferShapeAttr a standalone pass (#14193)
• [MXNET-1324] Add NaiveRunGraph to imperative utils (#14192)
• [MXNET-1352] Allow dynamic shape in while_loop and if conditionals (#14393)

Backend Engine

• Add infer_type_partial (#14214)
• Tidy up storage allocation and deallocation (#14480)
• Add MXEnginePushAsync and MXEnginePushSync C APIs (#14615)
• Enhance subgraph API (#14113)
• Enhance PartitionGraph (#14277)
• Allow clearing gpu cache (#14252)
• Fix warning / static function in header. (#14900)
• Simplify creation of NodeEntry instances and use emplace_back (#14095)
• Add unpooled gpu memory type (#14716)
• [MXNET-1398] Enable zero-copy from numpy to MXNet NDArray (#14733)
• Use DEFAULT macro in C APIs (#14767)
• Avoid unnecessary vector copies in imperative_utils.cc (#14665)
• Support populating errors back to MXNet engine in callback (#13922)
• Restore save/load ndarray to 1.4.1 (#15073)
• Enable serializing/deserializing ndarrays in np_shape semantics (#15090)
• [numpy] Support zero-dim and zero-size tensors in MXNet (#14661)
• Rename np_compat to np_shape (#15063)
• [MXNET-1330] Bring nnvm::Tuple to mxnet::Tuple (#14270)

Large Tensor Support

• Large array support for randint (#14242)
• [MXNET-1185] Support large array in several operators (part 1) (#13418)
• [MXNET-1401] adding more operators to test support for Large Tensor (#14944)
• [MXNET-1410]Adding Large Tensor Support for tensor transpose (#15059)

Quantization

• Exclude concat layer for gpu quantization (#14060)
• Enhance gpu quantization (#14094)
• Register fake grad to subgraph and quantized operators (#14275)
• Add int8 data loader (#14123)

Profiler

• [MXNET-857] Add initial NVTX profiler implementation (#12328)

CoreML

• Add more support for mxnet_to_coreml (#14222)

Front End API

Gluon

• Add pixelshuffle layers (#13571)
• [MXNET-766] add dynamic_unroll RNN for HybridBlock (#11948)
• add pos_weight for SigmoidBinaryCrossEntropyLoss (#13612)
• Rewrite dataloader with process pool, improves responsiveness and reliability (#13447)
• Complimentary gluon DataLoader improvements (#13606)
• [Fit-API] Adress PR comments (#14885)
• [Fit API] update estimator (#14849)
• [MXNET-1396][Fit-API] Update default handler logic (#14765)
• [Fit API] improve event handlers (#14685)
• move to gluon contrib (#14635)
• move estimator to contrib (#14633)
• [MXNet-1340][Fit API]Update train stats (#14494)
• [MXNet-1334][Fit API]base class for estimator and eventhandler (#14346)
• [MXNET-1333] Estimator and Fit API (#14629)
• Add support for fast variable-length LSTM (#14208)
• Add the Gluon Implementation of Deformable Convolution (#14810)
• hybridize rnn and add model graph (#13244)

Python

• Python BucketingModule bind() with grad_req = 'add' (#13984)
• Refine runtime feature discovery python API and add documentation to … (#14130)
• Runtime feature detection (#13549)
• Add dtype visualization to plot_network (#14066)
• [MXNET-1359] Adds a multiclass-MCC metric derived from Pearson (#14461)
• support long for mx.random.seed (#14314)
• Optimization of metric evaluation (#13471)
• [MXNET-1403] Disable numpy's writability of NDArray once it is zero-copied to MXNet (#14948)
• Refactor ImageRecordIter (#14824)

Language Bindings

Scala

• [MXNET-1260] Float64 DType computation support in Scala/Java (#13678)
• [MXNET-1000] get Ndarray real value and form it from a NDArray (#12690)
• Now passing DType of Label downstream to Label's DataDesc object (#14038)
• Scala interpreter instructions (#14169)
• Add default parameters for Scala NDArray.arange (#13816)
• [MXNET-1287] Up scala comp (#14667)
• [MXNET-1385] Improved Scala Init and Macros warning messages (#14656)
• Remove all usages of makefile for scala (#14013)
• Update scala-package gitignore configuration. (#13962)
• [MXNET-1177]Adding Scala Demo to be run as a part of Nightly CI (#13823)
• [MXNET-1287] Miscellaneous Scala warning fixes (#14658)
• Fix jar path and add missing ones for spark jobs (#14020)
• [MXNET-1155] Add scala packageTest utility (#13046)
• [MXNET-1195] Cleanup Scala README file (#13582)
• Add scalaclean to make clean (#14322)
• Add maven wraper to scala project. (#13702)
• Add new Maven build for Scala package (#13819)
• [MXNET-1287] Feat dep (#14668)
• add Apache header on all XML (#14138)
• update the version name (#14076)
• change to compile time (#13835)
• [MXNET-918] Random module (#13039)
• Avoid secondary deployment of package to local (#14647)

Java

• [MXNET-1180] Java Image API (#13807)
• [MXNET-1285] Draw bounding box with Scala/Java Image API (#14474)
• Add BERT QA Scala/Java example (#14592)
• [MXNET-1232] fix demo and add Eclipse support (#13979)
• [MXNET-1331] Removal of non-MXNET classes from JAR (#14303)
• Java install info update (#13912)
• [MXNET-1226] add Docs update for MXNet Java (#14395)
• [MXNET-1383] Java new use of ParamObject (#14645)
• MXNET-1302 Exclude commons-codec and commons-io from assembled JAR (#14000)

C++

• print error message for mxnet::cpp::Operator::Invoke when failed (#14318)
• build docs with CPP package (#13983)
• Update inception_inference.cpp (#14674)
• Optimize C++ API (#13496)

Clojure

• [Clojure] - Add Spec Validations to the Optimizer namespace (#13499)
• [Clojure] Add Spec Validations for the Random namespace (#13523)
• [Clojure] Correct the versions in the README so they correspond to the latest maven.org release ([#13507)
• Port of scala infer package to clojure (#13595)
• Clojure example for fixed label-width captcha recognition (#13769)
• Update project.clj file to use the snapshots repo to be able to pull (#13935)
• [Clojure] Add resource scope to clojure package (#13993)
• [clojure-package] improve docstrings in image.clj (#14307)
• [Clojure] Helper function for n-dim vector to ndarray (#14305)
• [clojure]: add comp-metric based on CompositeEvalMetric (#14553)
• [Clojure] enhance draw bounding box (#14567)
• [Clojure] Add methods based on NDArrayAPI/SymbolAPI (#14195)
• [Clojure] Clojure BERT QA example (#14691)
• [clojure-package][wip] add ->nd-vec function in ndarray.clj (#14308)
• [Clojure] Correct the versions in the README so they correspond to the latest maven.org release (#13507)
• Update version to v1.5.0 including clojure package (#13566)
• [clojure][generator] ndarray/symbol api random merged (#14800)
• upgrade codox to work with lein 2.9.0 (#14133)
• [clojure] fix: image test does not rely on s3 to run (#15122)

Julia

• Julia v0.7/1.0 support and drop v0.6 support (#12845)
• Julia: split ndarray.jl into several snippets (#14001)
• Julia: split symbolic-node.jl into several snippets (#14024)
• Julia: rename mx.clip to clamp for NDArray (#14027)
• Julia: add binding for runtime feature detection (#13992)

Perl:

• Two more gluon loss classes. (#14194)

R

• add NAG optimizer to r api (#14023)
• R-Package Makefile (#14068)

Performance Improvements

• Less cudaGet/SetDevice calls in Gluon execution (#13764)
• Improve bulking in Gluon (#13890)
• Increase perfomance of BulkAppend and BulkFlush (#14067)
• Performance improvement in ToTensor GPU Kernel (#14099)
• Performance improvement in Normalize GPU Kernel (#14139)
• Bulked op segments to allow Variable nodes (#14200)
• Performance improving for MKL-DNN Quantized FullyConnected (#14528)
• speedup SequenceMask on GPU (#14445)
• Dual stream cudnn Convolution backward() with MXNET_GPU_WORKER_NSTREAMS=2. (#14006)
• Speedup _contrib_index_copy (#14359)
• use mkl sparse matrix to improve performance (#14492)
• Re-enable static cached_op optimization (#14931)
• Speed up SequenceReverse (#14627)
• Improve FC perf when no_bias=False (#15033)
• Improve cached_op performance for static mode (#14785)

Example and Tutorials

• [MXNET-949] Module API to Gluon API tutorial (#12542)
• Support SSD f32/int8 evaluation on COCO dataset (#14646)
• [MXNET-1209] Tutorial transpose reshape (#13208)
• [Clojure] Add Fine Tuning Sentence Pair Classification BERT Example (#14769)
• example/ssd/evaluate/eval_metric.py (#14561)
• Add examples of running MXNet with Horovod (#14286)
• Added link to landing page for Java examples (#14481)
• Update lip reading example (#13647)
• [MXNET-1121] Example to demonstrate the inference workflow using RNN (#13680)
• [MXNET-1301] Remove the unnecessary WaitAll statements from inception_inference example (#13972)
• Modifying clojure CNN text classification example (#13865)
• [MXNET-1210 ] Gluon Audio - Example (#13325)
• add examples and fix the dependency problem (#13620)
• add quantization example to readme (#14186)
• Add an inference script providing both accuracy and benchmark result for original wide_n_deep example (#13895)
• Update autoencoder example (#12933)
• #13813 examples with opencv4/origami (#13813)
• [MXNET-1083] Add the example to demonstrate the inference workflow using C++ API (#13294)
• Add tutorial on how to use build from source jar (#14197)
• Gluon end to end tutorial (#13411)
• Update MXNetTutorialTemplate.ipynb (#13568)
• Simplifications and some fun stuff for the MNIST Gluon tutorial (#13094)
• Clarify dependency on OpenCV in CNN Visualization tutorial. (#13495)
• Update row_sparse tutorial (#13414)
• add clojure tutorials to index (#14814)
• Update lstm_crf.py (#14865)

Website

• Version switching user experience improvements (#13921)
• fix toctree Sphinx errors (#13489)
• fix link (#15036)
• fix website build (#14148)
• Fixed mailing list addresses (#13766)
• website publish updates (#14015)
• use relative links; update links (#13741)
• update social media section (#13705)
• [MXNET] Updated http://data.dmlc.ml/ links to http://data.mxnet.io/ (#15065)

How to build MXNet

Please follow the instructions at https://mxnet.incubator.apache.org/install/index.html

...