Table of Contents

Deprecation of Python 2

MXNet community voted to no longer support Python 2 in future releases of MXNet. Therefore, MXNet 1.6 release is going to be the last MXNet release to support Python 2.

New features

NumPy compatible interface and using TVM to generate operators

...

• [OP] Add a new arange_like operator to contrib (#15400)
• PDF operators for each distribution for which we have a random sampler (plus also the PDF of the Dirichlet). Supports probabilities and log-probabilities, as well as gradients. (#14617)
• Group Normalization (#14959)
• Add RROIAlign (#16017)
• Add fast implementation of LARS (#16122)
• Round and sign straight-through-estimators C operators. (#16373)
• New ops for RCNN + old ops improvements for RCNN (#16215)
• Comparison ops implemented using mshadow (#16414)
• Add mask target generator operator for Mask-RCNN (#16268)
• Move MRCNNMaskTarget op to contrib (#16486)
• Mxnet allclose (#14443)
• Aggregated adamw update (#16398)
• Make mrcnn_mask_target arg mask_size a 2d tuple (#16567)
• Dgl ops 2 (#16416)
• Lamb optimizer update (#16715)
• [OP] changing data type of 't' to int in lamb_update_phase1 (#16903)
• Multi Precision Lamb Update operator (#16885)

Feature improvements

Automatic Mixed Precision

• Interleaved MHA for CPU path (#17138) (#17211)

MKL-DNN v1.0 update

MKL-DNN v1.0 release improved its robustness and simplified the execution model compared to v0.x, while keeps improving its performance with Intel(R) DL Boost and the upcoming BFloat16 supports, and enabled more kernels for those popular and frontier models. Lots of PRs are completed to adopt the minor API changes as below. Please refer to https://github.com/apache/incubator-mxnet/projects/16 for more details.

• [mkldnn-v1.0] Initiate the transition to MKL-DNN v1.0 (#15706)
• [mkldnn-v1.0] Update MKL-DNN to v1.0.2 (#16012)
• [mkldnn-v1.0] Enable base code with new APIs. (#16064)
• [mkldnn-v1.0] Add MKL-DNN Convolution (#16141)
• [mkldnn-v1.0] Add MKL-DNN activation (#16195)
• [mkldnn-v1.0] Add MKL-DNN BN (#16199)
• [mkldnn-v1.0] Add MKL-DNN Transpose (#16250)
• [mkldnn-v1.0] Add MKL-DNN softmax (#16246)
• [mkldnn-v1.0] Add MKL-DNN FC (#16221)
• [mkldnn-v1.0] Add MKL-DNN deconv (#16259)
• [mkldnn-v1.0] Add MKL-DNN Pooling (#16272)
• [mkldnn-v1.0] Add MKL-DNN reshape&flatten&expand_dims (#16258)
• [mkldnn-v1.0] Add MKL-DNN int8 activation&pooling&flatten (#16425)
• [mkldnn-1.0] int8 conv quantize dequantize requantize (#16283)
• [mkldnn-1.0] mkldnn int8 elemwise_add (#16454)
• [mkldnn-v1.0] Fused RNN Op (#16420)
• [mkldnn-v1.0] Add MKL-DNN int8 fc (#16457)
• [mkldnn-v1.0] Update enabling flag for MKL dropout (#16433)
• [mkldnn-1.0] upgrade int8 concat to MKLDNN1.0 (#16466)
• [mkldnn-v1.0] Add MKL-DNN slice (#16484)
• [mkldnn-1.0] add mkldnn subgraph fc (#16468)
• [mkldnn-v1.0]enable mkldnn concat (#16507)
• [mkldnn-v1.0] Enable mkldnn cpp-test, copy op, concat op (#16503)
• [mkldnn-v1.0] Enable mkldnn test, copy op, concat op
• [mkldnn-1.0] add skipped case for mkldnn_v1.0 (#16470)
• [mkldnn-1.0]enable mkldnn elemwise_sum (#16521)
• [mkldnn-v1.0] Enable more checks for MXNET_USE_MKLDNN (#16520)
• [mkldnn-v1.0]Minor fix for leakyrelu compile flag (#16519)
• [mkldnn-v1.0] Skip flaky test for unidirectional rnn_relu (#16545)
• [mkldnn-v1.0]set fc weight layout as mkldnn v0.2x did (#16593)
• [mkldnn-v1.0] Upgrade to MKL-DNN v1.0.4 patch release (#16592)
• [mkldnn-1.0]Rebase to master (#16648)
• [mkldnn-v1.0]rebase with master (#16649)
• Revert "[mkldnn-1.0]Rebase to master (#16648)"
• [mkldnn-v1.0] Minor fix of mkldnn-v1.0 transition (#16644)
• [mkldnn-v1.0]rm int8 sum workaround (#16623)
• [mkldnn-v1.0] Must reorder and emplace weights for inference primitives (#16682)

Feature improvements

Automatic Mixed Precision

• [AMP] Move topk from FP16_FP32_FUNCS to FP32_FUNCS (#15342)
• Conversion from FP32 model to Mixed Precision model (#15118)
• Update fp16 docs
• [AMP] Move topk from FP16_FP32_FUNCS to FP32_FUNCS (#15342)
• Conversion from FP32 model to Mixed Precision model (#15118)
• Update fp16 docs: Block.cast is inplace (#15458)
• FP16 Support for C Predict API (#15245)
• Add AMP Conversion support for BucketingModule (#15528)

...

• Fixing build for gluon estimator test, including libtvm in pack libs (#16148)
• [Estimator] handle composite metrics in estimator (#16676)
• [Estimator] refactor estimator to allow overriding evaluate/fit of a batch (#16678)
• [Estimator] refactor estimator and clarify docs (#16694)
• [Gluon] Improve estimator usability and fix logging logic (#16810) (#16846)
• Backport Gluon estimator changes to 1.6 (#17048)
• fix parameter names in the estimator api (#17051) (#17162)

MKLDNN

• Upgrade MKL-DNN submodule to v0.20 release (#15422)
• Fix quantized concat when inputs are mixed int8 and uint8 (#15693)
• [MKLDNN]Enhance Quantization APIs and Tutorial (#15448)
• Add quantization support for GluonCV (#15754)
• add int8 bn mkldnn implementation and test (#15664)
• [Quantization]support exclude operators while quantization (#15910)
• [MKLDNN]Support fullyconnected and element-wise ops fusion (#15950)
• Disable test coverage for Clang MKLDNN (#15977)
• update support MKLDNN BN conditions (#15870)
• [MKLDNN] Fix out of bound access of req vector (#16000)
• add uint8 bn mkldnn implementation (#16003)
• Improve quantization flow (#15961)
• [MKLDNN] fix uint8 batch norm memory misuse (#16034)
• MKL-DNN RNN checks NDArray version (#16071)
• Float64 fallback for mkldnn subgraph and rnn op (#15853)
• Update MKL-DNN dependency (#16073)
• Integrate MKL-DNN leakyrelu (#16075)
• [MKLDNN] NDArray reorder in C API and deconv (#16265)
• Fix mkldnn reshape (#16455)
• [MKLDNN] Fix uint quantized fc when not fusing with requantize (#16523)
• [MKLDNN]Fix reorder2default (#16602)
• Upgrade MKL-DNN dependency to v1.0 (#16555)
• Revert "[MKLDNN]Fix reorder2default (#16602)" (#16697)
• [v1.6.x] Backport #16837 into v1.6.x (#16847)
• Initial checkin (#16856) (#16872(#16872)
• [v1.6.x] Cherry-pick MKL-DNN Rnn operator enhancements to v1.6.x (#17225)

Large tensor support

• [MXNET-1413] Adding Large Tensor support for sort operators (#15170)
• Large Index Support for Slice (#15593)
• Add large tensor support binary arithmetic (#15785)
• Large tensor support for random ops (#15783)
• Add Large Tensor Support for Sequence, NN Ops (#15807)
• Add power, exponent, log ops large tensor support (#15794)
• removing unnecessary int64 C apis that were added to support Large Tensors and Vectors (#15944)
• creating ndarray directly using mxnet ndarray primitives to reduce memory footprint of tests for topk, sort and argsort (#15900)
• Adding tests to verify support for Large Tensors in additional Ops along with new C_Apis supporting 64bit indexing (#15895)
• Added tests to verify Large Vector Support for initial set of ops (#15943)
• Added more tests for Large Indices (#15960)
• Add Large tensor vector test cases (#15941)
• Test large vector mean operator and fix a few bugs (#16079)
• Reducing memory footprint of one_hot for Large Array Testing (#16136)
• removing MXNDArrayLoadFromBuffer64 and MXNDArrayLoad64 (#16203)
• Fix large array tests (#16328)
• added more tests to verify support for large vector (#16477)
• added support for large tensors for Dropout operator and tests to verify support for more operators (#16409)
• adding large tensor support for add_n and tests for more ops (#16476)
• adding large tensor support for pad operator (#15126)
• Added large tensor support and test for gather_nd (#16371)
• Large Vector tests for DGL Ops Part 2 (#16497)
• Showing proper error message when an attempt is made to create large tensor but MXNet is not built with it (#16570)

...

• [bug] fix higher grad log (#15120)
• Showing proper error when csr array is not 2D in shape. (#15242)
• add 'asnumpy' dtype option to check_symbolic_backward (#15186)
• point fix the vector declaration in MultiBoxDetection (#15300)
• Temporarily Commenting out Flaky Test (#15436)
• Fix memory leak in NaiveEngine (#15405)
• fix nightly CI failure (#15452)
• Small typo fixes in batch_norm-inl.h (#15527)
• Bypass cuda/cudnn checks if no driver. (#15551)
• Julia path patch (#15561)
• Fix AMP Tutorial failures (#15526)
• Fix warnings in CLang: (#15270)
• Fix dumps for Constant initializer (#15150)
• fix normalize mean error bug (#15539)
• [fix] print self in warning. (#15614)
• [MXNET-1411] solve pylint error issue#14851 (#15113)
• [Flaky test] Skip test_operator_gpu.test_convolution_independent_gradients (#15631)
• Fix subgraph with custom_op (#15671)
• Fix USE_BLAS == openblas check (#15691)
• update previous flaky naive engine test (#15651)
• make TransposeShape infer shape form both sides (#15713)
• Skip Flaky Test (#15722)
• Revert "Dynamic Library Loading Support" (#15755)
• Fix flaky test test_global_metric (#15756)
• Fix PR #15489 (Dynamic Library Loading Support) (#15760)
• Refactor LibraryInitializer so it's thread safe. Fixes random sporadical concurrency crashes. (#15762)
• Fix backward_clip num inputs and type of clip params (#15688)
• fixing problem with existing Singleton Caching (#15868)
• Allow operators with multiple outputs in get_atomic_symbol (#15740)
• Fix ConcatType backward type inference (#15829)
• Add disable attr to subgraph property (#15926)
• Re-enable flaky test_prelu (#15777)
• declare explicitly the tblob default assign operator and copy constructor (#15937)
• Discard needless test cases in test_convolution_independent_gradients (#15939)
• fix naive engine for multi-threaded inference (#15574)
• Fix get_rows_per_block (#15979)
• Fix a memory misalignment in topk operator (#15948)
• Decouple dtype from shape for Random multinomial (#15980)
• Fix dtype inference in arange_like operator (#15930)
• Disable laop_6 (#15976)
• Fix flaky clojure profile test (#16058)
• fix test_pick test time is too long (#16066)
• [fix] Support nullop in transpose (#15865)
• fix flaky test (#16074)
• fix some test files test time is too long (#16067)
• Fix gradient tensor mutate in {adam/ftrl/rmprop/rmspropalex}_update. (#15768)
• Fix unary operator ceil/floor/trunc when data type is integer (#14251)
• Fix failing tests (#16117)
• Fixes NAG optimizer #15543 (#16053)
• avoid test relu at the origin due to discontinuous gradient (#16133)
• Fix remaining errors reported by D2L (#16157)
• use 1E-4 in groupnorm test(#16169)
• Sequence last fix (#16156)
• fixing test for model compatibility checker (#16159)
• assert_allclose -> rtol=1e-10 (#16198)
• [MEMORY] retry GPU memory allocation if fragmented (#16194)
• improve dataloader signals and messages (#16114)
• Update ndarray.py (#16205)
• fix flaky test (#16191)
• Solve #14116, #15143 (#15144)
• [MXNET-1422] Fix wrong results of min([inf, inf]) and max([-inf,-inf]) (#16226)
• Fix inconsistent interpolation method values (#16212)
• set fixed seed for profiler (#16155)
• Fix MXNDArrayGetData (#16289)
• fix atol for test_preloaded_multi_sgd (#16356)
• Fix windows flakiness (#16415)
• cuDNN non-persistant bidirectional RNN dgrad sync fix (#16391)
• [BUGFIX] Minor type issues in Squeeze (#16448)
• Fix Nightly Tests for Binaries (#16451)
• Fix dtype bug (#16467)
• Fix flakey pylint CI failures (#16462)
• Load NDArray only to GPU if GPU is present (#16432)
• Bug fix for the input of same axes of the swapaxes operator (#16513)
• Fix learning rate scheduler being unexpectedly overwritten by optimizer's default value (#16487)
• disable tests (#16536)
• fix pylint in CI (#16540)
• image crop gpu (#16464)
• Build dmlc-core with old thread_local implementation (#16526)
• fix doc for topk (#16571)
• RNNOp to call cudaEventCreate lazily (#16584)
• add encoding to the stub files for potential utf8 char in doc strings (#16580)
• Surpress subgraph log in CI (#16607)
• Fix dequantize memory corruption (#16606)
• Fix for wrong reqs set after switching from training to inference (#16553)
• Disables test_bulking_operator_gpu due to flakiness (#16611)
• Imagenet inference to nightly fix (#16599)
• Move some subgraph verbose to MXNET_SUBGRAPH_VERBOSE=2 (#16622)
• RNNOp only call cuda/cudnn if GPU ctx is requested (#16632)
• fix bad encode (#16641)
• Disable float16 test (#16643)
• Fix GetMKLDNNData for delay alloc (#16618)
• Move ops which don't support FP16 dtype to FP32 list (#16668)
• no such method => modified function args (#16610)
• fix cuDNN RNN dtype_with_fallback_ bug (#16671)
• Add check if scipy is imported in sparse.py (#16574)
• Added launch bounds to the reduce kernels (#16397)
• fix install dir (#16690)
• fix binary dependencies in CD and nightly (#16693)
• Fix SliceChannel Type inference (#16748) (#16797)
• fix flakiness of test_np_mixed_precision_binary_funcs (#16873)
• Fix test_gluon.py:test_sync_batchnorm when number of GPUS > 4 (#16835)
• Omp fork numthreads fix 1.6 (#17000)
• [BUGFIX] Fix race condition in kvstore.pushpull (#17007) (#17052)
• Backport #17002, #17068 and #17114 to 1.6 branch (#17137)
• Backport 3rdparty/openmp fixes (#17193)
• fix norm sparse fallback (#17149)

Front end API

• Expose get_all_registered_operators and get_operator_arguments in the… (#15364)
• Add magic method abs to NDArray and Symbol. (#15680)
• Dynamic Library Loading Support (#15489)
• [MXNET-1294] Add KVSTORE PushPull API (#15559)

...

• Add symbol api for randn and fix shape issue for randn ndarray and symbol api (#15772)
• Graph Partition API (#15886)

Language Bindings

Python

MXNet community voted to no longer support Python 2 in future releases of MXNet. Therefore, MXNet 1.6 release is going to be the last MXNet release to support Python 2.

C/C++

• [C++] Improve inference script to support benchmark on Imagenet (#15164)
• C Api for simplebind, fix comment for trigoops, add atol to assert (#16585)

...

• Fix build_ccache_wrappers: (#14631)
• Remove mhard-float option. This is already deprecated by Google. (#15435)
• CI: upgrade Julia version from 1.0.3 to 1.0.4 (#15502)
• Add -R option to ci/build.py to avoid rebuilding containers (#15426)
• [Dependency Update] Bump up the CI Nvidia docker to CUDA 10.1 (#14986)
• fixed config.mk and Makefile bugs for installing mkl (#15424)
• Add -DMXNET_USE_OPENMP to Makefiles so libinfo gets updated accordingly (#15498)
• [Dependency Update] Dependency update doc (#15045)
• Remove Scala package test on build (#15915)
• Refactor for windows CI 'out of heap space' errors (#15922)
• Fix Nightly Maven GPU (#15989)
• Windows cmake flags cleanup (#16013)
• Disable flaky test in test_amp_conversion (#16031)
• Updates git_init Jenkins utility function to support checking out a particular commit id
• Adds artifact repository scripts
• Adds CD pipeline framework
• Adds static libmxnet release pipeline
• Updates CD pipeline
• Adds documentation
• Updates kvstore functions to use pushd and popd
• Throws exceptions instead o magic numbers
• Updates artifact repository cli to use --libtype instead of --static or --dynamic
• Clarifies ci_utils and cd_utils origin remark
• Adds clarifying note on why ubuntu 14.04 is being used for compilation
• Removes MXNET_SHA
• Removes set_release_job_name
• Adds license headers
• Updates artifact repository to expect licenses
• Moves ci/cd to cd directory
• Takes downstream job name from environment
• Updates order of parameters
• Updates job type parameter to dropdown
• Adds libmxnet feature extraction code comments
• Removes ccache setup from static build
• Disable test coverage of C++ codebase on CI (#15981)
• Update readme and project.clj comment (#16084)
• Enable tvm_op for ci (#15889)
• Not to search for coverage files when none exist (#16107)
• Fixes openblas installation for static build
• Update python dependencies (#16105)
• CD Fixes (#16127)
• Adds dynamic libmxnet to CD pipeline (#16163)
• Fix README Build Status (#16183)
• subscribe to build and CD changes (#16192)
• [CD] Add COMMIT_ID param to release job (#16202)
• Fix lack of dylib support in Makefile when use lapack (#15813)
• Removes git status update stop gap solution (#16285)
• add mkl installation temp fix (#16304)
• add 'Release' cmake flag (#16294)
• S3 upload artifacts (#16336)
• Fix nightly scala pipeline (#16362)
• remove redundant branch name (#16372)
• Skipping installing nightly test (#16418)
• Adds PyPI CD Pipeline (#16190)
• upgrade the pytest version (#16429)
• Revert "add mkl installation temp fix (#16304)" (#16369)
• increase docker cache timeout (#16430)
• Adds pip requirements file to nightly gpu ci image (#16472)
• [CD] Adds python docker pipeline (#16547)
• Move imagenet inference to nightly (#16577nightly (#16577)
• Backport #16980 #17031 #17018 #17019 to 1.6 branch (#17213)

Miscellaneous changes

• update committer info (#15289)
• Typo fix in plan_memory relase -> release. (#15299)
• indent changes (#15321)
• Had a few PRs merged. Hope to become an official contributor and potentially a commiter. (#15451)
• cuda/cuDNN lib version checking. Force cuDNN v7 usage. (#15449)
• Improve diagnose.py, adding build features info and binary library path. (#15499)
• update ratcheck for apache-rat 0.13 release (#15417)
• add myself to interested modules (#15590)
• 1.5.0 news (#15137)
• bump up version from 1.5.0 to 1.6.0 on master (#15072)
• Remove myself from CODEOWNERS (#15617)
• remove mshadow submodule
• import mshadow source tree
• cuDNN support cleanup (#15812)
• Remove requests_failed_to_import handling
• Update CODEOWNERS. (#15972)
• Improve diagnose.py to display environment variables (#15715)
• Update README.md (#16035)
• [Dev] update ps-lite dependency (#15936)
• Typedef cleanup (#15899)
• add KEY for Tao Lv (#16081)
• remove 'foo' and other print msg from test (#16088)
• Revert accidental change to CMakelists (#16040)
• Update env_var.md (#16145)
• Update dmlc-core (#16149)
• adding codeowners (#16165)
• Factorize CUDA_KERNEL_LOOP used in CUDA kernels (#16197)
• add code of conduct and conflict resolution (#16343)
• simple typo error in NEWS.md (#16344)
• update NEWS.md and README.md (#16385)
• split issue templates (#16558)
• Create SECURITY.md (#16573)

How to build MXNet

Please follow the instructions at https://mxnet.incubator.apache.org

...

/get_started

Users that build MXNet from source are recommended to build release 1.6.0 without jemalloc to avoid incompatibilities with llvm's openmp library (details in issue 17043 and PR 17324). This is done for cmake builds by setting USE_JEMALLOC "OFF" in ./CMakeLists.txt, or for make builds with "USE_JEMALLOC = 0" in make/config.mk.

List of submodules used by Apache MXNet (Incubating) and when they were updated last

Note

...

On July 18, 2019 mshadow project (which was previously a submodule of MXNet) was imported into MXNet codebase under 3rdparty/mshadow as a result of a vote in https://github.com/dmlc/mshadow/issues/373.