THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
(Draft)
New Features
TODO: Summarize and highlight all new features
New Operators
- split_v2 (#13687)
- Gradient multiplier (contrib) operator (#13632)
- Image normalize operator - GPU support, 3D/4D inputs (#13802)
- Image ToTensor operator - GPU support, 3D/4D inputs (#13837)
- Add Gluon Transformer Crop (#14259)
- GELU (#14449)
- AdamW operator (Fixing Weight Decay Regularization in Adam) (#13728)
Feature Improvements
Operators
- make ROIAlign support position-sensitive pooling (#13088)
- Add erfinv operator for calculating inverse error function (#13811)
- Added optional parameters to BilinearResize2D to do relative scaling (#13985)
- MXNET-1295 Adding integer index support to Sequence* family of operators. (#13880)
- Export resize and support batch size (#14014)
- CUDNN dropout (#13896)
- Relaxing type requirements for slice_like op (#14097)
- Relaxing type requirements for reshape_like op (#14325)
- Parallelize CPU version and add GPU version of boolean_mask op (#14090)
- Add NHWC layout support to Pooling (cpu, gpu cuda, gpu cuDNN) (#13749)
- Multi-precision AdamW update op (#14171)
- [op] add back support for scalar type rescale_grad argument for adamw_update/mp_adamw_update (#14221)
- move choose_element_0index to operator (#14273)
- Optimize NMS (#14290)
- Optimize NMS part 2 (#14352)
- add backgroud class in box_nms (#14058)
- Use cudnn for dropout by default (#14278)
- In-place updates for Nadam, Adadelta, Adamax and SGLD (#13960)
- Aggregate SGD (#13346)
- Add proper exception message for negative shape in array creation routines (#14362)
- Support multi-threading for Custom Operator (#14363)
- moveaxis operator now accepts negative indices and sequence of ints as well. (#14321)
- Support SyncBatchNorm5D (#14542)
- Add nd.power and sym.pow (#14606)
- Change RNN OP to stateful (#14476)
- Add imresize and copyMakeBorder to mx.image (#13357)
MKLDNN
- Feature/mkldnn static (#13628)
- Feature/mkldnn static 2 (#13503)
- support mkl log when dtype is fp32 or fp64 (#13150)
- Add reshape op supported by MKL-DNN (#12980)
- Move the debug output message into MXNET_MKLDNN_DEBUG (#13662)
- Integrate MKLDNN Conv1d and support 3d layout (#13530)
- Making MKL-DNN default on MXNet master (#13681)
- Add mkldnn OP for slice (#13730)
- mkldnn s8 conv API change for master (#13903)
- [MKLDNN] Enable signed int8 support for convolution. (#13697)
- add mkldnn softmax_output (#13699)
- MKLDNN based Quantized FullyConnected Operator and its fusion (#14128)
- Fix entropy for uint8 (#14150)
- Update MKL-DNN to v0.18 release (was: fix the Dense layer issue) (#13668)
- [MKL-DNN] Enable s8 support for inner product and 3d input with flatten=false (#14466)
- Optimize transpose operator with MKL-DNN (#14545)
ONNX
- ONNX export: Instance normalization, Shape (#12920)
- ONNX export: Logical operators (#12852)
- ONNX import/export: Size (#13112)
- ONNX export: Add Flatten before Gemm (#13356)
- ONNX import/export: Add missing tests, ONNX export: LogSoftMax (#13654)
- ONNX import: Hardmax (#13717)
- [MXNET-898] ONNX import/export: Sample_multinomial, ONNX export: GlobalLpPool, LpPool (#13500)
- ONNX ops: norm exported and lpnormalization imported (#13806)
- [MXNET-880] ONNX export: Random uniform, Random normal, MaxRoiPool (#13676)
- ONNX export: Add Crop, Deconvolution and fix the default stride of Pooling to 1 (#12399)
- onnx export ops (#13821)
- ONNX export: broadcast_to, tile ops (#13981)
- ONNX export: Support equal length splits (#14121)
TensorRT
- [MXNET-1252][1 of 2] Decouple NNVM to ONNX from NNVM to TenosrRT conversion (#13659)
- [MXNET-703] Update to TensorRT 5, ONNX IR 3. Fix inference bugs. (#13310)
- [MXNET-703] Minor refactor of TensorRT code (#13311)
FP16 Support
- Update mshadow to support batch_dot with fp16. (#13716)
- float32 → float16 cast consistency across implementations (#13857)
- modifying SyncBN doc for FP16 use case (#14041)
- support dot(vector, vector) for fp16 inputs on GPU (#14102)
- softmax for fp16 with fp32 accumulator (#14098)
- [MXNET-1327] Allow RNN Layers to be initialized to fp16 (#14219)
- fp16 safe norm operator (#14616)
DGL support
Horovod Integration
- Add extra header file to export for error checking (#13795)
- whitelist symbols for using MXNet error handling externally (#13812)
- Use CPUPinned context in ImageRecordIOParser2 (#13980)
- Add pin_device_id option to Gluon DataLoader (#14136)
Dynamic Shape
- [MXNET-1315] Add checks for dynamic-shaped operators in CachedOp (#14018)
- [MXNET-1325] Make InferShapeAttr a standalone pass (#14193)
- [MXNET-1324] Add NaiveRunGraph to imperative utils (#14192)
Backend Engine
- Add infer_type_partial (#14214)
- Tidy up storage allocation and deallocation (#14480)
- Add MXEnginePushAsync and MXEnginePushSync C APIs (#14615)
- Enhance subgraph API (#14113)
- Enhance PartitionGraph (#14277)
Numpy
- [MXNET-1330] Bring nnvm::Tuple to mxnet::Tuple (#14270)
- [numpy] Support zero-dim and zero-size tensors in MXNet (#14661)
Large Tensor Support
- Large array support for randint (#14242)
- [MXNET-1185] Support large array in several operators (part 1) (#13418)
Quantization
- Exclude concat layer for gpu quantization (#14060)
- Enhance gpu quantization (#14094)
- Register fake grad to subgraph and quantized operators (#14275)
- Add int8 data loader (#14123)
CoreML
- Add more support for mxnet_to_coreml (#14222)
Website
- Version switching user experience improvements (#13921)
Front End API
Gluon
- Add pixelshuffle layers (#13571)
- [MXNET-766] add dynamic_unroll RNN for HybridBlock (#11948)
- add pos_weight for SigmoidBinaryCrossEntropyLoss (#13612)
- Rewrite dataloader with process pool, improves responsiveness and reliability (#13447)
- Complimentary gluon DataLoader improvements (#13606)
Python
- Python BucketingModule bind() with grad_req = 'add' (#13984)
- Refine runtime feature discovery python API and add documentation to … (#14130)
- Runtime feature detection (#13549)
- Add dtype visualization to plot_network (#14066)
- [MXNET-1359] Adds a multiclass-MCC metric derived from Pearson (#14461)
- support long for mx.random.seed (#14314)
- Optimization of metric evaluation (#13471)
Language Bindings
Scala
- [MXNET-1260] Float64 DType computation support in Scala/Java (#13678)
- [MXNET-1000] get Ndarray real value and form it from a NDArray (#12690)
- Now passing DType of Label downstream to Label's DataDesc object (#14038)
- Scala interpreter instructions (#14169)
- Add default parameters for Scala NDArray.arange (#13816)
- [MXNET-1287] Up scala comp (#14667)
Java
- [MXNET-1180] Java Image API (#13807)
- [MXNET-1285] Draw bounding box with Scala/Java Image API (#14474)
- Add BERT QA Scala/Java example (#14592)
C++
- print error message for mxnet::cpp::Operator::Invoke when failed (#14318)
Clojure
- [Clojure] - Add Spec Validations to the Optimizer namespace (#13499)
- [Clojure] Add Spec Validations for the Random namespace (#13523)
- [Clojure] Correct the versions in the README so they correspond to the latest maven.org release ([#13507)
- Port of scala infer package to clojure (#13595)
- Clojure example for fixed label-width captcha recognition (#13769)
- Update project.clj file to use the snapshots repo to be able to pull (#13935)
- [Clojure] Add resource scope to clojure package (#13993)
- [clojure-package] improve docstrings in image.clj (#14307)
- [Clojure] Helper function for n-dim vector to ndarray (#14305)
- [clojure]: add comp-metric based on CompositeEvalMetric (#14553)
- [Clojure] enhance draw bounding box (#14567)
- [Clojure] Add methods based on NDArrayAPI/SymbolAPI (#14195)
- [Clojure] Clojure BERT QA example (#14691)
- [clojure-package][wip] add ->nd-vec function in ndarray.clj (#14308)
Julia
- Julia v0.7/1.0 support and drop v0.6 support (#12845)
- Julia: split ndarray.jl into several snippets (#14001)
- Julia: split symbolic-node.jl into several snippets (#14024)
- Julia: rename mx.clip to clamp for NDArray (#14027)
Perl:
- Two more gluon loss classes. (#14194)
R
Performance Improvements
- Less cudaGet/SetDevice calls in Gluon execution (#13764)
- Improve bulking in Gluon (#13890)
- Increase perfomance of BulkAppend and BulkFlush (#14067)
- Performance improvement in ToTensor GPU Kernel (#14099)
- Performance improvement in Normalize GPU Kernel (#14139)
- Bulked op segments to allow Variable nodes (#14200)
- Performance improving for MKL-DNN Quantized FullyConnected (#14528)
- speedup SequenceMask on GPU (#14445)
- Dual stream cudnn Convolution backward() with MXNET_GPU_WORKER_NSTREAMS=2. (#14006)
- Speedup _contrib_index_copy (#14359)
- use mkl sparse matrix to improve performance (#14492)
Build and Test
- compatibility with opencv4 (#14313)
- Flaky test #14189 (#14190)
- Enforce determinism for backwards compatibility checker (#14463)
- Change CUB submodule to track Nvidia CUB project. (#13322)
- Updates gpu tests to use CUDNN_VERSION supplied by the environment but default to 7.0.3 if not set (#14595)
- [Dependency Update] Upgrade the libtiff to 4.0.10 (#14623)
Example and Tutorials
- [MXNET-949] Module API to Gluon API tutorial (#12542)
- Support SSD f32/int8 evaluation on COCO dataset (#14646)
Bug-fixes
- Fallback to dense version for grad(reshape), grad(expand_dims) (#13599)
- Fix the bug of BidirectionalCell (#13575)
- set _scale in Trainer using optimizer rescale_grad (#14593)
- [MXNET-1379] update reshape operator (#14600)
- Add repr for SymbolBlock (#14423)
- Cudnn conv dgrad algo filtering (#14310)
- Fix memory leak for size-zero ndarray (#14365)
- Fixes the test_sgld (#14473)
- Revert "Fix memory leak for size-zero ndarray (#14365)" (#14477)
- fix custom operation in fork (#14451)
- Fixes test_operator_gpu.test_multinomial_generator (#14475)
- support leading dimension of -1 in ravel/unravel (#14356)
- begin=end not a valid input (#14403)
- Fix NaN value comparisons in relu, max and min ops (#14262)
- fix engine crash in shutdown phase (#14382)
- fix OOM error during resource allocation (#14444)
- Fix relative difference scala (#14417)
- Correct update count with Gluon trainer and update_on_kvstore=False (#14377)
- Fix crashes on visualization (#14425)
- Reorder module import orders for dist-kvstore (#13742)
- Fixes for trainer with update_on_kvstore=False (#13721)
- Fix errors in docstrings for subgraph op; use code directive (#13463)
- Add resiliency to onnx export code (#13426)
- update github location for sampled_block.py (#13508)
- Revert "Manually track num_max_thread (#12380)" (#13501)
- Revert "Feature/mkldnn static 2 (#13503)" (#13540)
- [MXNET-1110] Add header files required by horovod (#13062)
- [MXAPPS-1020] Clean up some Sphinx warnings. (#13539)
- Fixing a 404 in the ubuntu setup doc (#13542)
- [MXNET-1249] Fix Object Detector Performance with GPU (#13522)
- [MXNET-769] Use MXNET_HOME in a tempdir in windows to prevent access denied due t… (#13531)
- Chi_square_check for discrete distribution fix (#13543)
- Fix use-before-assignment in convert_dot (#13511)
- fix the situation where idx didn't align with rec (#13550)
- fix link for gluon model zoo (#13583)
- Fix exception handling api doc (#13519)
- [MXNET-1253] fix control_flow_op (#13555)
- fix the Float not showing correctly problem (#13617)
- fix quantize pass error when the quantization supported Op are excluded in the model (#13596)
- Fix for import mxnet taking long time if multiple process launched (#13602)
- Revert "Feature/mkldnn static (#13628)" (#13638)
- updated reference to Apache MXNet (#13645)
- Fix incorrect delete in MXExecutorReshape exception handling (#13376)
- add build fix for Scala/Java build (#13655)
- remove omp which can cause ssd accuracy variance (#13622)
- Fix Jetson compilation (#13532)
- Revert "Fix Jetson compilation" (#13665)
- Fix Jetson compilation (#13666)
- Revert "Revert "[MXNET-43] Fix Jetson compilation" (#13665)" (#13672)
- fix unpicklable transform_first on windows (#13686)
- Fix NDArray ToDLPack Bug (#13698)
- Fix the quantization script to support Python2 (#13700)
- Update basic_layers.py (#13732)
- [MXNET-1231] Allow not using Some in the Scala operators (#13619)
- [MXNET-244] Work around likely compiler bug on nested inlines and temporary acces… (#13535)
- Use curl to download sample data instead of wget. (#13761)
- fix bipartite match memory corruption (#13727)
- remove attributs clear on TRT nodes for GetOptimizedSymbol (#13703)
- fix redirection issues; set default version to master (#13796)
- fix for params with no dims in onnx (#13413)
- Remove semicolon in libmxnet.sym file (#13822)
- remove useless code (#13777)
- Fixing a symlink issue with R install (#13708)
- fix minor indentation (#13827)
- Fix Tree Reduction on new instance type p3dn.24xlarge (#13852)
- [Clojure] package infer tweaks (#13864)
- Fix cpp examples build on Mac. (#13826)
- Fix launch bounds in spatial transformer (#13188)
- Update example scripts classpath. (#13849)
- fix ssd quantization script error (#13843)
- Avoid adding SegfaultLogger if process already has sig handler. (#13842)
- fix the fetching GPU problem (#13889)
- Fix SN-GAN example doc (#13877)
- update Spectral Normalization Code (#13868)
- Fixed java benchmark failing error by fixing the classpath (#13891)
- Fix the order of error term's operands (#13745)
- fix bug in nag optimizer (#13683)
- Fix BatchNorm converter for CoreML when fix_gamma=True (#13557)
- Fix for test always returning true (#13911)
- Add error checking for cpp examples. (#13828)
- julia: fix
argmax
for NDArray (#13871) - test_ImageRecordIter_seed_augmentation flaky test fix (#12485)
- Julia: fix filename quoting in docstring (#13894)
- Flaky maven binary download (#13974)
- [MXNET-1293] Adding Iterables instead of List to method signature for infer APIs in Java (#13977)
- Sample python bilinear initializer at integral points in y-direction (#12983)
- Fix inconsistent handling for FResourceRequestEx for imperative and symbolic executor (#14007)
- [MXNET-1258] fix unittest for ROIAlign Operator (#13609)
- Fix performance regression in normalize operator (#14055)
- Remove inplace support for ToTensor operator (#14083)
- Addresses comments in runtime feature discovery API (#13964)
- The latest version of leiningen has a dependency problem with codox (#14132)
- Fix quote on LBSGD docs (#13975)
- Fixes spelling (#14168)
- Fix broken amalgamation (#12792)
- Fix nd.pick large array issue (#14082)
- Fix req=null in SliceLikeBackward (#14209)
- onnx broadcast ops fixes (#13604)
- fix update params (#14218)
- MXNet Java bug fixes and experience improvement (#14213)
- reverting broadcasting fixes (#14299)
- fix memory-related issues to enable ASAN tests (#14223)
- FIX: flaky test exponential generator (#14287)
- fix SoftmaxOutput resource bug (#14302)
- Fix shape inference pass (#14153)
- Limit workspace for cudnnGet results (#14326)
- #14199: catch subprocess.CalledProcessError in get_gpus() (#14212)
- Fixes #14181, validate model output shape for ObjectDetector. (#14215)
- Optimizer MXKVStoreUpdater bug fix in serializeState method (#14337)
- Add proper exception message for negative shape in array creation routines (#14362)
- Fix NaN value comparisons in relu, max and min ops (#14262)
- fix engine crash in shutdown phase (#14382)
- Flaky test #14189 (#14190)
- Correct update count with Gluon trainer and update_on_kvstore=False (#14377)
- Fix relative difference scala (#14417)
- fix OOM error during resource allocation (#14444)
- Fix crashes on visualization (#14425)
- begin=end not a valid input (#14403)
- Fix memory leak for size-zero ndarray (#14365)
- Fixes the test_sgld (#14473)
- Revert "Fix memory leak for size-zero ndarray (#14365)" (#14477)
- fix custom operation in fork (#14451)
- Fixes test_operator_gpu.test_multinomial_generator (#14475)
- Fix script retrieval (#14519)
- Memory fixes. Resolves #10867, and resolves #14080 (#14372)
- Chouffe/clojure fix tests (#14531)
- [clojure][image] add draw-bounding-box interop (#14533)
- fix tests (#14565)
- Do not touch GPU 0 during ReleaseAll (#14550)
- [MXNET-1357] Fix the cpp-examples to add exception handling (#14441)
- fix build cpp examples option (#14562)
- Fix flaky test poisson generator & test_negative_binomial_generator (#14571)
- Fixing unintentional variable overloading (#14438)
- fix quantize graph pass (#14605)
- replace std::random_shuffle to std::shuffle (#14523)
- Add exception handling support for waitall (#14397)
- split_and_load can now handle num_ctx > num_data. Issue #13909 (#14607)
- Fix aspect ratio sampling for RandomResizedCrop (#14585)
- [MXNET-400] support string type for kvstore key in cpp-package (#10792)
- Fix warning on macro expansion using defined. (#14598)
- Fix scaladoc scalastyle violations in Infer package (#14671)
- Fix profiler check (#14677)
- Tweak the copy for the cudnn autotuning warning. (#14680)
- Properly handling custom op exception by modify engine (#14693)
- Disable USE_GPERFTOOLS (#14711)
- Reference engine from chunk via weak pointer (#14591)
API Changes
TODO
Depreciations
- Julia: deprecate
mx.empty
, replace it withUndefInitializer
(#13934)
Known Issues
How to build MXNet
Please follow the instructions at https://mxnet.incubator.apache.org/install/index.html
List of submodules used by Apache MXNet (Incubating) and when they were updated last
name | commit-id | last updated in MXNet | last update in module |
---|---|---|---|
dlpack | 10892ac | Oct 30, 2017 | Mar 28, 2019 |
dmlc-core | 3ffea86 | Mar 24, 2019 | Apr 12, 2019 |
googletest | ec44c6c | Jul 14, 2016 | Apr 16, 2019 |
mkldnn | 7de7e5d | Mar 15, 2019 | Apr 1, 2019 |
mshadow | 95ebe0f | Feb 28, 2019 | Apr 11, 2019 |
nvidia_cub | c3cceac | Feb 16, 2018 | Jan 31, 2019 |
onnx-tensorrt | f1c7aa6 | Nov 10, 2018 | Apr 10, 2019 |
openmp | 37c7212 | Nov 14, 2017 | Apr 17, 2019 |
ps-lite | 8a76389 | Apr 25, 2018 | Apr 7, 2019 |
tvm | 0f053c8 | Oct 8, 2018 | Apr 17, 2019 |