Page History

...

1. ~150 most commonly used operators will be tested on CPU(with and without MKL), GPU, FP32, FP64. See Appendix 1 for list of operators.
2. Operators will be tested with NDArray and Gluon interface only i.e., symbol interface is not used for testing owing to plans of deprecation.
3. Python interface is used - faster and get a check in place.along with MXNet profiler.
4. Time and Memory usage are Only timing is measured to start with.
5. Statistics - Mean of the metric.

...

1. Cover remaining operators left out from Phase 1.
2. Support memory performance measurements.
3. Integrate with MXNet Profiler to capture - time, memory metrics.
4. Add more statistics - p50, p90, p99, min, max.

Phase 3

1. Explore and have CPP performance tests for most commonly used operators. This will give the true measurements compared to using Python Interface.
2. Integrate with property based testing libraries like Hypothesis, to randomly generate test cases with different tensor shapes and inputs.

...