...
At the time of writing MXNet uses a bundled as a submodule version of OpenMP which is from 11/2017. It's pull pulled from a specific revision and built, then explicitly linking to it. The proposed by the compiler library is not removed. When built with MKLML the intel version is explicitly removed from linked libraries.
...
Same behaviour we see in the treatment group no matter which OpenMP is used.
Control group
Treatment group shows no difference other than that "GCC-swing". Normalizing the data gives us average scores with ~1% difference, which is close to standard error.
Treatment group
ResNet152
Now we can observe a beautiful saturation of the throughput. Optimal batch size is between 16 and 32.
...
But the overall differences are pretty close to standard error and don't even reach 2%.
faster-rcnn Benchmark
As we can see, GOMP delivers ~3-5% worse performance than OMP.
...