...
$ numactl --physcpubind=0-17 --membind=0 python …
Category | Model | Latency batchsize=1 (ms, small is better) | Throughput batchsize=128 (fps, |
---|
higher no mkldnn | release 1.3 + mkldnn | speedup | no mkldnn | w/o MKL-DNN | w/ MKL-DNN | speedup | w/o MKL-DNN | w/ MKL-DNN |
---|
release 1.3 + mkldnn | speedup |
---|
CNN/classification | ResNet-50 v1 | 97.19 |
1894513132051284189352112712792634665110001916669649459281947533120912179143073252738054153040081014692396433149068986259632 |
inception-resnet v2 | 181.10 |
111281636939126614 |
CNN/object detection | Faster R-CNN | 1175.74 |
951512361051123612748566273590(batchsize=224) | 19.13 |
SSD-MobileNet |
23910075 239 4577318 1003068199763(batchsize=64) | 6.83 |
GAN | DCGAN | 8.94 | 0. |
2241364059743720Inference Accuracy
The model is from gluon model zoo by pre-trained parameters. The top1 and top5 accuracy are verified by MKL-DNN backend.
...