THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
When to switch between Single and Multiple tree
(a) Parameter sweep of MXNET_KVSTORE_GPUARRAY_BOUND (b) 1 Push-Pull before Wait (c) 150 Push-Pulls before Wait
Figure 7. VGG-16 performance as function of MXNET_KVSTORE_GPUARRAY_BOUND using batch size 4 per GPU. These figures show that beyond 1M-10M float32's, multi-tree begins to do better than a single tree.
Alternative Approaches considered
...