Page History

...

Both methods are shown in Figure 34.

                          (a) Single tree algorithm                                             (b) Multiple tree algorithm

Figure 34. Proposed Reduce and Broadcast algorithms.

...

                                   (c) Reduce                                                                                                                         (d) Broadcast
Figure 45. Block diagram of proposed addition. Changes to old initialization (InitMergeBuffersAndComm), Reduce and Broadcast are illustrated.

...

When to switch between Single and Multiple tree

Figure 56. VGG-16 performance as function of MXNET_KVSTORE_BIGARRAY_BOUND using batch size 4 per GPU.

...

Figure 67. End-to-end training results on synthetic data showing speed-up vs. NCCL on fp32 and fp16.

...