THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
The ring Reduce communication pattern used by NCCL (Figure 1a) and Parameter server Reduce (Figure 1b) currently used in MXNet are not optimal for small batch sizes on p3.16xlarge instances with 8 GPUs. Note: To clarify, usage of "parameter server" refers to a single-machine communication rather than the distributed sense the term is usually used in. This may cause some confusion.
View file | ||||
---|---|---|---|---|
|
View file | ||||
---|---|---|---|---|
|
...