THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
- making sure to pass the DistributedOptimizer object into Trainer instead of into Module
- using hvd.broadcast_params parameters on the exposed initialized parameters
...