一个 Pytorch 训练实践 (分布式训练+半精度or混合精度训练)

Source

内容速览

1. 'train.py': single training process on one GPU only.
2. 'train_parallel.py': signle training process on multiple GPUs using Dataparallel (包括不同GPU之间的负载均衡).
3. 'train_distributed.py' (recommended): multiple training processes on multiple GPUs using Nvidia Apex & Distributed Training:

Read more »