Horovod v0.21: Optimizing Network Utilization with Local Gradient Aggregation and Grouped Allreduce
We originally open-sourced Horovod in 2017, and since then it has grown to become the standard solution in industry for scaling deep learning training...
