Elastic Deep Learning with Horovod on Ray
Introduction
In 2017, we introduced Horovod, an open source framework for scaling deep learning training across hundreds of GPUs in parallel. At the time, most...
Horovod v0.21: Optimizing Network Utilization with Local Gradient Aggregation and Grouped Allreduce
We originally open-sourced Horovod in 2017, and since then it has grown to become the standard solution in industry for scaling deep learning training...
