OpDisPath: Optimal Communication Scheduling in GPU Clusters Through Disjoint Path Selection
The rapidly growing demand for distributed training has motivated providers to deploy large-scale GPU clusters. As a provider, we manage multiple clusters that hosts numerous distributed model training jobs that generate voluminous, highly dynamic concurrent flows. These flows may traverse the same link, causing severe...