Tuning Collective Patterns to Alleviate Congestion in Shared AI Clusters
This paper builds a system, REACT, that tunes the recurring pattern of data exchange between GPU nodes (known as communication collectives) in response to congestion, and improves communication performance by 13%-38% under network congestion.