Fumer: Proactive Time-Shifting for Synchronized Periodic Traffic in Distributed Training
The growth of distributed training models, with parameters now reaching the billion-scale, has shifted the system bottleneck from computation to communication. While Remote Direct Memory Access (RDMA) is widely deployed to improve network performance by circumventing the kernel mechanism, the synchronization-computatio...