Г. Г. Миронов

SP-Allreduce: Symmetric Tree Schedules for Tori and Beyond

Allreduce is a fundamental communication primitive in large-scale distributed training, and torus interconnects remain widely deployed in high-performance systems. We introduce SP-allreduce, a new allreduce algorithm for $d$-dimensional tori that exploits the symmetry of the topology and routes only along shortest path...