Aditya Ramamoorthy

Communication-Efficient Distributed Training via Ring-Based Coded Approximate All-Reduce

This work presents a communication-efficient Ring All-Reduce (CERAR) protocol for ``approximate''gradient aggregation, and presents an explicit Vandermonde matrix based construction whose approximation error can be made arbitrarily small, with communication and storage rates approaching one as $c$ increases.