HOCCL: Offloading Collective Communication from GPU Cores to Accelerate Distributed Training
HOCCCL, a zero-SM collective communication framework consisting of three components: a stream manager, a point-to-point (P2P) executor, and a collective scheduler, which improves end-to-end training throughput by up to 5%.