Preprint
Sep 2026
SlideDP: Scaling Host-Resident LLM Fine-Tuning Across Multiple GPUs
SlideDP is presented, a synchronous data-parallel runtime for shared-host multi-GPU systems that maintains one authoritative host state, decouples communication routes from state layout, and pipelines parameter delivery, gradient aggregation, and CPU updates across ranks and chunks.
Rui-Jia Yang, Shi-Yuan Lin, Yu-Long Ao et al.
· 0 citations