RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks
A random-walk-based LoRA fine-tuning scheme that eliminates the need for global synchronization, substantially reduces communication and computation costs, and avoids aggregation errors is proposed and provides rigorous convergence guarantees for non-convex objectives under standard assumptions.
Xing-Ran Chen, Rohit Bhagat, Ghadir Ayache et al.
· 1 citation