Mayank Bakshi

SeFoRA: Sketch-Aggregated Federated Low-Rank Adaptation with Heterogeneous Client Ranks

This work proposes SeFoRA, a sketch-aggregated federated LoRA algorithm in which each client transmits a linear sketch of its local updates, enabling direct aggregation at the federator, and introduces a rank-homogeneous version called SeFoRA-Ho which allows for direct adapter aggregation in this setting.