Geometric Calibration of Supervised Fine-Tuning
Recent studies investigate the weight geometry through its spectral structure, which characterizes how the weights transform different input directions. Within this view, supervised fine-tuning (SFT) is shown to induce dense updates that perturb the spectral geometry of pre-SFT weights along their principal directions,...
Jian-Zhu Bao, Jia-Chen Gu, Qing-Yuan Liu et al.
· 0 citations