Seok-Bum Ko

LYRA: Low-Frequency Rank Adaptation via Factored DCT Coefficients for Parameter Efficient Fine Tuning of Transformers

As pretrained language models grow, full parameter updates become prohibitively expensive, motivating parameter-efficient fine-tuning (PEFT). LoRA decomposes weight updates into low-rank factors, but its per-module cost $r(m{+}n)$ imposes a structural floor at rank one. FourierFT breaks this floor via frequency-domain...