Parameter-efficient fine-tuning (PEFT) enables large language models (LLMs) to adapt to specialized tasks, but often at the cost of degrading general capabilities acquired during pretraining. Existing approaches primarily mitigate this trade-off through data replay or regularization, relying on additional data or expli...
Zhi-Qian Pang, Zi-Hong Sun, Qi Xie et al.· 0 citations
Flash EQ-Linear is proposed, an exact acceleration algorithm that reduces the cost to $2(T-1)/T^2$ of the original dense formulation of the equivariant group size by combining the Fourier convolution theorem along the group dimension with the conjugate symmetry of the real DFT.
Experimental results demonstrate that proposed Bayesian domain weighting method could achieve stable and efficient domain weights learning, and identifies optimal mixtures while consuming substantially less data than search-based function-fitting methods, revitalizing optimization-based domain weighting for large-scale...
Xiang Yuan, Kai-Qing Lei, Zhenyu Jin et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.