Single-stage fine-tuning is pervasive, yet catastrophic forgetting persists when the source pre-training data is inaccessible. Data weighting affords a natural lever to modulate sample influence. Yet existing weighting methods lack a principled mechanism to explicitly arbitrate knowledge retention against new-task adap...
HiLoRe is introduced, which allocates graph-attributed recovery units among high-precision storage, low-precision compression, and deterministic recomputation using measured recovery utility and update-conditioned approximation risk, and combines high-precision storage and deterministic recomputation with low-precision...
Xin-Rui Chen, Mengyang Li, Ou Wu et al.· 0 citations
MemAgent, featuring a content-aware routing architecture and a training-data synthesis pipeline, is proposed, featuring a content-aware routing architecture and a training-data synthesis pipeline that improves average accuracy and outperforms every individual memory method across all three benchmarks.
Yong-Xian Wei, Yi-Lin Zhao, Run-Xi Cheng et al.· 0 citations
CompKV is introduced, the first compensation-aware sparse attention framework that divides tokens into blocks and explicitly optimizes selection for the downstream compensation mechanism, and shows that the residual left by block-level mean compensation is governed by both block attention mass and within-block logit va...
Zheng-Hong Huang, Rui-Zhe Yao, Dan-Yi Liu et al.· arXiv.org· 1 citation· ⚡1
Efficient large language model (LLM) instruction tuning requires selecting response supervision with supporting prompt context. Existing methods typically value both sides separately, risking selection-state mismatch between valuation and retained training subsets. BRIDGE (Budgeted Response-Prompt Interaction via Direc...
Can Wu, Xin-Rui Chen, Ou Wu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.