Zeroth-order optimization (ZO) has demonstrated remarkable promise in efficient fine-tuning tasks for Large Language Models (LLMs). In particular, recent advances incorporate the low-rankness of gradients, introducing low-rank ZO estimators to further reduce GPU memory consumption. However, most existing works focus so...
Y. Sun, Qi-Xin Zhang, Liang Ding et al.· IEEE Transactions on Pattern...· 11 citations· ⚡1
OmniPack is proposed, a training-free framework that coordinates structural compression before the LLM with task-relevant semantic refinement within the LLM, and consistently achieves the best performance-efficiency trade-off across diverse retention ratios, outperforming all existing methods.
Wan-Shun Su, Yang Shi, Fei Liu et al.· 2 citations
This work introduces a failure-driven adaptation recipe spanning continued pre-training (CPT) and supervised fine-tuning (SFT) and formalizes an evidence-gated constraint-evaluation interface enforcing three-valued logic where unverified product evidence remains unknown rather than satisfied.
Liang Ding, Zhiang Xu, Yu-Yang Sheng et al.· 0 citations
LT-MKT first integrates textual information from questions and their associated concepts to construct a Multi-domain Hierarchical Graph, leveraging the advanced representational capabilities of large language models (LLMs) to bridge isolated domains and proposes a novel method incorporating cognitive Load and knowledge...
Haotian Zhang, Shucun Wang, Jinze Wu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
