On-policy distillation (OPD) has become a standard component of frontier post-training pipelines, yet how much its training data actually contributes has gone largely unexamined. On the two teacher--student pairings most common in practice, we find OPD almost indifferent to its data: eight prompts already match a 17k-p...
Gengsheng Li, Mao Zheng, Ming-Yang Song et al.· 0 citations
Anchored by this tri-axial framework, representative methods are systematically surveyed, the ongoing transition of continual learning is traced, and the key challenges, broader implications, and future directions arising from this paradigm shift are discussed.
Zhi-Yan Hou, Dan Zhang, Tao Feng et al.· 0 citations
This work presents Language Model Security Modules (LMSM), a security framework that adapts the separation behind Linux Security Modules (LSM) to LLM serving and gives advances in interpretability and model-internal analysis a common path to runtime enforcement.
XiuYu Zhang, Bo-Nan Ruan, Jun-Feng Fang et al.· 0 citations
This work identifies memory-induced cognitive traps: even faithfully recorded and semantically relevant memories can distort model reasoning or beliefs and degrade current task performance, and proposes AdaptiveMem, a simple yet effective inference-time method that instructs LLMs to avoid memory traps.
Mengru Wang, Haozhe Luo, Zhen-Qiang Xu et al.· 0 citations
Mechanist is an agentic system that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI intelligence, and develops a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, infer the beliefs of others, and how these mechanisms emerge during pretr...
Mengru Wang, Junfeng Fang, Shuofei Qiao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.