Large Reasoning Models (LRMs) produce explicit chains of thought to improve transparency, but they frequently overthink, generating long yet inefficient reasoning chains that inflate computational costs and risk introducing errors. While numerous reinforcement learning methods struggle to penalize verbosity to achieve...
Cong Jiang, Xiao-Feng Zhang, Tomas Ko et al.· 0 citations
Integrated Policy Gradient is introduced, a novel, training-free framework that brings policy-based attribution into mechanistic interpretability for LLM reasoning and enables modulation of reasoning capabilities, offering a powerful and reliable framework for understanding LLM reasoning.
Chang-Ming Li, Kai-Xin Zhang, Haoyun Xu et al.· 4 citations
Large language models (LLMs) rely on long-horizon tool invocation sequences for complex tasks, where each invocation can alter the task state and condition subsequent decisions. In long-horizon tool use, final-outcome rewards provide weak credit assignment over long interaction traces. Step-level rewards can offer more...
Textual skills enable large language model (LLM) based agents to accumulate reusable procedural knowledge without updating model parameters. Yet existing skill evolution remains largely confined to the text space: an optimizer must diagnose success and failure patterns, and revise skills solely from long execution traj...
Kai-Xin Zhang, Chang-Ming Li, Ying-Dong Shi et al.· arXiv.org· 0 citations
LLM-as-a-Judge is increasingly used to evaluate policy responses on open-ended tasks that lack ground-truth answers. Existing work often directly converts the resulting judgments into reward signals for policy training, paying limited attention to intrinsic judgment quality and largely restricting the use of Judges to...
Zheng Zhang, Lu Li, Xin-Yue Tan et al.· 0 citations
This paper introduces Bootstrapped On-Policy Self-Distillation (B-OPSD), which temporarily trains the policy ahead to obtain a future teacher, restores the student to the original policy state, and then uses the future teacher to supervise the restarted student.
Zheng Zhang, Xin-Yue Tan, Lu Li et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.