On-policy distillation (OPD) trains a student on its own generated prefixes with token-level teacher feedback, but transmitting or storing the teacher's full-vocabulary distribution at every token is costly. Entropy-aware OPD (EOPD) adds forward supervision to reverse KL to help the student recover plausible tokens it...
Zi-Xiang Ni, Zhuo-Lun Hu, Ren-Jie Cao et al.· arXiv.org· 0 citations
OPSRD is introduced, which uses a fixed expert role as privileged teaching context for on-policy self-distillation without reference solutions for on-policy self-distillation without reference solutions.
Wei-Jie Ren, Yan-Wen Zhang, Hao Li et al.· arXiv.org· 0 citations
Multi-agent collaboration lets large language models (LLMs) improve question answering through deliberation and feedback. Yet shared discussion couples correction with exposure to the same mistakes, which can erode the diversity needed for voting. Self-consistency offers sampling diversity without feedback, while singl...
Wei-Jie Ren, Yan-Wen Zhang, Hao Li et al.· 0 citations
Across four base/RL-teacher pairs spanning different scales, architectures, and pre-training lineages, RIDE approaches or exceeds the RL-trained teacher on every pair and is the only method whose mean does so, and it consistently outperforms output-space extrapolation, which degrades the student whenever the teacher is...
Hao Li, Mei-Jia Chen, Wei-Jie Ren et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.