Sep 2026· Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing· Vol abs/2609.34418· 0 citations· 23 references
Computer Science
TL;DR
This work proposes OSPD, an on-policy self-distillation framework where the same model serves as both teacher and student under asymmetric information: the teacher receives a complete character profile while the student sees only a brief summary, and the student generates trajectories from its own policy.
Abstract
Maintaining persona consistency across multi-turn dialogues remains a core challenge for role-playing language models. Off-policy distillation from external teachers incurs distribution mismatch that compounds across dialogue turns, while reinforcement learning struggles with reward ambiguity inherent in subjective persona fidelity. We propose OSPD, an on-policy self-distillation framework where the same model serves as both teacher and student under asymmetric information: the teacher receives a complete character profile while the student sees only a brief summary, and the student generates trajectories from its own policy. We find that teacher confidence in role-playing dialogue exhibits a bimodal structure---sharply peaked at character-critical tokens yet diffuse at generic utterances---and introduce role-aware divergence switching to match this structure. A progressive trait masking curriculum further forces staged internalization of character knowledge along semantic dimensions. Experiments on CharacterBench, CharacterEval, and SocialBench show that OSPD substantially improves persona consistency over supervised fine-tuning and multi-turn RL baselines, without requiring any external teacher or reward model.
OPSRD is introduced, which uses a fixed expert role as privileged teaching context for on-policy self-distillation without reference solutions for on-policy self-distillation without reference solutions.
Wei-Jie Ren, Yan-Wen Zhang, Hao Li et al.· arXiv.org· 0 citations
Multimodal dialogue understanding requires integrating heterogeneous signals such as language, audio, and visual cues while capturing complex contextual dynamics in multi-party interactions. Although recent models achieve strong performance, they typically rely on unified representations and treat self-distillation as...
Ryo Ishii, Chihiro Takayama, Jiro Nagao et al.· Proceedings of the 28th Inte...· 0 citations
RoleBreak, an open benchmark for long-horizon role-playing robustness in spoken dialogue, is introduced and four key patterns are found, highlighting persistent gaps in both long-horizon robustness and vocal expressiveness in spoken role-playing systems.
Yu-Qi Wang, Feng-Yuan Liu, Hao-Chen Luo et al.· 0 citations
GAD-RL is introduced, which adaptively regulates teacher supervision during joint post-training according to the student's current task performance and local distributions, and weights forward KL by the student's probability of the teacher's Top-1 token, moderating local auxiliary updates when student support for that...
Bao-De Wang, Zu-Ming Huang, Ke Ren et al.· 0 citations
While reinforcement learning with verifiable rewards provides reliable outcome supervision for VideoLLMs, sequence-level rewards offer limited token-level guidance. On-policy self-distillation addresses this limitation by conditioning a self-teacher on privileged information to provide dense token-level supervision. Ho...
Wei-Jie Zhu, Han Fang, Han-Yun Fu et al.· 0 citations
A controllable multi-speaker dialogue TTS framework that formulates synthesis as critique-driven iterative refinement, and unifies instruction-following synthesis and natural-language-guided attribute editing, enabling correction of expressive errors without full regeneration.
Kang-Xiang Xia, Xin-Fa Zhu, Hang-Rui Hu et al.· 0 citations