Skip to content
#artificial intelligence Conference Open access

OSPD: On-Policy Self-Distillation for Persona-Consistent Dialogue

Sep 2026 · Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing · Vol abs/2609.34418 · 0 citations · 23 references
Computer Science

TL;DR

This work proposes OSPD, an on-policy self-distillation framework where the same model serves as both teacher and student under asymmetric information: the teacher receives a complete character profile while the student sees only a brief summary, and the student generates trajectories from its own policy.

Abstract

Maintaining persona consistency across multi-turn dialogues remains a core challenge for role-playing language models. Off-policy distillation from external teachers incurs distribution mismatch that compounds across dialogue turns, while reinforcement learning struggles with reward ambiguity inherent in subjective persona fidelity. We propose OSPD, an on-policy self-distillation framework where the same model serves as both teacher and student under asymmetric information: the teacher receives a complete character profile while the student sees only a brief summary, and the student generates trajectories from its own policy. We find that teacher confidence in role-playing dialogue exhibits a bimodal structure---sharply peaked at character-critical tokens yet diffuse at generic utterances---and introduce role-aware divergence switching to match this structure. A progressive trait masking curriculum further forces staged internalization of character knowledge along semantic dimensions. Experiments on CharacterBench, CharacterEval, and SocialBench show that OSPD substantially improves persona consistency over supervised fine-tuning and multi-turn RL baselines, without requiring any external teacher or reward model.

Read PDF

Similar papers

OPSRD: On-Policy Self-Role Distillation

OPSRD is introduced, which uses a fixed expert role as privileged teaching context for on-policy self-distillation without reference solutions for on-policy self-distillation without reference solutions.

Wei-Jie Ren, Yan-Wen Zhang, Hao Li et al. · 0 citations
Book Open access Oct 2026

Self-Distillation as a Structure-Dependent Mechanism in Multimodal Dialogue via Multi-Context Modeling

Multimodal dialogue understanding requires integrating heterogeneous signals such as language, audio, and visual cues while capturing complex contextual dynamics in multi-party interactions. Although recent models achieve strong performance, they typically rely on unified representations and treat self-distillation as...

Ryo Ishii, Chihiro Takayama, Jiro Nagao et al. · 0 citations
#artificial intelligence Preprint Sep 2026

RoleBreak: Benchmarking Long-Horizon Role-Playing Robustness in Spoken Dialogue

RoleBreak, an open benchmark for long-horizon role-playing robustness in spoken dialogue, is introduced and four key patterns are found, highlighting persistent gaps in both long-horizon robustness and vocal expressiveness in spoken role-playing systems.

Yu-Qi Wang, Feng-Yuan Liu, Hao-Chen Luo et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Improving OCR Faithfulness via Gated and Attenuated On-Policy Distillation

GAD-RL is introduced, which adaptively regulates teacher supervision during joint post-training according to the student's current task performance and local distributions, and weights forward KL by the student's probability of the teacher's Top-1 token, moderating local auxiliary updates when student support for that...

Bao-De Wang, Zu-Ming Huang, Ke Ren et al. · 0 citations
#artificial intelligence Preprint Oct 2026

MeSD: Multi-Evidence Self-Distillation for VideoLLM

While reinforcement learning with verifiable rewards provides reliable outcome supervision for VideoLLMs, sequence-level rewards offer limited token-level guidance. On-policy self-distillation addresses this limitation by conditioning a self-teacher on privileged information to provide dense token-level supervision. Ho...

Wei-Jie Zhu, Han Fang, Han-Yun Fu et al. · 0 citations
#natural language process... Preprint Sep 2026

From Script to Drama: An Agentic Framework for Controllable Multi-Speaker Dialogue TTS

A controllable multi-speaker dialogue TTS framework that formulates synthesis as critique-driven iterative refinement, and unifies instruction-following synthesis and natural-language-guided attribute editing, enabling correction of expressive errors without full regeneration.

Kang-Xiang Xia, Xin-Fa Zhu, Hang-Rui Hu et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.