Hesitation-Aware On-Policy Distillation for Diffusion Language Models
Diffusion large language models (dLLMs) generate text by iterative unmasking. At each denoising step, a dLLM proposes a token at every masked position, but the decoder commits only a confident subset of these proposals. Trace-based on-policy distillation (TOPD) builds on this process by matching the student to a strong...
Jian-Guo Huang, Li-Peng Wan, Yan-Chen Deng et al.
· 0 citations