Skip to content

Author

Wei-Jie Ren

We have 4 of 4 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

ReTaCo: Residual-Target Control for On-Policy Distillation

On-policy distillation (OPD) trains a student on its own generated prefixes with token-level teacher feedback, but transmitting or storing the teacher's full-vocabulary distribution at every token is costly. Entropy-aware OPD (EOPD) adds forward supervision to reverse KL to help the student recover plausible tokens it...

Zi-Xiang Ni, Zhuo-Lun Hu, Ren-Jie Cao et al. · 0 citations

OPSRD: On-Policy Self-Role Distillation

OPSRD is introduced, which uses a fixed expert role as privileged teaching context for on-policy self-distillation without reference solutions for on-policy self-distillation without reference solutions.

Wei-Jie Ren, Yan-Wen Zhang, Hao Li et al. · 0 citations
#artificial intelligence Preprint Sep 2026

SEPAL: Separated Expert Pairs with Answer-Level Fusion for Reliable LLM Collaboration

Multi-agent collaboration lets large language models (LLMs) improve question answering through deliberation and feedback. Yet shared discussion couples correction with exposure to the same mistakes, which can erode the diversity needed for voting. Self-consistency offers sampling diversity without feedback, while singl...

Wei-Jie Ren, Yan-Wen Zhang, Hao Li et al. · 0 citations
#machine learning Preprint Sep 2026

The Teacher Is a Direction, Not a Destination: Extrapolating RL-Induced Representation Residuals in On-Policy Distillation

Across four base/RL-teacher pairs spanning different scales, architectures, and pre-training lineages, RIDE approaches or exceeds the RL-trained teacher on every pair and is the only method whose mean does so, and it consistently outperforms output-space extrapolation, which degrades the student whenever the teacher is...

Hao Li, Mei-Jia Chen, Wei-Jie Ren et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.