Skip to content

Activation-Conditioned Self-Distillation

Sep 2026 · arXiv.org · Vol abs/2609.38342 · 1 citation · 26 references
Computer Science

TL;DR

Activation-Conditioned Self-Distillation (ACSD), which extracts a steering vector by contrasting activations of self-generated trajectories that reach verified correct answers within a generation budget with those of all remaining trajectories, achieves the highest mean accuracy over four mathematical benchmarks among the evaluated methods.

Abstract

On-policy self-distillation uses a model as its own teacher to provide dense supervision for reasoning, often through reference-solution conditioning. Providing privileged information does not by itself ensure effective token-level supervision throughout long responses. We introduce Activation-Conditioned Self-Distillation (ACSD), which extracts a steering vector by contrasting activations of self-generated trajectories that reach verified correct answers within a generation budget with those of all remaining trajectories. A frozen copy of the base model applies this vector at each prediction position, and the student learns from its next-token distributions on student-generated prefixes. Outcome verification is used for direction construction and calibration; distillation requires neither problem-specific reference text nor teacher parameter updates. The distilled student is used alone at inference. On each of five models, ACSD achieves the highest mean accuracy over four mathematical benchmarks among the evaluated methods. On DeepSeek-R1-0528-Qwen3-8B, mean mathematical accuracy reaches 71.9\% and LiveCodeBench v6 pass@12 reaches 70.9\%, compared with 69.0\% and 66.3\% for the reference-conditioned OPSD baseline. Contrasts among correct trajectories also support distillation, and extracted directions can be reused across mathematical training datasets. On fixed student trajectories, ACSD maintains more stable late-position logit-update magnitudes than OPSD.

View source

Similar papers

#natural language process... Preprint Oct 2026

Self-Distillation with Verbal Regularization

Self-distillation with verbal regularization (VRSD), which uses system prompts to regulate how the teacher uses PI, thereby encouraging rethinking and exploration to counteract premature commitment, improves Pass@1 over OPSD in 11 of 12 model-task settings, while generally maintaining higher token entropy.

Jiazheng Li, Yawei Li, Jun-Ru Lu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

VASPO: Self-Distillation with Privileged Value Estimation

On-policy self-distillation provides dense supervision for reasoning models via a privileged self-teacher. However, this supervision reflects local token preferences rather than the expected return of an unguided student continuation. Recent methods combine this signal with GRPO. They use GRPO for update directions and...

Fei Wu, Zhen-Rong Zhang, Jian-Shu Zhang et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Not Every Rollout Needs Distillation: Correctness-Filtered On-Policy Self-Distillation

Correctness-filtered OPSD (C-OPSD), which generates one response per problem and distills only a subset of incorrect responses, is introduced, which connects outcome-conditioned supervision redundancy to more compute-efficient distillation.

Harrison Zhao, Lei Huang, Zhi-Yuan Zhao et al. · 0 citations

Teach Yourself Where to Look: On-Policy Attention Self-Distillation for Reasoning

On-Policy Attention Self-Distillation (OPASD), which complements token-level supervision with solution-conditioned attention distillation, shows that solution-conditioned attention provides a complementary supervision signal that makes on-policy self-distillation more accurate, stable, and compute-efficient.

Safaeid Hossain Arib, Rabeya Akter, I. N. Swapnil et al. · 1 citation

Recursive Self-Improvement via On-Policy Distillation for Reasoning

DCE+SRCL outperforms OPSD across multiple model scales and four competition-level mathematics benchmarks, and its comprehensive evaluations show that DCE+SRCL outperforms OPSD across multiple model scales and four competition-level mathematics benchmarks.

Shang-Jian Yin, Ze-Hao Zhao, Kavosh Asadi et al. · 6 citations
#machine learning Preprint Oct 2026

Teach Me After You Learn It: Lookahead Self-Distillation for RLVR

On-policy self-distillation (OPSD) has emerged as a popular training paradigm for LLMs, providing dense, token-level supervision by aligning a model's own distribution with the distribution induced by privileged information. However, existing OPSD methods derive supervision solely from privileged information of the sam...

Ze-Peng Zhai, Mei-Lin Chen · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.