On-policy distillation consolidates specialist models into one student at the price of rollouts, teacher passes and gradient steps for every teacher. We show that the consolidation does not need training: with forward passes only, the capabilities of separately tuned specialists can be aggregated into one model that is...
Bang Yang, Jia-Jun Fan, Zhao-Yu Zhu et al.· 0 citations
How much can test-time computation improve a language model, and at what cost? Test-time scaling is widely proposed as a substitute for larger models, but existing comparisons mostly evaluate one domain at a time and rarely charge selection to the budget. We introduce SELF-POT, a benchmark and evaluation framework that...
Bang Yang, Jing-Yuan Li, Jia-Jun Fan et al.· 0 citations
Scaling reasoning typically spends more compute on reinforcement learning (RL) or on inference. We show that a completed RL training history can yield policies stronger than the checkpoints visited by its optimizer. We call this policy-space scaling: expanding the deployable policy set accessible from a fixed RL histor...
Bang Yang, Jia-Jun Fan, Hong-Bo Ma et al.· 0 citations
This work is the first to unveil the critical role of the null space and harness it for model optimization, and uses a simplified analytical model about optimization to demonstrate why null-space can effectively reduce attention entropy, thereby improving the efficiency of reasoning.
Hong-Bo Ma, Sansheng Cao, Jia-Jun Fan et al.· 0 citations
SpeechGym is presented, an audio-native agentic environment in which two omni-modal models converse in native audio, with no external ASR or TTS and no API boundary, over the unmodified tasks, tools and success check of an established text agentic benchmark, so that the interaction modality is the only variable and the...
Reading a base Qwen3-Omni with a logit lens at the audio-token positions, it is found that the answer to a spoken question becomes legible - in words - in the model's middle layers, before it emits any token.
This work introduces Constraint-First Reasoning (CFR), a training-free two-stage prompting protocol that improves direct CoT on multiple backbones and positions CFR as a targeted test-time intervention whose benefit depends on recoverable constraints and reliable Stage 1 extraction.
Hongbo Ma, Bang Yang, Y. Cheng et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.