Skip to content

When2Think: Learning When and How Much to Reason

Sep 2026 · arXiv.org · Vol abs/2609.19671 · 1 citation · 59 references
Computer Science

TL;DR

This work proposes When2Think, an RLVR-based post-training framework for instance-adaptive computation allocation that requires neither a learned reward model nor a learned critic, and offline reference caching avoids online reference-model queries during policy updates.

Abstract

Large Reasoning Models (LRMs) often overthink easy problems and underthink hard ones, leading to inefficient computation allocation. Existing methods regulate generated computation or select between direct answering and explicit reasoning, but do not jointly control whether}to reason and how much computation to allocate within reasoning. We call the resulting difficulty-dependent loss in accuracy under computation reduction the efficiency tax. We propose When2Think, an RLVR-based post-training framework for instance-adaptive computation allocation. Its core mechanism, Instance-level Difficulty-Aware Control (IDAC), uses cached reference statistics of success and token cost to modulate a correctness-gated efficiency bonus based on generated token count. Importance sampling supports exploration of Think and NoThink, while Batch-Wise Standardization constructs standardized advantages for critic-free optimization. The framework requires neither a learned reward model nor a learned critic, and offline reference caching avoids online reference-model queries during policy updates. On AIME24, When2Think improves Pass@3 by 10.0 percentage points while reducing token usage by 27.9% relative to the backbone.

View source

Similar papers

Preprint Aug 2026

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation

A reasoning model is built that adaptively chooses how much to reason for each problem, and the brief modes end up more accurate than \textsc{Long}, which shows that the router sorts problems by difficulty rather than at random.

Gijs Kassenaar, Zhao Yang, Vincent François-Lavet · 2 citations
#natural language process... Preprint Sep 2026

From Base Rollouts to RL Reasoning: A Budgeted Search Perspective

Results support a qualified internalized-search reading: under the recipe the authors test, much of the measured RL gain corresponds to a change in sampling efficiency toward operating points the base model can already reach under search.

Wen-He Sun, Cun-Xiang Wang, Zi-Jun Yao et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Adaptive Power Sampling for LLM Reasoning

Experiments across diverse reasoning tasks, including MATH500, HumanEval, and GPQA, show that APS consistently outperforms power sampling with a fixed sharpening exponent, without additional training.

Bing-Nan Xiao, Chen-Hao Yang, Bing-Cong Li et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Mitigating the Length-Scaling Tax with Online Distillation

LSD is proposed, which routes solved prompts to on-policy distillation and retains the original RL objective for unsolved prompts and achieves comparable or better performance than RL across multiple variants, while substantially curbing response-length growth on easy queries.

Xu Wan, Wen-Yue Xu, Sheng-Jie Zhao et al. · 0 citations

The Model Knows Another Way: Strategy Switching for Effective RLVR Exploration

This work introduces Problem--Strategy Rollout Allocation (PSRA), which treats unguided and strategy-conditioned prompts as competing exploration arms and uses Bayesian sequential allocation to direct a fixed rollout budget toward arms most likely to yield informative, non-saturated groups.

Jin Cui, Xin-Yue Long, Bo-Ran Zhao et al. · 1 citation

OpenJev-RLCD: A Working RLCD Implementation

Decision models such as Jev answer questions with probabilities, which are only useful if they are calibrated. Open-source reproductions rely on supervised fine-tuning plus temperature scaling, while reinforcement learning from verifiable rewards (RLVR) makes reasoning models overconfident. We present a working impleme...

Zhi-Min Gao, Pi-Chao Wang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.