Think Less, Act Warranted: Efficient Tool-Integrated Reasoning via Dual-Efficiency Regularization
TL;DR
LightTIR, a dual-penalty reward framework, is proposed to achieve efficient TIR and can reduce redundancy and trajectory expansion while maintaining answer correctness, achieving more efficient RL-based TIR.
Abstract
Recent methods using Reinforcement Learning (RL) have improved Tool-Integrated Reasoning (TIR) by training large language models to learn end-to-end policies for multi-step tool usage, enabling them to solve complex tasks more effectively. Despite these advances, existing methods often suffer from overthinking at both the action and reasoning levels: models tend to invoke tools redundantly and generate excessively long reasoning trajectories, resulting in high computational cost. To address this, in this paper, we propose LightTIR, a dual-penalty reward framework, to achieve efficient TIR. For action efficiency, LightTIR estimates the marginal utility of each tool call through prefix-aligned counterfactual trajectories, encouraging calls that contribute meaningful information while penalizing low-utility or redundant invocations. For reasoning efficiency, LightTIR introduces a length-aware regularization term, adaptively penalizing intermediate reasoning steps that exceed the minimal effective trajectory required for correct prediction. Extensive experiments demonstrate that LightTIR can reduce redundancy and trajectory expansion while maintaining answer correctness, achieving more efficient RL-based TIR. Code is available at https://github.com/ekventitas/LightTIR.