Recent methods using Reinforcement Learning (RL) have improved Tool-Integrated Reasoning (TIR) by training large language models to learn end-to-end policies for multi-step tool usage, enabling them to solve complex tasks more effectively. Despite these advances, existing methods often suffer from overthinking at both the action and reasoning levels: models tend to invoke tools redundantly and generate excessively long reasoning trajectories, resulting in high computational cost. To address this, in this paper, we propose LightTIR, a dual-penalty reward framework, to achieve efficient TIR. For action efficiency, LightTIR estimates the marginal utility of each tool call through prefix-aligned counterfactual trajectories, encouraging calls that contribute meaningful information while penalizing low-utility or redundant invocations. For reasoning efficiency, LightTIR introduces a length-aware regularization term, adaptively penalizing intermediate reasoning steps that exceed the minimal effective trajectory required for correct prediction. Extensive experiments demonstrate that LightTIR can reduce redundancy and trajectory expansion while maintaining answer correctness, achieving more efficient RL-based TIR. Code is available at https://github.com/ekventitas/LightTIR.
Yichen Xiao, Siyu Gong, Linan Yue· Proceedings of the 32nd ACM...· 0 citations
Recent advances in multimodal large reasoning models (MLRMs) have demonstrated impressive capabilities on complex multimodal tasks, yet their reliance on long Chain-of-Thoughts (CoTs) often leads to redundant reasoning and high computational cost. Existing chain-based distillation and refinement approaches alleviate redundancy to some extent but remain limited when handling ultra-long reasoning chains, suffering from weakened dependency modeling, accumulated redundancy, and the omission of critical reasoning steps. To this end, in this paper, we propose GRACE, a Graph-based Rationalized CoT Compression framework that transforms long CoTs into reasoning graphs to enable efficient CoT compression, where reasoning steps are represented as nodes and semantic dependencies as edges. Specifically, after constructing the reasoning graph, GRACE applies a graph rationale learning approach with a selector–predictor architecture. Among them, the selector estimates node-level rationale scores to sample a compact rationale subgraph, while the predictor leverages this subgraph for final prediction. The resulting subgraph serves as the distilled core rationale, retaining essential reasoning while removing irrelevant steps. Finally, the compressed CoTs are then used to build supervised fine-tuning datasets for further MLRMs tuning. Experiments on multiple multimodal benchmarks show that GRACE substantially improves inference efficiency and interpretability without sacrificing accuracy.
Yizhi Wang, Linan Yue, Deng-Bao Wang et al.· Proceedings of the 32nd ACM...· 0 citations