Tool-Integrated Reasoning (TIR) is a fundamental capability for LLM agents to solve complex tasks by interacting with external tools iteratively. Reinforcement Learning (RL) has become the dominant paradigm for enabling this capability. However, existing approaches typically assign uniform trajectory-level advantages and treat all correct tool calls equally, ignoring the varying difficulty and learning value across trajectories and reasoning steps. This can lead to imprecise learning signals that do not adequately distinguish between trivial and challenging tool-use patterns. To address this limitation, we propose HiDiffTIR, a Hierarchical Difficulty-aware policy optimization framework for multi-turn TIR. HiDiffTIR performs difficulty-aware credit assignment at both trajectory and turn levels, enabling the policy to focus on more informative trajectories and harder reasoning steps. Notably, this fine-grained optimization is achieved without additional supervision, relying solely on group-level statistics derived from standard RL rollouts. Extensive experiments on three tool-using benchmarks demonstrate that HiDiffTIR consistently improves multi-turn TIR performance and tool invocation accuracy over strong RL baselines, highlighting the necessity of difficulty-aware credit assignment for effective policy optimization in tool-integrated LLM agents.
Yucan Guo, Xiaohan Wang, Miao Su et al.· 0 citations
DeepRepro dynamically transforms evolving repository states and runtime feedback into fine-grained implementation subplans, keeping planning aligned with execution throughout repository construction, and consistently outperforms strong scientific and commercial code-agent baselines.
Hongru Song, Ruqing Zhang, Jiafeng Guo et al.· 0 citations
TokenWall is proposed, a runtime defense framework that acts as a semantic firewall over agent token flows, demonstrating that semantic runtime containment can achieve a practical security-utility trade-off for persistent AI agents.
Puji Wang, Yingchen Zhang, Ruqing Zhang et al.· 1 citation
This work conducts a systematic study of overthinking and overretrieval in search agents from a scaling perspective and proposes a lightweight post-hoc reflection framework that converts the proposed evaluation signals into explicit feedback rewards to guide agents' reasoning trajectories.
Xin Liu, Ruqing Zhang, Yu-An Liu et al.· Annual International ACM SIG...· 0 citations
This paper introduces Search in the Model, a novel training-free framework that performs fine-grained identification and rectification of vulnerable neurons directly within the model, and develops a maximum marginal vulnerability criterion to precisely locate the top-K neurons most responsible for model vulnerability.
Yuansan Liu, Ruqing Zhang, Hongru Song et al.· Annual Meeting of the Associ...· 0 citations