Skip to content

When Tools Get in the Way: The Effect of Unnecessary Tool Availability on LLM Answering

Sep 2026 · 0 citations · 27 references
Computer Science

TL;DR

This work asks whether making a related but unnecessary tool available affects a model's ability to answer from its own knowledge, and whether a preceding tool interaction changes this behaviour.

Abstract

Large language models (LLMs) are increasingly deployed with external tools that extend what they can do beyond their own knowledge. Tools help on tasks that need external information, but their availability may also change how a model handles questions that do not need them. Prior work has mostly asked whether models select and use tools appropriately; whether an unnecessary tool changes the correctness of answers has received less attention. We ask whether making a related but unnecessary tool available affects a model's ability to answer from its own knowledge, and whether a preceding tool interaction changes this behaviour. We construct 500 query pairs across 10 knowledge domains. Each pair consists of a tool query, which needs the domain's tool, and a closed-domain query, which does not. Six LLMs are evaluated with the tool unavailable, available, and available after a prior tool call. Across 3,000 baseline trials the pooled answer rate is 98.2%. When an unnecessary tool is available it falls to 63.5%, with large differences between models. The decrease occurs even when the tool is rarely called, so it cannot be explained by unnecessary tool invocation alone. A one-sentence scope-aware system instruction recovers most of the lost answers.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

This work compares country-continent questions with noun, adjective, and code answers while keeping several fitted measurements distinct across Qwen, Llama, and Gemma to separate early readability, natural strength, causal steering, and later content dependence.

Wen-Lin Wei, Yuan Fang, Ren-He Jiang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Toollery: Scaling LLM Agents to Thousands of Skills and Tools

Toollery is presented, a training-free candidate-compression framework for scalable LLM skill/tool selection that keeps online selection bounded to a compact top-$k$ candidate set and improves recall over ordinary specification retrieval.

Xiang-Xi Tian, Ran-Yun Guan · 0 citations
#natural language process... Open access Oct 2026

Structured but Silent: Probing Capability Requirements in LLM Hidden States

Reliable tool use requires more than triggering a mechanism or matching a query to an API description. Before selecting a specific tool, an agent must first infer the capability requirements implied by the user query. In this paper, we investigate whether these query-side capability requirements are linearly decodable...

Kyo-Nam Choo, Mi-Ra Song, Yun-Ju Kang et al. · 0 citations
Preprint Sep 2026

Tool Retrievers Are Underestimated: Annotation Expansion Reveals True Capability

The proposed ToolEX (Tool Equivalent eXpansion), a framework that automatically discovers and annotates the tool combinations functionally equivalent to the labeled ones, and confirms that the one-to-one problem extends beyond tool retrieval.

Yan-Yu Zhu, Chen-Heng Zhang, Shao-Shen Chen et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.