Utility-Guided Orchestration for Cost-Efficient Tool-Augmented LLM Services
TL;DR
A lightweight utility-guided orchestration framework that formulates agent control as a costaware sequential decision problem over a compact action space, intended as an inspectable control layer for practical LLM services rather than a universally dominant accuracy optimizer.
Abstract
Tool-augmented large language model (LLM) services can solve complex tasks through retrieval and external tools, but current execution paradigms often trade adaptability for efficiency. Fixed workflows are predictable but rigid, while freeform reasoning loops such as ReAct may over-execute and issue redundant tool calls. We propose a lightweight utility-guided orchestration framework that formulates agent control as a costaware sequential decision problem over a compact action space: respond, retrieve, tool call, verify, and stop. An interpretable utility function balances expected gain, step-cost proxies, uncertainty, and redundancy. Experiments on multi-hop question answering show that the policy offers a controllable quality-cost trade-off and reduces token consumption by up to 10.6% in the semantic-redundancy setting while preserving similar answer quality. The framework is intended as an inspectable control layer for practical LLM services rather than a universally dominant accuracy optimizer.