Skip to content

Self-Evolving Agents via Likelihood-Guided Tool-Space Optimization

Sep 2026 · 0 citations · 59 references
Computer Science

TL;DR

This work introduces LOTS (Likelihood-Only Tool Scoring), which evolves an agent's tool space from accumulated output experience while keeping model parameters fixed and demonstrates that task-specific spaces persist and continue to improve over time, while cross-model experiments show that learned configurations transfer across different models.

Abstract

Self-evolving agents can continually improve their behavior, while tools define the executable action space through which they interact with the environment. However, exposing the full tool library to model introduces substantial irrelevant context and can impair tool-use decisions. We study tool-space self-evolution, where each recurring task type maintains a persistent tool space which is constructed from accumulated output experience. We identify three limitations of existing methods: (1) output-unaware selection: they rely primarily on tool descriptions or model priors rather than observed tool outputs; (2) statelessness across request: they select tools independently for each request without consolidating prior output experience into persistent task-specific state; (3) inference cost: they repeatedly search, rank, or reason over candidate tools for subsequent requests of the same task. We address these limitations through output-aware tool scoring, persistent task-specific tool spaces, amortized tool selection, and reusable configurations across models. We introduce LOTS (Likelihood-Only Tool Scoring), which evolves an agent's tool space from accumulated output experience while keeping model parameters fixed. After each request, LOTS holds the model's generated answer and estimates each tool's contribution by measuring how much the answer likelihood changes when its observed output is removed. These contributions are aggregated within each recurring task to rank tools and update its persistent space. Across three benchmarks, LOTS improves task performance while substantially reducing tool context. More importantly, sequential experiments demonstrate that task-specific spaces persist and continue to improve over time, while cross-model experiments show that learned configurations transfer across different models.

View source

Similar papers

Experience Funnel: A State-Policy Alternating Loop for Self-Evolving Agents

Experiments across diverse agent benchmarks show that \textit{Experience Funnel} consistently improves agent capability over state-only evolution and policy-internalization approaches, while progressively converting useful explicit experience into autonomous policy competence.

Wen-Bo Gao, Zhao-Mou Song, Zhi-Yuan Ji et al. · 1 citation
Preprint Aug 2026

Rethinking Self-Evolving Agents: Do We Still Need Prescribed Optimization Pipelines?

This work introduces Open-Ended Optimization (OEO), which keeps the objective, permitted interactions, resource budget, data boundary, and evaluation fixed while allowing the optimizer to compose the improvement process online.

Xue Hui, Fan Yang · 3 citations

When Successful Strategies Fail: Adaptation to Environmental Novelty in Terminal Agents

This work introduces AGNI, an automated pipeline that extracts trajectory-relevant assumptions, injects targeted environmental changes, and validates that the resulting novel tasks remain solvable and highlights a gap between task competence and adaptive capability and motivate environmental variation as a core dimensi...

Janvijay Singh, Vaishnavi Shrivastava, Dilek Hakkani-Tur et al. · 0 citations

The Right Lesson at the Right Step: Deriving Control Updates for Self-Evolving Agents

Results show that self-evolving agents should place experience inside the control flow, rather than only store it as text, and show how learned conventions must reach the right execution step on long tool-use environments where learned conventions must reach the right execution step.

Yun-He Su, Zi-Yi Dong, Tong Yu et al. · 0 citations

CompoWorld: Compositional Environment Scaling for General Agents

This work introduces Compositional Environment Scaling (\textbf{CompoWorld}), which expands the task space by composing a finite library of reusable services, and guides reinforcement learning toward full task completion by emphasizing criteria with lower pass rates within each rollout group.

Xiao-Wen Yang, Wei-Yi Xu, Wen Da et al. · 1 citation
#artificial intelligence Preprint Sep 2026

SelfSearch: Reward-Free Search for Self-Improving Agents

SelfSearch is introduced, a reward-free search procedure in which agents modify themselves using records of previous self-improvement episodes, providing concrete experience for improving both task solving and self-modification.

Jungwoo Yang, InJin Kong, Yohan Jo · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.