This work introduces LOTS (Likelihood-Only Tool Scoring), which evolves an agent's tool space from accumulated output experience while keeping model parameters fixed and demonstrates that task-specific spaces persist and continue to improve over time, while cross-model experiments show that learned configurations transfer across different models.
Abstract
Self-evolving agents can continually improve their behavior, while tools define the executable action space through which they interact with the environment. However, exposing the full tool library to model introduces substantial irrelevant context and can impair tool-use decisions. We study tool-space self-evolution, where each recurring task type maintains a persistent tool space which is constructed from accumulated output experience. We identify three limitations of existing methods: (1) output-unaware selection: they rely primarily on tool descriptions or model priors rather than observed tool outputs; (2) statelessness across request: they select tools independently for each request without consolidating prior output experience into persistent task-specific state; (3) inference cost: they repeatedly search, rank, or reason over candidate tools for subsequent requests of the same task. We address these limitations through output-aware tool scoring, persistent task-specific tool spaces, amortized tool selection, and reusable configurations across models. We introduce LOTS (Likelihood-Only Tool Scoring), which evolves an agent's tool space from accumulated output experience while keeping model parameters fixed. After each request, LOTS holds the model's generated answer and estimates each tool's contribution by measuring how much the answer likelihood changes when its observed output is removed. These contributions are aggregated within each recurring task to rank tools and update its persistent space. Across three benchmarks, LOTS improves task performance while substantially reducing tool context. More importantly, sequential experiments demonstrate that task-specific spaces persist and continue to improve over time, while cross-model experiments show that learned configurations transfer across different models.
Experiments across diverse agent benchmarks show that \textit{Experience Funnel} consistently improves agent capability over state-only evolution and policy-internalization approaches, while progressively converting useful explicit experience into autonomous policy competence.
Wen-Bo Gao, Zhao-Mou Song, Zhi-Yuan Ji et al.· arXiv.org· 1 citation
This work introduces Open-Ended Optimization (OEO), which keeps the objective, permitted interactions, resource budget, data boundary, and evaluation fixed while allowing the optimizer to compose the improvement process online.
This work introduces AGNI, an automated pipeline that extracts trajectory-relevant assumptions, injects targeted environmental changes, and validates that the resulting novel tasks remain solvable and highlights a gap between task competence and adaptive capability and motivate environmental variation as a core dimensi...
Results show that self-evolving agents should place experience inside the control flow, rather than only store it as text, and show how learned conventions must reach the right execution step on long tool-use environments where learned conventions must reach the right execution step.
Yun-He Su, Zi-Yi Dong, Tong Yu et al.· arXiv.org· 0 citations
This work introduces Compositional Environment Scaling (\textbf{CompoWorld}), which expands the task space by composing a finite library of reusable services, and guides reinforcement learning toward full task completion by emphasizing criteria with lower pass rates within each rollout group.
Xiao-Wen Yang, Wei-Yi Xu, Wen Da et al.· arXiv.org· 1 citation
SelfSearch is introduced, a reward-free search procedure in which agents modify themselves using records of previous self-improvement episodes, providing concrete experience for improving both task solving and self-modification.