Large language model (LLM)-based web search agents have demonstrated remarkable capabilities in autonomous information retrieval and multi-step reasoning. However, their robustness under real-world distribution shifts, such as evolving webpage structures, dynamic content layouts, and unseen task environments, remains a critical open challenge. Existing approaches predominantly rely on static supervised fine-tuning with human-annotated trajectories, which not only incurs substantial labeling cost but also lacks the adaptive capacity to handle the inherent stochasticity of live web environments. In this paper, we propose TAPE (Trajectory Alignment and exPerience pool Evolution), a novel self-evolving fine-tuning framework designed to enhance the generalization and robustness of web search agents without requiring large-scale human annotation. TAPE introduces a dual-stream experience pool that continuously accumulates both successful and failed agent trajectories during self-play execution. A trajectory alignment module maps heterogeneous execution paths into a unified semantic space, enabling contrastive learning to identify failure-inducing operations and reward generalizable search strategies. The framework further incorporates an adaptive pool evolution mechanism that filters, reweights, and distills experiences to prevent knowledge stagnation and distributional collapse. Extensive experiments on the GAIA benchmark and BrowseComp dataset demonstrate that TAPE consistently outperforms strong baselines across all three difficulty levels, achieving up to 6.2% absolute improvement in task success rate on GAIA Level-2 tasks (a 134.8% relative gain over the SFT-Only baseline on the GAIA validation split) and 1.3% absolute (217% relative) improvement on BrowseComp hard-tier queries, while exhibiting significantly greater resilience to webpage structure perturbations. These improvements are consistent across three open base models spanning two model families and the 7B–14B scale range. Our analysis further suggests that the contrastive self-play signal can serve as a useful partial surrogate for human preference labeling in agent trajectory optimization, substantially reducing, though not eliminating, reliance on human-annotated trajectories.
Minjae Rhee, Jitong Zou, Tianjun Mo et al.· IEEE Access· 0 citations
Large Language Model (LLM)-based autonomous agents have shown significant promise in automating software engineering tasks, yet existing systems still suffer from two fundamental limitations: i) the lack of persistent experiential knowledge across debugging sessions, which forces agents to repeat exploratory mistakes, and ii) the static nature of prompt structures, which prevents agents from adapting their reasoning strategy to recurring failure modes. To address these gaps, we propose LS-CM (Long-Short Collaborative Memory), a dual-memory architecture that couples a Retrieval-Augmented Generation (RAG)-based short-term memory for code-base context with a Reinforcement Learning (RL)-driven long-term memory that distills strategic patterns from historical test failures and patch iterations. The core innovation is a self-evolving context engineering module that automatically refines the agent’s internal prompt structure based on execution feedback signals, treating the prompt as a configurable policy object that a gradient-free controller selects among pre-generated variants, rather than a fixed artifact. We evaluate LS-CM on two industry-standard benchmarks: SWE-bench, for end-to-end issue resolution on real-world GitHub repositories, and Defects4J, for bug localization and program repair in Java projects. Across multiple LLM backbones, LS-CM resolves 33.7% of SWE-bench Lite issues, a modest gain over the strongest reproduced baseline (Agentless, 32.0%), and improves Top-1 fault localization accuracy on Defects4J from 42.6% to 47.6% while reducing editing churn by 31%. We find that LS-CM’s most robust advantages lie in debugging efficiency, cross-episode strategy transfer, and run-to-run stability rather than in a single headline resolved-rate number. Ablation studies confirm that short-term retrieval, long-term policy learning, and self-evolving prompts each contribute to the gains. Our work demonstrates that explicitly modeling memory and prompt adaptation as first-class learning objects substantially enhances agent productivity on real-world software engineering tasks.
Tianjun Mo, Changhao Zhang, Jitong Zou et al.· IEEE Access· 0 citations
