Skip to content

RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments

Sep 2026 · 10 citations · 58 references
Computer Science

TL;DR

RSIAgent is introduced, a training-free multi-agent framework for recursive self-improvement through autonomous memory construction that substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.3.

Abstract

Digital agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce \textbf{RSIAgent}, a training-free multi-agent framework for recursive self-improvement through autonomous memory construction. RSIAgent coordinates curriculum, actor, and verifier agents to continually explore the environment, validate outcomes, and retain environment-specific knowledge, including reusable causal relationships between actions, conditions, and consequences. It further adopts a \textbf{broad-then-deep} exploration strategy, combining parallel broad recursive self-exploration for discovering diverse environment structures with focused deep self-exploration for uncovering hard cases, hidden constraints, boundary conditions, and previously unknown causal dependencies. The resulting memory is frozen and can be directly reused for downstream tasks without updating model parameters. Experiments on OSWorld-v2 and Agent's Last Exam show that RSIAgent substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.

View source

Similar papers

Preprint Aug 2026

SPADE: Self-Play in Adaptive Synthetic Executable Environments

SPADE (Self-Play in Adaptive Synthetic Executable Executable Environments), a self-play RL framework in which a single LLM plays two roles: an Environment Designer that writes complete, long-horizon training environments as executable code with an OpenAI Gym-style reset()/step() interface, and a Reasoning Agent that le...

Bo Liu, Simon Yu, Yiding Jiang et al. · 7 citations
#artificial intelligence Review Sep 2026

RSI-Master: Structuring Experiments to Guide Autonomous Model Improvement

RSI-Master consists of an Experiment OS, which enables regularized experimental actions and maintains persistent, traceable experimental records, and Reviewer-Guided Research Orchestration, which organizes Workers and Reviewers in a dynamically growing research DAG.

Ya-Xin Du, Xi-Yuan Yang, Zhi-Fan Zhou et al. · 1 citation
#natural language process... Preprint Sep 2026

Dream-RSI: Recursive Self-Improvement through Evolving Worlds

A lightweight orchestration layer makes exploration explicit and programmable while leaving the underlying base agent unchanged, and a key insight is that accumulated discovery history can act as a replay simulator over the realized search space.

Tong Zheng, Xidong Wu, Zheng Zhang et al. · 13 citations
#artificial intelligence Preprint Sep 2026

Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representations

This paper investigates whether model's internal representations provide stronger signals of eventual task success in multi-turn agentic setups, and introduces two complementary methods: Latent Trajectory Dynamics (LTD) and the Action Representation Probe (ARP), which predicts success from representations formed at act...

Priyanka Mary Mammen, Emil Joswin, Srujananjali Medicherla · 1 citation
#artificial intelligence Preprint Sep 2026

MASkills: Continual Skills Optimization for Multi-Agent LLM Systems

MASkills presents a new agent-optimization pipeline that integrates skill-conditioned credit assignment, hierarchical credit aggregation, and momentum-smoothed optimization, enabling agent skill libraries to evolve through refinement, induction, consolidation, and pruning.

Huai-Yuan Yao, Xia-Oou Liu, Charles Fleming et al. · 2 citations · ⚡1
#natural language process... Preprint Sep 2026

NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness

NeoHorse-1 is presented, a family of agent-native models developed to explore this path toward harness-mediated RSI across successive iterations, and provides an initial prototype of this feedback-driven process and a path toward harness-mediated RSI across successive iterations.

NeoHorse Team, Guo-Liang Cao, Guo-Hao Dai et al. · 3 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.