Skip to content

ScienceClaw: Benchmarking Continual Self-Evolution of AI-for-Science Agents Across the Natural and Social Sciences

Oct 2026 · 0 citations · 35 references
Computer Science

TL;DR

This work formalizes ScienceClaw as fixed-parameter program self-evolution that unifies task solving, scientific verification, and program updates and measures scientific correctness, evolutionary gain, retention, cross-dataset transfer, and evolution cost through sequential streams and independent reset evaluation.

Abstract

Large language model agents are accelerating scientific automation, yet verified executions rarely become persistent program-level improvements, and existing evaluations do not examine this process across sequential tasks in both the natural and social sciences. We formalize ScienceClaw as fixed-parameter program self-evolution that unifies task solving, scientific verification, and program updates. ScienceClaw-Eval spans 23 disciplines and measures scientific correctness, evolutionary gain, retention, cross-dataset transfer, and evolution cost through sequential streams and independent reset evaluation. Our framework repairs executable workflows through multi-turn interaction, converts re-execution-verified failure--success trajectories into linked Skill and Operator candidates, and retains an update only when source-task replay reproduces the repair and independent scientific tasks improve. Code is available at https://github.com/beita6969/ScienceClaw.

View source

Similar papers

Conference Open access Sep 2026

GENE: Generalizable Need-Driven Evolution of Coding Agents with Post-Evolution Verification

Agent harnesses critically affect coding-agent performance, but their automatic evolution can overfit individual failures and introduce unforeseen regressions. We present GENE, a training-free framework for generalizable and verifiable harness evolution. Instead of translating each task failure directly into a harness...

Bin Zhou · 0 citations
Preprint Oct 2026

Exo: An Architecture for Self-Improving Systems at Runtime

Agent harnesses govern how language models act in an environment; designing their policies by hand is difficult, while automated techniques require representative datasets and costly executions. We present Exo, an agent runtime architecture for safe learning during ongoing work that collapses the optimization loop into...

Alexander Krentsel, Sylvia Ratnasamy, Ion Stoica et al. · 0 citations
Preprint Oct 2026

SCI-AGENT-EXAM: Benchmarking Agents on Scientific Research Tasks

Large language model (LLM) agents are increasingly capable of supporting scientific workflows, but their capabilities across diverse research activities remain difficult to assess systematically. Existing benchmarks often focus on selected scientific capabilities or task types, limiting a comprehensive characterization...

Tian-Yun Zhong, Wei Wang, Jia-Ling Li et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Self-Evolving Agent Harness

Harness self-evolution revises an agent's software scaffolding, including prompts, memory, workflows, and executable components, while keeping model weights fixed. We formalize a feedback-driven process in which accepted revisions persist across runs. The framework separates update targets from proposal and admission m...

Chu-Yang Wei, Hao-Xiang Guan, Ming-Hang Zhu et al. · 0 citations
Preprint Nov 2026

ScienceCUA: Resource-to-Skill Construction and Execution-Guided Skill Evolution for Scientific Computer Use Agents

Computer-use agents for scientific software require application-specific expertise. Much of this expertise is documented in human-authored software manuals and web pages. However, turning these resources into executable, reusable skills remains challenging due to mismatches between documented and deployed software vers...

Dai-Qiang Li, Jing Peng, Rui Hu et al. · 0 citations
#artificial intelligence Preprint Nov 2026

SciMARS: Multi-Agent Recursive Self-Improvement for Closed-Loop Scientific Discovery

We propose SciMARS, a framework for recursive task-local adaptation of multi-agent harnesses in closed-loop scientific discovery. Multi-agent systems coordinate analysis, experimental design, and verification as scientific evidence evolves. However, two problems limit their effectiveness: (i) without shared memory, age...

Wei-Min Wu, Han Liu · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.