Skip to content

Opera: A Verbal Critic Framework for Long-horizon Coding Agents

Sep 2026 · 0 citations · 39 references
Computer Science

TL;DR

Opera is presented, a verbal critic framework that treats each correction as a persistent note, followed until the diagnosed problem is resolved, and achieves the highest mean resolve rate among competitive critic baselines on all three benchmarks.

Abstract

Long-horizon coding agents need timely corrections, yet feedback can be ineffective or even harmful when it misjudges ongoing work or fails to address the underlying problem. Existing critics focus on evaluating trajectories and generating feedback, but rarely track what happens after feedback is delivered. We present Opera, a verbal critic framework that treats each correction as a persistent note, followed until the diagnosed problem is resolved. Opera decides when to review through periodic and event-driven triggers, diagnoses issues with typed operators, audits feedback against visible evidence before delivery, and tracks the agent's subsequent actions to distinguish mere compliance from actual resolution. As a test-time critic, Opera improves the resolve rate of non-critic agents by up to 12.4, 15.0, and 8.9 percentage points on Terminal-Bench 2.1, a SWE-Bench Pro subset, and DeepSWE v1.1, respectively, across four policy models, and achieves the highest mean resolve rate among competitive critic baselines on all three benchmarks, and also improves policy models when the policy critiques itself. Beyond inference, Opera-guided rollouts provide approximately on-policy training data: fine-tuning Qwen3.5-9B on them improves its resolve rate on held-out SWE-Bench Pro repositories by 10.2 percentage points without a critic at inference time, matching fine-tuning on rollouts from a stronger model, while preserving its performance when switching harness, i.e., from Openhands to Terminus-2, which the latter substantially degrades. Our code is available at: https://github.com/dongyuanjushi/Opera.

View source

Similar papers

StateTape: Action-Conditioned Evidence Lifecycle Modeling for Long-Horizon Coding Agents

A novel and scalable framework that rewrites a coding agent's context as the repository changes rather than as the context grows, and can achieve a higher resolve rate in all experiments spanned by six coding agents and three edit-heavy benchmarks with little computational overhead.

Zi-Yang Yu, Liang Zhao, Bo-Wen Zhu et al. · 0 citations

AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agents

Coding agents solve repository-level software engineering tasks through long trajectories of code inspection, search, editing, and testing. As a task progresses, earlier exploration becomes stale, so managing context is more than avoiding overflow: an agent must decide when to compact, what working state to preserve, a...

Xuan Zhang, Long-Tao Zheng, Cun-Xiao Du et al. · 7 citations
#natural language process... Preprint Aug 2026

CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents

CAST, a critique-aware training framework that converts sparse task outcomes into action-level supervision for critique learning and policy optimization, is presented, demonstrating that critique-aware training improves the robustness of LLM agents in realistic dynamic environments.

Amir Saeidi, Zeng Zhang, Rishi Singh et al. · 2 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.