Skip to content

RESCUE: Repairing Language Model Errors to Sparse Circuits via Reinforcement Learning

Sep 2026 · 0 citations · 54 references
Computer Science

TL;DR

This work introduces RESCUE (Reasoning-Error Sparse-Circuit Uncovering and Editing), a framework that localizes error-associated circuits and surgically repairs them for performance enhancement, and introduces a pruning technique and precisely fine-tunes error circuits to correct task failures, thereby translating error localization into a sparse and targeted model update.

Abstract

Large language models (LLMs) exhibit strong general capabilities that mechanistic interpretability has attributed to sparse computational circuits. However, existing circuit studies emphasize preserving functionality or explaining safety, leaving the mechanisms underlying failures across a broader range of tasks largely unexplored. Extending circuit analysis from abilities to errors, we explore the perspective that such failures may likewise arise from erroneous internal computations and that targeted tuning of the corresponding parameters can correct such errors while largely preserving other capabilities. Motivated by this insight, we introduce RESCUE (Reasoning-Error Sparse-Circuit Uncovering and Editing), a framework that localizes error-associated circuits and surgically repairs them for performance enhancement. General tasks typically involve multi-step reasoning and long-form generation, where early deviations can cause prefixes to drift from supervised references, leading SFT-based mask optimization to overlook circuits involved in generation-time errors. RESCUE therefore refines these masks through reinforcement learning with multiple masked-model rollouts, improving their relevance to observed task failures. Finally, RESCUE introduces a pruning technique and precisely fine-tunes error circuits to correct task failures, thereby translating error localization into a sparse and targeted model update. We validate RESCUE on heterogeneous repair sets across two domains: (1) mathematical reasoning, identifying a math error circuit of 1.40% density whose repair raises accuracy from 6.0% to 75.5%; and (2) medical QA, where a similarly compact 1.44% circuit improves repair-set accuracy from 0% to 81%. Our code is available at: https://github.com/chuanpupig/RESCUE.

View source

Similar papers

Delayed Convergence and Emergent CoT Reliance in RL-Tuned Language Models

This analysis reveals that intermediate log-probability is an unreliable indicator for reasoning capability; instead, reasoning performance results from a shift of internal confidence allocation where RL fine-tuning delays internal convergence, exhibiting prolonged mid-layer exploration before converging sharply at the...

Pablo Pérez-Lázaro, Rocío Aznar-Gimeno, F. J. Lacueva-Pérez et al. · 0 citations
#machine learning Preprint Oct 2026

Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability

It is shown that intervention-based faithfulness can prefer an equally sized circuit that reproduces the model's behavior less well, and restoring selected intact-model signals reverses 96 of 100 selected persistent misrankings, demonstrating that context distortion contributes to these failures.

Chu-Qin Geng, Li Zhang, Hao-Lin Ye et al. · 0 citations
#machine learning Preprint Oct 2026

FORGE: Verification-Gated Behavioral Repair for Generative Language Models

Generative large language models (LLMs) inherit undesirable behaviors from pre-training, including demographic bias and toxic generation, that often emerge only after deployment and affect a small subset of inputs. A repair should eliminate the identified defect, preserve the model's overall functionality and, ideally,...

Hsin-Ling Hsu, Min-Yue Chen, Nai-Chia Chen et al. · 0 citations
#machine learning Preprint Oct 2026

Training Language Models to Fail Gracefully

This work proposes training models to fail gracefully: to preserve successful behavior whenever possible while learning to execute a safe fallback policy when acting would be unreliable, and shows that reinforcement learning can not only incentivize models to succeed, but also to reason about the limits of their capabi...

Daman Arora, Xue-Chen Liu, H. Singh et al. · 0 citations
#machine learning Preprint Aug 2026

Learning Generalizable Behaviors for Terminal Agents

River, a simple training recipe that improves reward quality by filtering low-quality environments and augmenting outcome rewards with process-level behavior regularization is proposed, which achieves the best performance among evaluated open-source RL-trained 8B models across four terminal-agent benchmarks.

Yi-Fan Yao, Bo Pang, Xuan-Phi Nguyen et al. · 2 citations
#artificial intelligence Preprint Sep 2026

Compositional Reasoning in Language Models under Reinforcement Learning Post-Training

A dependency-graph framework is proposed to formalize compositional reasoning, yielding three levels of compositionality with increasing complexity, and preliminary evidence that the decomposed-to-composed asymmetry can extend to practical settings is presented.

Yu He, Ying-Xi Li, Yi-Fei Wang et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.