Skip to content
Review

From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale

Jul 2026 · arXiv.org · Vol abs/2607.29516 · 0 citations · 15 references
Computer Science

TL;DR

ARCTIC, an AI-powered Code Critique system that reframes code review around three capabilities: intent prediction, which infers why a change was made from conversation logs and metadata; drift detection, which measures divergence between the developer's intent and the agent's output via backtranslation; and code spotlight, which ranks the regions of a diff most warranting human scrutiny.

Abstract

AI coding agents are generating code at volumes that exceed the capacity of traditional peer review. At the same time, existing AI code review tools over-index on low-value suggestions such as style and best practices while under-indexing on the concerns human reviewers prioritize most: correctness, security, and performance. We present ARCTIC, an AI-powered Code Critique system that reframes code review around three capabilities: intent prediction, which infers why a change was made from conversation logs and metadata; drift detection, which measures divergence between the developer's intent and the agent's output via backtranslation; and code spotlight, which ranks the regions of a diff most warranting human scrutiny. We ground these capabilities in a six-theme taxonomy derived from 18,000 code reviews. Offline evaluation shows that intent prediction achieves 0.86 F1, drift detection reaches near-perfect ordinal agreement with human annotators (QWK = 0.907), and spotlight outperforms the baseline AI reviewer by 2.4x on quality estimation at 5x fewer tokens. In the experimental rollout, the drift scores reduces code misalignment by an additional 5.76 points (p = 0.026), intent prediction receives 90.2% approval, and zero defects have been attributed to self-reviewed diffs since launch.

View source

Similar papers

Review Jul 2026

Code Review is a Conversation: Toward Conversational AI Review Assistants

This vision reframes AI code review from automated commenting to human-AI sensemaking before integration, and outlines a research agenda for studying review conversations, designing conversational AI review capabilities, and evaluating their impact on software evolution and maintenance.

Rosalia Tufano · 0 citations
Review Sep 2026

Beyond Lexical Metrics: Sentence-Embedding Detection of Reviewer Habituation in AI Code Review

Code review is a key quality checkpoint between AI-generated code and production and reviewer adaptation is detectable in latent distributional structure rather than in classical lexical metrics, and language shifts follow rather than precede changes in approval behavior.

Hao-Ran Yu, Li-Fei Liu, Dan Zhang · 0 citations
#artificial intelligence Preprint Sep 2026

What is the Difference Between Me and You? Benchmarking the Quality Gap Between Human-Written and AI-Generated Code

This work compares human-written and AI-generated code at scale, characterize structural complexity and statistical naturalness, and map static-analysis findings onto Orthogonal Defect Classification for defects and the Common Weakness Enumeration for vulnerabilities, making authors and languages directly comparable.

Cristina Improta, Pietro Liguori, D. Cotroneo · 0 citations
Review Open access Aug 2026

Large language models as judges: recent advances in LLM-based evaluation, critique, preference modeling, and feedback for text and code

This survey provides a comprehensive overview of recent advances in LLM-based evaluation, covering techniques, applications, and challenges across domains, with future directions emphasizing standardized protocols, uncertainty estimation, and human–AI collaboration.

M. Nadăş · 0 citations
Review Sep 2026

Not All Agents Are Equal: Code Quality and Post-Merge Maintenance Across Five Autonomous Coding Agents in the Wild

This paper studies 37,623 provenance-labeled pull requests from five commercial agents and combines the AIDev dataset with 58,792 cached GitHub API responses to measure security smells in added code, structural maintainability, post-merge churn, revert rates, and human review behavior.

Obada Kraishan · 0 citations
Review Jul 2026

"Go Home Copilot, You're Drunk": Understanding Developer Responses to Agent-Generated Code Review Comments

The first large-scale empirical study on the resolution of agent-generated code review comments is presented, revealing that the presence of an inline code suggestion is the strongest predictor of comment resolution, while lengthy and complex comments are less likely to be acted upon.

Shamse Tasnim Cynthia, Ratnadira Widyasari, Banani Roy et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.