When AI Agents Say "Passed": Evidence-Based Verification and Quality Gates in Agentic Software Development

Abstract

Agentic software-development systems can generate code, analyse repositories, execute development workflows, and report the results of automated checks. However, a system reporting successful completion does not necessarily establish that its output has satisfied the required quality and verification criteria. This technical report examines evidence verification and quality-gate enforcement in agentic software-development systems, using Recon as a case study. It focuses on the distinction between model-reported success and independently verifiable evidence, with particular attention to quality-gate enforcement, evidence validation, evaluation methodology, and software-development governance. An initial evaluation involving four known software flaws is discussed. The observed results provide a limited case study rather than a general measure of agentic software-development reliability. The small evaluation set does not establish performance across different repositories, programming languages, models, or development environments. The report identifies important engineering considerations for systems that use AI agents to analyse and modify software. These include deterministic verification, fail-closed quality gates, traceable evidence, secure handling of model errors, reproducible evaluations, and clear separation between implemented capabilities and proposed improvements. The report also outlines a more rigorous evaluation methodology using larger test sets, independent verification of findings, repeated runs, and explicit measurement criteria. The aim is to contribute a practical engineering perspective on how agentic software-development systems can make their results more transparent, testable, and trustworthy. Keywords: Agentic AI, AI Agents, Software Engineering, Software Quality, Quality Gates, Evidence Verification, Autonomous Software Development, AI-Assisted Programming, Software Testing, Engineering Governance. Research status: Technical report. The findings are limited to the evaluation described in the report and should not be interpreted as proof of general performance or production readiness.