Why LLM Agents Collapse Without Oversight: The Enforcement Gap as the Mechanism Behind Emergence World Failures
TL;DR
Reidual attack success concentrates where flags are unparseable or auditors leak; an RL-trained enforcement controller handles hedged and malformed verdicts that rule-based parsing cannot, cutting ambiguous-critique failure to a fraction of the rule-based baseline.
Abstract
Binding the audit flag in Reflexion-style agents --- without changing the auditor --- reduces attack success rate substantially, reaching near zero on models whose flags parse cleanly. This single control-flow change exposes the \textbf{enforcement gap}: the controller receives a safety flag and executes anyway. Separating detection probability $p_d$ from enforcement probability $p_e$ establishes that $p_e \approx 0$ by default across every framework we tested, making detection quality \emph{formally irrelevant} to security when enforcement is absent --- a finding consistent with the spontaneous collapses recorded in unsupervised frontier-agent deployments~\citep{emergence2026}. Residual attack success concentrates where flags are unparseable or auditors leak; an RL-trained enforcement controller handles hedged and malformed verdicts that rule-based parsing cannot, cutting ambiguous-critique failure to a fraction of the rule-based baseline. Concurrent filtering and information-flow defenses address detection, not enforcement, leaving the binding constraint untouched. The Audit Enforcement Specification (AES) packages three concrete requirements that close each residue independently; each primitive is adoptable without redesigning the host framework, and no deployed framework currently implements any of them.
