COMPLIANCE THEATRE: RETHINKING EVALUATION AND ENFORCEMENT IN FRONTIER AI REGULATION

Abstract

This article argues for a re-evaluation of the current orthodoxy in artificial intelligence (AI) regulation, driven by the inconvenient reality that existing mechanisms for measuring AI capabilities are inadequate for the task of evaluating general-purpose AI. Lawmakers and legal scholars alike have drastically overestimated our understanding of frontier AI systems and how they work. Nascent governance frameworks often assume a basic capacity for evaluating general-purpose AI that is simply not supported by the technical literature – they are built on a house of cards. The rapid pace of progress on the AI frontier has taken general-purpose AI past the point where a human expert can reliably interpret or interrogate their behaviour, particularly to the legal standards that lawmakers have codified to date. This article explores how technological complexities within general-purpose AI create sui generis challenges for regulators and the law. Faced with the prospect of an anthropocentric ceiling to efforts to understand frontier AI systems, this article proposes a new paradigm for lawmakers, Sentinel Governance, grounded in governance-oriented innovation and experimentation to supplement human oversight of AI. New mechanisms for evaluation and enforcement are needed to avoid AI regulations subsiding into a checkbox exercise.