Toward Agentic TestOps: A Systematic Literature Review and Autonomy Taxonomy for AI-Driven Testing in DevOps

Abstract

As DevOps pipelines accelerate release cycles from weeks to hours, testing practices face a fundamental misalignment: most AI-driven approaches automate isolated tasks rather than orchestrating testing activities across the delivery lifecycle. Although generative AI has broadened the capabilities of software testing, this task-centric focus leaves a critical autonomy gap unaddressed. To characterize this gap, this study presents a systematic literature review (SLR) of AI-driven software testing in DevOps environments. Through a structured and reproducible methodology that assessed study quality, 31 articles were selected from Scopus and Web of Science. The SLR evaluates the level of autonomy achieved by existing approaches and examines their integration within continuous integration and continuous delivery (CI/CD) pipelines. Within the journal-based corpus captured by the predefined review protocol, 27 of the 31 articles (87%) operate at task-level assistance, 3 (10%) demonstrate stage-level integration, only 1 (3%) exhibits partially agentic behavior within a bounded domain, and none demonstrates lifecycle-wide, goal-driven orchestration. Two additional shortcomings emerge from the literature: the absence of standardized criteria for evaluating testing autonomy and the limited empirical validation of proposed approaches in operational CI/CD environments. Based on these findings, this study introduces a four-level autonomy taxonomy for AI-driven testing and outlines the architectural, operational, and governance conditions required for future Agentic TestOps systems, understood as goal-driven and self-adaptive testing systems operating across delivery pipelines. Together, these contributions help define a research agenda for advancing DevOps testing toward more autonomous testing approaches.