Automated Testing and Its Impact on Software Development Quality and Maintainability: A Focused Structured Review
TL;DR
A transition from code-level defect metrics toward process- and system-level maintainability concerns is highlighted, and five research gaps are identified: separating testing practice from testing metrics, context-aware empirical design, standardized measurement and reporting, longitudinal maintainability tracking, and cross-domain replication.
Abstract
Automated testing is widely adopted in modern software development, yet empirical evidence on its relationship to software quality and maintainability remains fragmented across studies that differ in scope, metrics, and methodology. This focused structured review examines the development of automated-testing evidence from early defect-density and code-churn research to large-scale open-source mining studies, industry experiments, and systematic reviews of test-driven development (TDD). The review analyzes five foundational studies covering code churn, industrial TDD adoption, comparative automated-versus-manual testing, large-scale coverage analysis, and systematic synthesis of TDD quality effects. It then incorporates five recent studies extending this foundation toward industry experimentation, Kitchenham-protocol systematic review, practitioner-perceived technical debt, cross-domain validation in quantum software, and a meta-analysis of the reliability of existing TDD systematic reviews. The selected studies are analyzed according to their objectives, evidence source, methodology, evaluation outcomes, and limitations. The synthesis highlights a transition from code-level defect metrics toward process- and system-level maintainability concerns, and identifies five research gaps: separating testing practice from testing metrics, context-aware empirical design, standardized measurement and reporting, longitudinal maintainability tracking, and cross-domain replication. The review concludes by outlining corresponding future directions for evaluating automated testing's role in software quality and long-term maintainability.
