Dredd: Scalable Source-Level Mutation Testing for Large C/C++ Codebases

Abstract

We present Dredd, a new tool that supports source-level mutation testing for large C/C++ codebases. Following the mutant schemata approach, Dredd applies a source-to-source transformation to a chosen set of files from the system under test (SUT), rewriting them into a metamutant that simulates a large number of distinct mutants, each selected via an environment variable holding the desired mutant’s ID. With no mutant selected, the metamutant behaves like the original SUT; otherwise it behaves as if a specific edit—such as replacing a binary operator or deleting a statement—had been applied. Because the metamutant is compiled and linked only once, mutation analysis across many mutants avoids expensive per-mutant recompilation and linking. We evaluate Dredd’s scalability on the Clang/LLVM compiler (a large C++ project) and the zstd compression library (a sizeable C project), and showcase an application of Dredd in compiler testing, using it to compare how effective the Csmith and YARPGen v1 and v2 tools are at killing mutants in Clang/LLVM.