multiplex: A Modular LLM-Based Mutation Framework

Abstract

Mutation testing introduces synthetic defects into a program under test to assess test suite adequacy. While recent advances in large language models (LLMs) have led to more realistic and hard-to-kill mutants, the field is currently dominated by single-purpose research prototypes that tightly couple the mutant generation technique with the LLM provider and build environment, thereby requiring mutation testing researchers to re-implement techniques for empirical comparison. To address this challenge, this paper introduces multiplex, a modular LLM-based mutation framework. multiplex enables researchers to implement their own LLM-based mutant generation (MG) modules to prototype or compare techniques, as well as configure and create language, LLM or framework-specific modules for extended study. In addition to providing an overview of the system design and implementation, this paper demonstrates how researchers can use multiplex to prototype and compare techniques, along with sharing experiments conducted with multiplex and outlining future extensions. The tool is available at https://github.com/LLM-Mutation/multiplex, and a video demonstration is available at https://youtu.be/l-hj7Pmn3oo.