Compilomorphic fuzzing is introduced, a new validation approach that turns a compiler into its own test oracle via structure-preserving transformations over programs and structured inputs, establishing a practical, model-free foundation for validating compilers in critical DSL domains.
Abstract
Compilers for structured data pipelines, often specified using domain-specific languages such as P4, are notoriously difficult to validate. We introduce compilomorphic fuzzing, a new validation approach that turns a compiler into its own test oracle via structure-preserving transformations over programs and structured inputs. The core idea is to generate pairs of semantically equivalent but structurally distinct inputs (e.g., packets) and programs (e.g., pipeline specifications), and then assert that the compiler's observable behaviour remains invariant. Any deviation exposes a deep semantic bug. We present the formal basis for this approach, compilomorphism, and a prototype fuzzing framework that operationalises it, establishing a practical, model-free foundation for validating compilers in critical DSL domains.
Fuzzing Rust library APIs requires constructing well-typed, compilable call sequences that satisfy ownership rules, generic parameters, and trait bounds; existing tools ignore these constraints or use shallow heuristics, yielding low coverage. We present GRAFT, which extracts structured API information from Rust documentation, builds an API dependency graph via recursive generics-aware type matching, and uses topology-guided traversal plus LLM synthesis with compiler-error feedback to produce compilable fuzz targets. On 13 crates from crates.io, GRAFT achieves 80.75% macro-average API coverage at 96.19% compilation success, outperforming RULF and RPG by 4.76x and 2.43x, and reaching 1.41x the average API coverage of deepSURF on crates with unsafe-reaching APIs.
Yiming Chen, Kaiwen Zhang, Guanjun Liu et al.· 0 citations
It is shown that generative compilation reduces non-compiling outputs and improves functional correctness, relative to standard post-generation feedback, by detecting a broad range of errors close to their source and early during generation, thereby reducing errors cascades and enabling focused diagnostics.
Niels Mündler-Sasahara, Hristo Venev, Dawn Song et al.· 0 citations
MongoDB maintains client libraries in a dozen programming languages, used by tens of thousands of organizations and millions of developers. Most are implemented natively rather than as wrappers around a shared core. Ensuring consistent behavior across these libraries, comprising millions of lines of code, is hard but essential. Over eleven years, we developed a specification-based testing approach: tests are written once in YAML and executed by language-specific interpreters for each library. We describe the evolution from many ad-hoc formats to a Unified Test Format, which allowed us to delete over 22,000 lines of test code. The rate of nonconformance bugs fell up to 86% in drivers that adopted YAML tests (though results varied). We report lessons learned about declarative test design, test architecture, schema evolution, and the limits of unification.
A. Jesse, A. Jesse Jiryu Davis, Jeremy Mikola et al.· 0 citations
Transpilers are increasingly used for software development, especially in industrial domains that rely on domain-specific languages (DSLs), to allow engineers to work with familiar concepts and appropriate abstractions. Ensuring the correctness of these instruments is therefore critical in many industrial settings. This paper observes that existing approaches for compiler testing hardly generalize to transpilers. Differential testing approaches are hindered as multiple equivalent implementations of the transpiler under test are seldom available in practice. The approaches based on metamorphic testing assume the ability to execute the compiled binaries, an assumption that cannot be always made for transpilers, which oftentimes produce results expressed as source code, requiring complex toolchains, hardware-in-the-loop setups, and depending on non trivial inputs. This paper introduces a novel metamorphic testing technique tailored to transpilers. Instead of reasoning about the runtime behavior of compiled programs, our approach defines metamorphic relations directly over the source code produced by the transpiler. These relations capture a property that we call mutation consistency of the (transpiled) programs: mutation-style changes in the input DSL program must induce predictable and structurally consistent changes in the generated output. We implemented this idea in a tool, MCP-Tester, and evaluated it through a case study conducted in the context of a technology-transfer project. Our current empirical results indicate that the proposed approach can effectively reveal faults that would remain undetected with pure fuzzing.
Enea Raffaele Ilario Papaleo, Luca Guglielmo, G. Denaro· 0 citations
