Finding Typing-Related Bugs in JVM Implementations

Abstract

As an object-oriented language, Java relies heavily on its type system. Existing testing efforts mainly target type-related implementations in the Java compiler, while the substantially different type-related implementations in the Java Virtual Machine (JVM) remain insufficiently tested. Although many techniques have been proposed for direct JVM testing, they mostly focus on specific components such as the JIT compiler or garbage collector, leaving type-related implementations underexplored. In this paper, we present Caster, the first technique for testing JVM type-related implementations by directly generating bytecode test programs guided by type specifications in the Java Language Specification (JLS). Specifically, Caster designs a set of feature primitives to encapsulate the operations needed for type-related testing, and leverages a Large Language Model (LLM) to identify type-related test scenarios from the JLS and guide primitive composition. Based on these scenarios, Caster organizes the selected primitive combinations into a multi-unit test-program generator, whose Cartesian-product expansion produces a large number of concrete test programs. To support large-scale execution, Caster dynamically loads these generated programs from memory and invokes them via reflection across different JVM implementations, collects their result lists, and compares them to identify inconsistencies and detect potential JVM bugs. Extensive experiments on four Long-Term Support OpenJDK versions of three popular JVM implementations (i.e., HotSpot, OpenJ9, and Dragonwell) demonstrate that Caster outperforms the state-of-the-art techniques in detecting more JVM inconsistencies. Moreover, Caster has uncovered a total of 21 previously unknown defects in the latest JVM releases, among which 18 have been confirmed or fixed by developers.