AsyncLeakBench: A Curated Benchmark of Asynchronous Resource Leaks in Open-Source Java Projects

Abstract

Asynchronous programming is widely used in modern Java software, including server-side processing, network I/O, reactive streams, task scheduling, and RPC communication. Unlike synchronous executions, resource creation and release in asynchronous programs are distributed across callbacks, Future/Promise chains, threads, and scheduler boundaries. Consequently, resource lifecycles may depend on execution ordering, races, cancellation, and timeouts, making resource leaks a significant source of performance degradation and reliability failures. Prior research and public datasets have largely focused on synchronous resource leaks, which typically involve missing close calls or unhandled exceptional paths within a single call stack. In asynchronous environments, however, resource release may depend on callback execution, Future completion, and scheduler decisions, and the release path may change under cancellation, timeouts, or reordered execution. Existing datasets therefore do not adequately capture the triggering conditions and repair strategies of asynchronous resource leaks, limiting the systematic evaluation of detection and automated repair techniques. To address this gap, we present AsyncLeakBench, a public benchmark of real-world asynchronous resource leak defects fixed in open-source Java projects. Using a semi-automatic mining workflow, we collected 16,242 candidate defect–patch pairs from 31 open-source Java projects. Through iterative filtering, duplicate removal, async-specific validation, and manual inspection of resource lifecycles and patches, we identified 902 high-confidence defect–patch pairs. We further classify these cases into 11 categories that characterize major triggers and repair strategies involving cancellation, timeouts, scheduler boundaries, and other asynchronous events. An initial evaluation of existing resource leak detectors reveals limited effectiveness, particularly for leaks triggered by cancellation and timeouts. AsyncLeakBench provides a realistic and reproducible basis for evaluating resource leak detection and repair, static and dynamic analysis, fault localization, and LLM-based debugging. By characterizing asynchronous resource leaks as a distinct defect class and providing a standardized benchmark, this work enables systematic research on their detection, localization, and repair.