Jun 2026· Proceedings of the ACM on Software Engineering· Vol 3, pp. 943 - 966· 0 citations· 87 references
TL;DR
LTMiner, which mines rare patterns from large-scale projects and detects potential bugs by checking for violations of these patterns, effectively curbs pattern explosion and false positives, uncovering previously unknown bugs in large-scale projects at an acceptable cost.
Abstract
Using data mining to extract frequent code patterns for bug detection has proven effective. However, prior studies have overlooked the prevalence of infrequent (rare) patterns, even though violations of such patterns can also lead to bugs. In this paper, we present LTMiner, which mines rare patterns from large-scale projects and detects potential bugs by checking for violations of these patterns. In practice, rare patterns far outnumber frequent ones and lack strong statistical support. Consequently, we face a pattern explosion, and many rare patterns and their violations are uninteresting. LTMiner addresses this by using instance-based ranking and filtering to prioritize violations of rare patterns. It further employs a large language model (LLM) as a domain expert to audit top-ranked violations; mined information supports in-context learning, and task decomposition and self-reflection mitigate possible hallucinations. This pipeline effectively curbs pattern explosion and false positives, uncovering previously unknown bugs in large-scale projects at an acceptable cost. Applied to Linux kernel 6.12.1, LTMiner identified 42 previously unknown bugs, 27 of which have been confirmed by developers. These results indicate that, although rare-pattern bugs are sparse, a considerable number remain and exhibit a non-negligible long tail. We believe that rare-pattern bugs constitute a promising blue ocean for bug detection.
Analysis of exception handling bugs in Python projects reveals systematic relationships between root causes and repair strategies, indicating that exception handling bugs often follow predictable patterns.
Jairo Souza, Eric Coelho, J. Correia et al.· Information and Software Tec...· 0 citations
Log parsing is a critical step in automated log analysis, enabling tasks such as debugging, monitoring, root cause analysis, and anomaly detection by transforming unstructured log messages into structured representations. Despite extensive research on log parsing, real-world log data exhibits a pronounced long-tail structure at the template level: a small number of frequent log templates dominate the data, while a large fraction of templates occur only a few times. These rare templates often correspond to failures, abnormal behaviors, or rare system events, yet they are severely underrepresented in commonly used benchmarks such as LogHub-2.0.In this paper, we conduct a comprehensive empirical study across multiple real-world log datasets to quantify the prevalence of rare log templates and to examine their impact on log parsing evaluation. Our analysis shows that, on average, approximately 12% of log templates are rare, while collectively accounting for less than 0.03% of log messages. In our research, rare log events defined as groups that having fewer than five associated log instances. We further demonstrate that commonly used evaluation metrics, such as Parsing Accuracy (PA) and Grouping Accuracy (GA), are dominated by frequent templates and can obscure substantial performance degradation on rare ones. Using template-level metrics, including F1_score of Template accuracy(FTA) and F1_score of grouping accuracy(FGA), we show that strong overall accuracy can coexist with poor performance on rare but operationally critical log templates. These findings highlight the need for evaluation protocols that explicitly account for template frequency when assessing log parsers in real-world settings.
The first empirical study of attention patterns in LLM-based program repair is presented, providing interpretable insights into how models process bug reports and where their attention is concentrated during repair, and indicates that stronger alignment between model attention and developer-identified key sections and phrases is associated with higher repair success.
Ramtin Ehsani, Irene Manotas, Saurabh Pujar et al.· 0 citations
Source code clones pose risks ranging from intellectual property violations to unintended vulnerabilities. Effective and efficient scalable clone detection, especially for diverged clones, remains challenging. Large language models (LLMs) have recently been applied to clone detection tasks. However, the rapid emergence of LLMs raises questions about optimal model selection and potential LLM-ensemble efficacy. This paper addresses the first question by identifying 76 LLMs and filtering them down to suitable candidates for large-scale clone detection via an LLM-encoder framework known as SSCD. The candidates were evaluated on two public, industry-defined datasets, BigCloneBench, and a commercial, large-scale dataset. No uniformly ’best-LLM’ emerged, though CodeT5+ 110M, CuBERT and SPTCode were top-performers. Regression analysis suggests that embedding size, tokenizer vocabulary, and training dataset characteristics are associated with clone detection performance. To address the second question, this paper explores the ensembling of selected LLMs to improve effectiveness. Results suggest the importance of score normalization and favoring ensembling methods like maximum or sum over averaging. Also, findings indicate that ensembling approaches can be statistically significant and effective on larger datasets, where the best-performing ensemble improved performance by 37% over its individual LLMs on the commercial large-scale code.
M. Chochlov, Gul Aftab Ahmed, J. Patten et al.· ACM Transactions on Software...· 1 citation
This paper presents a formal mathematical model for categorizing the outcome of generated-tests into four classes, a couple of basic metrics: Bug-Revealing Rate (BRR) and Bug-Validating Rate (BVR); and two basic statistical tests to ensure that the results are rigorous.
Zeyad Farooq Lutfi· Al-Noor Journal of Engineeri...· 0 citations
With the observed progress in machine learning (ML), and particularly the introduction of Large Language Models (LLMs), several activities related to code maintenance could be automated. That includes not only detection and evaluation of design flaws, but also code transformation and refactoring. However, the general-purpose LLMs, while being commonly used and popular, have not been specifically trained for code analysis, and may not be suitable for conducting software maintenance tasks due to biases, and inherent shortcomings of the models. In this paper, we explore if the widely available LLMs could aid the detection and the refactoring of code smells. We focus on four common smells (God Class, Long Method, Feature Envy, and Refused Bequest) and consider five prompts of diverse complexity, asking the model for detecting and removing the identified code smells. Results suggest that general-purpose LLMs cannot be reliably used for that. They can effectively detect or remove code smells only in simple cases, and frequently produce invalid code. However, their performance depends on various factors, e.g., the model, the specific code smell or the prompt objective and composition.
Giorgia Paisi, Francesca Arcelli Fontana, Bartosz Walter· WiPiEC Journal - Works in Pr...· 0 citations