Test-Free Fault Localization Through Deep Source Code Analysis Using Encoder-Based Language Models

Leveraging Code Language Models for Code Smells Detection

Improving software quality remains a major challenge in modern software engineering. Among the different techniques used to address this issue, code smell detection plays an important role, as code smells may negatively affect software maintainability, readability, and evolution. Recent advances in machine learning, particularly pre-trained code representation models, have opened new perspectives for automating this task. In this study, code smell detection is formulated as a multi-label classification problem in order to reflect realistic scenarios where multiple smells may co-occur within the same code fragment. To enable the evaluation of pre-trained code representation models in a multi-label setting, six multi-label datasets containing source code were derived from two existing datasets, namely MLCQ and Fontana, by merging instances sharing common code smells. The study investigates multiple experimental configurations based on CodeBERT, GraphCodeBERT, and CodeT5, including direct transformer-based classification, sliding window processing for long code sequences, and hybrid approaches combining transformer embeddings with classical machine learning classifiers. Among the evaluated configurations, the hybrid approach based on CodeBERT embeddings combined with a Random Forest classifier achieved the best overall performance, reaching an accuracy of 0.91 and a micro-F1 score of 0.90 on the Fontana datasets.

Applying Cost-Sensitive Learning to Mitigate the Effect of Class Imbalance for Deep Learning-based Fault Localization

Automated software fault localization has become a research focus with the growing software scale. Deep learning-based fault localization (DLFL) has been widely studied due to its excellent performance. Most existing DLFL methods use the execution traces of the test cases as features. However, the number of passing test cases is far more than that of failed test cases in the practical test suite. The severe class imbalance problem is ignored in existing DLFL methods. To alleviate the impact of the class imbalance on DLFL, we propose applying cost-sensitive learning to set different weights for each sample to optimize the performance of fault localization. We reduce the number of passing test cases and compute the weight for each sample. The sample weight is combined with a loss function to optimize the accuracy of the DLFL method. To evaluate our approach, we conduct within-project and cross-project fault localization experiments on the Defects4J benchmark. Experimental results show our approach outperforms the traditional DLFL and existing data augmentation methods in fault localization (e.g., localizing 133 faults within Top-5). The experimental results show that applying cost-sensitive learning can effectively alleviate the class imbalance problem in DLFL.

A deep learning framework for vulnerability detection in open-source repositories using code embeddings

CodeBERT-SENet: Adaptive Syntax-Semantic Fusion via Gated Attention for Python Bug Detection and Localization

Multilingual Source Code Vulnerability Detection Using Deep Learning: A Semantic Representation and Transfer Learning Approach

Detecting vulnerabilities in source code remains a major challenge as modern software systems increasingly span multiple programming languages. This work presents a deep learning approach for multilingual vulnerability detection that emphasizes semantic transfer rather than architectural complexity. The proposed method combines contextualized embeddings from CodeBERT/GraphCodeBERT with a BiLSTM and attention mechanism to capture code semantics and adopts a cross-lingual transfer setting where models trained on one language (e.g., Python) are evaluated on another (e.g., Java). To improve robustness under data imbalance, SMOTE and stratified cross-validation are incorporated into the training process. Experiments on Juliet and CodeXGLUE show that the model achieves an F1-score of about 0.87 and a ROC-AUC of 0.85 in intra-language settings, while guided fine-tuning improves cross-language F1-score by an average of 0.18 and ROC-AUC by approximately 0.13 compared with the direct transfer baseline. These results suggest that stabilizing semantic representations during transfer is key to improving generalization while maintaining practical efficiency under moderate computational constraints.

TokenGated-CodeBERT With Lightweight Attention-Based Token Selection for Software Defect Prediction

Class imbalance poses a key challenge in defective code module detection, severely limiting the performance of deep learning models. To address this issue, we propose TokenGated-CodeBERT, a new methodology designed for scenarios with severe class imbalance. Constructed on the basis of vanilla CodeBERT, the model adopts a lightweight token gating module to raise the weights of defective minority tokens and lower the weights of non-defective majority tokens. It dynamically fuses token embeddings based on defect correlation, emphasizing fault-related tokens and eliminating noise to enhance minority defect detection in feature learning. We conduct extensive validation on the PROMISE dataset for cross-project and cross-version defect prediction tasks, with supplementary tests on the Defects4J dataset for fine-grained code snippet prediction to comprehensively examine the generalization performance of the newly designed structural module. Experimental results illustrate that our method achieves competitive Matthews Correlation Coefficient (MCC) performance against all baseline approaches, demonstrating its effectiveness in class-imbalanced scenarios. An ablation study confirms that our token-level gating mechanism is the key driver of this performance gain, as it more effectively captures defect-related signals distributed throughout the code sequence. Our framework supports full end-to-end training, keeps the original CodeBERT encoder intact, requires no additional auxiliary optimization objectives, and features a lightweight architecture.