Two-Phase Feature Optimization for Fast and Accurate Software Defect Prediction

Abstract

In software defect prediction, improving model efficiency and training speed is as important as maintaining accuracy. Traditional machine learning and deep learning methods often rely on a large number of features, which can lead to excessive complexity and slower training times without providing significant gains in predictive power. These challenges are particularly prominent in complex software architectures, where understanding feature dependencies and interactions is critical for effective defect prediction. Reducing model complexity and enhancing training efficiency through the generation and analysis of feature interactions is therefore essential for optimizing performance. In this paper, we propose a two-phase training approach integrated with Synthetic Minority Over-sampling Technique (SMOTE) to address the inherent class imbalance in defect data. In the first phase, mutual information and Lasso regularization are applied to reduce feature redundancy, filtering out irrelevant features and retaining only those most predictive of defects. This streamlined feature extraction aggressively shrinks the feature space, directly reducing the overall model complexity. In the second phase, we focus on exploring multi-feature interactions by generating interaction terms using polynomial feature expansion. Subsequently, Partial Dependence Plots (PDP) are employed strictly as a post-hoc interpretability tool to visualize and validate the impact of these high-order relationships, ensuring the model captures essential feature synergies without altering original code semantics. Extensive experiments on the BugHunter dataset demonstrate the superiority of our approach. By explicitly decoupling offline feature preparation from online model fitting, we reveal that our dimensional reduction leads to a substantial 53.1% decrease in pure model fitting time for baseline classifiers like Logistic Regression. More importantly, comparative evaluations against classical baselines, including Principal Component Analysis (PCA) and tree-based feature importance, confirm that our method achieves an optimal balance between computational efficiency and predictive robustness. When applied to complex architectures such as Neural Networks, our selected feature subset attains a peak overall F1-Score of 0.786 and a Matthews Correlation Coefficient of 0.363. Furthermore, perturbation-based consistency evaluations demonstrate a 97% stability rate across iterations, proving that the extracted interactions represent robust architectural patterns rather than statistical anomalies. Ultimately, this method simplifies the model while preserving robust performance, making it more suitable for large-scale software defect prediction tasks, and contributing to both improved training efficiency and reliable defect prediction.