Software Change Risk Prediction Using Supervised Machine Learning and Feature Impact Analysis

Abstract

Predicting risky software changes before deployment can help prioritize code review and testing, particularly in continuous integration and continuous deployment (CI/CD) environments. This paper presents an interpretable supervised learning framework for commit-level software change risk prediction using structural, historical, and process-related features extracted from the odoo open-source ERP repository. We compare Naive Bayes, logistic regression, support vector machine, feedforward neural network, and Random Forest classifiers. Random Forest achieves the strongest performance, with 82.2% accuracy, 81.8% precision, 84.5% recall, and an F1-score of 83.2%. Pairwise McNemar tests against the competing classifiers y ield p < 0.05, providing statistical support for the observed performance differences. Feature-subset and SHAP (SHapley Additive exPlanations) analyses show that code-churn characteristics—lines added, lines deleted, and files changed—provide the strongest predictive signal, with the churn-only representation performing close to the full feature set. Random Forest inference requires approximately 0.067 ms per commit in the revision experiment, indicating low prediction overhead. Overall, the results demonstrate the potential of lightweight, interpretable commit-level risk prediction for supporting risk-aware testing and code review in CI/CD workflows.