Early Warning Analytics for Information System Development Project Delays: A Comparative Study of Statistical and Machine-Learning Models Using Audit Reports

Abstract

Information system development projects (ISDPs) frequently miss their planned schedules, and prior studies of ISDP success and failure have largely relied on surveys, interviews, and case analyses that limit objectivity and predictive use. This article proposes an audit-report-based early-warning framework and benchmarks eleven classifiers on 446 class-balanced ISDPs (223 on-time, 223 delayed) executed for South Korean mobile telecommunication operators between 2011 and 2019: two statistical models (MDA, LR), four classical machine-learning models (MLP, CART, C5.0, SVM), and five modern tabular baselines (RF, GB, XGBoost, LightGBM, CatBoost). Ten Project Performance Management System predictors are used under nested stratified 5-fold cross-validation with inner-fold grid search; performance is reported with accuracy, precision, recall, F1, ROC-AUC, PR-AUC, and MCC together with per-model 95 percent confidence intervals across the five outer folds. The RBF-SVM attains the highest observed mean accuracy at 67.04 percent, followed by LR (65.03 percent) and MDA (64.81 percent); however, after Holm–Bonferroni correction across the ten pairwise McNemar tests reported in Table 8, no baseline is separated from SVM at the 5 percent level (all Holm-adjusted p $\gt =0.109$ ), so the SVM lead is directionally consistent but not statistically decisive on this corpus. Cross-referencing SVM KernelSHAP, SVM permutation importance, gradient-boosting TreeSHAP, and decision-tree root-node analysis identifies PM (project-manager commitment) and ENO (number of regular workers) as consistently top-five across all four methods, with TEST (verification and validation) additionally top-five in three of the four; these predictors form a compact interpretable early-warning indicator core for project management offices.