CodeFault: Predicting Fault Risks from Code Changes via Multi-modal Learning
Abstract
Just-In-Time defect prediction identifies fault-inducing code changes but relies solely on static code properties, missing runtime fault patterns. Meanwhile, log-based anomaly detection captures runtime behaviors but lacks visibility into causal code modifications. This disconnect delays fault identification and complicates root cause analysis. We present CodeFault, a multi-modal fault prediction framework that fuses code change semantics with runtime log patterns through two novel mechanisms. Hierarchical Fault Spectrum Alignment (HFSA) projects both modalities into a shared, two-level fault taxonomy space—four coarse and eight fine-grained categories—providing a structured information bottleneck that enables principled cross-modal knowledge transfer. Spectrum-Guided Cross-Modal Attention (SGCA) performs token-level fusion conditioned on fault spectrum agreement, avoiding the degeneracy of pooled-vector approaches while focusing attention on fault-relevant token pairs. A dual-granularity contrastive objective further leverages the taxonomy structure to separate fault-inducing from benign embeddings at both instance and fault-type levels. We evaluate CodeFault on two complementary testbeds. On the public benchmark of 31,142 samples from Defects4J, BugsInPy, and ManySStuBs4J, CodeFault reaches 0.891 AUC-ROC and 0.857 F1-score against 15 prior-art baselines and 2 self-constructed strong late-fusion controls, improving over the strongest code-only baseline by 5.1% AUC and over the strongest late-fusion control by 3.1% AUC (Wilcoxon signed-rank test, Cliff's δ ≥ 0.73). On 2,147 real co-occurring CI/CD pairs collected from 55 GitHub Actions repositories with the CI build outcome as ground truth—the largest non-synthetic CI/CD fault test set in JIT defect prediction to date—CodeFault reaches 0.843 AUC, +4.3% over JIT-Smart, retaining 94.6% of the synthetic-set headline. The framework generalizes to unseen projects with 2.5% average AUC improvement and maintains 97.9% AUC under realistic 15% positive rates. A knowledge-distilled code-only variant achieves 0.862/0.831 AUC on synthetic/real test sets without log inputs, enabling pre-commit deployment.