Explainable Machine Learning for DDoS Attack Detection with Physical Network Validation

Abstract

Distributed Denial-of-Service (DDoS) attacks remain one of the most disruptive threats to network infrastructure, yet many machine learning (ML)-based detection studies report only offline benchmark performance without verifying whether that performance holds under real network conditions. This study evaluates two explainable ML classifiers, XGBoost and Random Forest, for DDoS detection and examines whether their near-perfect offline accuracy translates into reliable physical-network operation. The study combines offline benchmarking on the CIC-DDoS2019 dataset (293,485 flows) with physical-network validation using a working Intrusion Detection System (IDS) prototype under a controlled SYN-flood attack. Session-disjoint stratified sampling prevented flow-level leakage across attack sessions, while SHapley Additive exPlanations (SHAP) interpreted global and local feature importance. Offline, both classifiers achieved near-perfect performance (accuracy 99.99% for XGBoost, 99.98% for Random Forest; F1 = 0.9999; ROC-AUC up to 1.0000), with no statistically significant difference between them (McNemar's exact test, p = 0.2188), though XGBoost achieved approximately 3.69 times higher inference throughput (1,819,816 flows/s). SHAP identified Min Packet Length, Fwd Packet Length Min, Inbound, Protocol, and Init_Win_bytes_forward as the most influential features. In physical deployment, however, the IDS prototype flagged 6.26% of captured flows (6,935 of 110,762) as ATTACK during the SYN-flood test, and a separate 397-flow ambient-benign subset yielded a 6.80% false positive rate (95% Wilson CI: 4.72–9.71%), with short-duration SSDP/UPnP-style UDP control traffic accounting for 70% of observed false positives. This gap shows that near-perfect offline accuracy does not guarantee low false positives in real deployment, indicating that offline benchmarks alone are insufficient for validating IDS readiness.