ML-Driven Cloud Intrusion Detection … Automated Response using the BETH Dataset

Abstract

Cloud infrastructure increasingly depends on automated security tools to detect and respond to threats at scale. This paper introduces a machine-learning-based intrusion detection and automated response framework, which is tested on the BETH dataset, a large set of Linux syscall events derived from honeypots. We develop five behavioural features: event rarity, user activity level, privilege status, syscall failure rate, and per-user rolling failure bursts. To tackle a 462:1 imbalance between normal and suspicious events, we apply SMOTE oversampling. Three supervised classifiers have been trained and tested: Random Forest, XGBoost, and Logistic Regression. Random Forest yields the best F1 score of 0.5152 on the suspicious class at 91% recall, better than XGBoost (F1 0.3682) and Logistic Regression (F1 0.3076). A confidence-gated automated response system sits on top of this classifier, reducing high-severity remediation actions by 99.61% compared to a naive all-flag approach. The complementary rule-based system and per-user rolling failure analysis are described. Overall, these results suggest that engineered behavioural signals and machine learning classification can develop a high-performing, transparent, and efficient cloud threat response pipeline. The framework is applied to 952,111 syscall events from BETH data and demonstrates the complete preprocessing and automated response pipeline, including a system architecture view.