Evaluasi Kritis Random Forest dalam Prediksi Kegagalan CI/CD Pipeline pada Dataset Log Sintetis
TL;DR
It is concluded that the use of a synthetic log dataset constitutes the primary limitation and the root cause of the model's generalization failure, and future research is recommended to utilize actual operational logs from real-world CI/CD platforms to enable the model to demonstrate more valid causal relationships.
Abstract
Continuous Integration and Continuous Deployment (CI/CD) pipelines are crucial in modern DevOps environments; however, pipeline failures often hinder software delivery. This study aims to implement Machine Learning specifically the Random Forest algorithm to predict the specific stage of failure (Build, Test, or Deploy phases) using execution logs. The research methodology encompasses data preprocessing, the extraction of three key dynamic features (execution duration, CPU usage, and memory consumption), model training using an 80:20 train-test split, and performance evaluation via a confusion matrix. Evaluation based on 9,000 test samples representing 20% of the total 45,000 synthetic failure logs yielded an overall accuracy of 33.22%, with F1-scores of 0.33 for the Build class, 0.35 for Deploy, and 0.32 for Test. These results indicate that the Random Forest model possesses very limited generalization capability because the synthetic dataset lacked naturally robust failure trace patterns, as evidenced by an accuracy level approaching that of random guessing. It is concluded that the use of a synthetic log dataset constitutes the primary limitation and the root cause of the model's generalization failure in this study. Therefore, future research is recommended to utilize actual operational logs from real-world CI/CD platforms, such as GitHub Actions or Jenkins, to enable the model to demonstrate more valid causal relationships.