Comparative Analysis of CatBoost, Random Forest, and XGBoost for Predicting Student Academic Performance

Abstract

Predicting student academic performance has become an important topic in educational data mining because it enables educational institutions to identify students who may require academic support at an early stage. This study compares the performance of three ensemble learning algorithms—CatBoost, Random Forest, and XGBoost—in predicting students' final academic grades using the Student Performance Dataset obtained from Kaggle. The dataset contains 649 student records with demographic, family, behavioral, and academic information. Before model development, the data were preprocessed through categorical feature encoding and feature engineering, resulting in five additional variables: parent_education, previous_grade, total_alcohol, attendance_category, and study_efficiency. The dataset was divided into training and testing sets using an 80:20 ratio. Hyperparameter optimization was performed only for CatBoost using the Optuna framework with 20 optimization trials, while Random Forest and XGBoost were trained using predefined parameter settings. Model performance was assessed using Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and the Coefficient of Determination (R²). The results indicate that CatBoost achieved the best overall performance, with an RMSE of 1.2888 and an R² of 0.8297, outperforming the other two algorithms. Feature importance analysis also revealed that G2 and previous_grade were the strongest predictors of students' final academic performance. These results suggest that CatBoost is a reliable approach for student performance prediction and can support data-driven academic decision-making.