Evaluasi Komparatif Random Forest, LightGBM dan MLP pada Dataset Tabular UCI dengan Beragam Karakteristik

Abstract

Classification performance on tabular data can vary according to dataset characteristics, class distribution, feature structure, and learning mechanisms. This study aims to comparatively evaluate the performance of Random Forest, LightGBM, and Multi-Layer Perceptron on heterogeneous UCI tabular datasets using a consistent and leakage-controlled experimental framework. The study employed Breast Cancer Wisconsin, Wine Quality, Adult Income, and Spambase datasets. Data preprocessing included missing-value imputation, correlation-based feature reduction, class-imbalance handling using synthetic oversampling, categorical encoding, and feature scaling for the neural network model. Hyperparameters were optimized using Grid Search with three-fold inner cross-validation, while five-fold stratified nested cross-validation was used for unbiased performance estimation. Evaluation employed accuracy, precision, recall, weighted F1-score, and area under the receiver operating characteristic curve. Descriptively, LightGBM achieved the highest average accuracy of 0.846608 and weighted F1-score of 0.850399 across datasets, whereas Random Forest achieved the highest average area under the receiver operating characteristic curve of 0.868093. However, the Friedman test found no statistically significant performance differences among the three models for accuracy, weighted F1-score, or area under the receiver operating characteristic curve. Permutation feature importance further revealed that dominant features varied across models and datasets. These findings indicate that model performance is context-dependent and that tabular model benchmarking should integrate dataset diversity, rigorous validation, statistical testing, and model interpretability.