Sentiment Analysis of Indonesian Mobile Legends: Bang Bang Reviews Using Random Forest, KNN, TF-IDF, and SMOTE
TL;DR
The ablation study demonstrates that the effectiveness of SMOTE is model-dependent, improving Random Forest but degrading KNN performance, and provides empirical evidence of SMOTE impact on different classifiers and employs cross-validation-based K selection to prevent test data leakage.
Abstract
Mobile Legends: Bang Bang (MLBB) is one of the most popular mobile games, generating a large number of user reviews on the Google Play Store. The large volume of reviews makes manual sentiment analysis impractical, requiring an automated machine learning approach. This study compares the performance of Random Forest and K-Nearest Neighbors (KNN) for classifying sentiment in Indonesian-language MLBB reviews. A total of 6,994 reviews were obtained through web scraping and preprocessing. The proposed framework includes text preprocessing, rating-based sentiment labeling, TF-IDF feature extraction, SMOTE-based class balancing, model training, and evaluation using Accuracy, Precision, Recall, F1-score, Macro-F1, Weighted-F1, and 5-fold cross-validation. Random Forest with SMOTE achieved the best performance, with an Accuracy of 84.0% and a Macro-F1 score of 80.8%, outperforming KNN with SMOTE, which achieved 73.8% Accuracy and 71.9% Macro-F1. The ablation study demonstrates that the effectiveness of SMOTE is model-dependent, improving Random Forest but degrading KNN performance. This study provides empirical evidence of SMOTE impact on different classifiers and employs cross-validation-based K selection to prevent test data leakage.