Comparison of Naive Bayes, SVM, and Logistic Regression for Sentiment Analysis of the Makan Bergizi Gratis Program

Abstract

The Makan Bergizi Gratis Program (MBG) became one of the widely discussed public issues on platform X and generated diverse responses from users. These responses included supportive, critical, and neutral opinions, making sentiment analysis relevant for understanding public opinion toward the program. This study compares three text classification algorithms, namely Naive Bayes, LinearSVC based Support Vector Machine, and Logistic Regression, to analyze the sentiment of tweets concerning the Makan Bergizi Gratis Program. Data were collected from platform X on 19 June 2026 using tweet crawling and were subsequently filtered and manually annotated. The final dataset consisted of 1,061 Indonesian language tweets classified into negative, neutral, and positive sentiment. Two annotators independently assigned sentiment labels, followed by discussion to resolve disagreements. The inter annotator agreement reached 94.62%, with a Cohen's Kappa coefficient of 0.9097. Text preprocessing consisted of cleaning, normalization, tokenization, stopword removal, and stemming. TF-IDF features were generated within a pipeline to ensure that feature fitting was performed only on the training portion of each evaluation fold. The data were divided into training and testing sets using an 80:20 stratified split. Hyperparameter optimization was performed for each classifier, followed by evaluation using accuracy, macro precision, macro recall, macro F1 score, confusion matrix, and stratified 5 fold cross validation. The test results show that SVM achieved the highest accuracy of 72.77%, precision of 75.11%, recall of 65.17%, and macro F1 score of 68.28%. Logistic Regression obtained a macro F1 score of 67.05%, while Naive Bayes obtained 63.69%. SVM also achieved the highest average macro F1 score in cross validation at 60.25%, compared with 59.94% for Logistic Regression and 56.53% for Naive Bayes. These results indicate that SVM provided the strongest overall performance for the dataset examined in this study.