Comparison of Machine Learning Algorithms for Sentiment Analysis of Trans Jogja on Social Media

Abstract

The social media platform X has become an important channel for the public to express opinions and share experiences regarding public services, including Trans Jogja. User-generated content from this platform provides valuable insights into public perceptions of service quality. However, because these data consist of unstructured text, sentiment classification techniques based on machine learning are required to analyze them effectively. This study aims to compare the performance of several machine learning algorithms for sentiment classification, including Naïve Bayes, Support Vector Machine (SVM), Random Forest, Neural Network, Logistic Regression, and Decision Tree, in classifying user sentiment toward Trans Jogja on the X platform. Data were collected through a web crawling process using Tweet Harvest with keywords related to Trans Jogja, covering the period from January 1, 2025, to June 10, 2026, resulting in a dataset of 3,035 tweets. The preprocessing stage included data cleaning, case folding, tokenization, normalization, stopword removal, and stemming. Text representation was performed using the Term Frequency–Inverse Document Frequency (TF-IDF) method. The dataset was then divided into training and testing sets using five train–test split ratios: 90:10, 85:15, 80:20, 75:25, and 70:30. Model performance was evaluated using a confusion matrix and the corresponding accuracy, precision, recall, and F1-score metrics. The experimental results demonstrate that the Support Vector Machine (SVM) consistently outperformed the other algorithms across different data split ratios. At the 85:15 train–test split, the SVM achieved an accuracy of 91%, precision of 91%, recall of 91%, and an F1-score of 91%, indicating that it is the most effective algorithm for sentiment classification of Trans Jogja users on the X platform.