Improving Sentiment Classification Performance Using Pseudo-Labeling with Naive Bayes and Random Forest

Abstract

Sentiment analysis has been applied to understand users' opinions expressed in unstructured text on various online platforms. Supervised machine learning approaches typically require large amounts of labeled data; however, annotation is expensive and laborious. This research aims to implement semi-supervised learning via pseudo-labeling to reduce reliance on labeled data and improve the performance of sentiment classification models relative to a supervised baseline. Contributions of this research include a pseudo-labeling framework implemented with limited labeled data, benchmarking of Naive Bayes and Random Forest algorithms across three Indonesian-language datasets with different linguistic characteristics (IndoNLU, E-commerce, and BCA Mobile app reviews), and an analysis of model performance sensitivity to the pseudo-labeling threshold. Count Vectorizer and TF-IDF were utilized for feature extraction with unigram and bigram parameters. Accuracy and F1-score were measured to benchmark model performance. Results show that pseudo-labeling improved performance across all classification models compared to the baseline by an average of 0.005–0.010, and up to 0.030 on the E-commerce dataset. Naive Bayes was found to have more consistent results across datasets, while Random Forest capitalized more on the implementation of pseudo-labeling under the right circumstances. Overall, TF-IDF outperformed Count Vectorizer, and larger threshold values yielded more consistent performance improvements across datasets, though lower values offered greater potential for gains on large, diverse datasets. These results suggest pseudo-labeling is a viable method for incorporating unlabeled data. Its efficacy depends on dataset attributes, feature extraction methods, and the threshold value.