Evaluating GPT-4o-based Data Augmentation for Imbalanced Multiclass Sentiment Classification of GoPay Reviews Using IndoBERT-LoRA

Abstract

This study evaluated GPT-4o-based data augmentation for imbalanced multiclass sentiment classification of GoPay user reviews using IndoBERT-LoRA. The main problem addressed in this study was the limited representation of minority sentiment classes, particularly the neutral class, which could reduce the model’s ability to recognize all sentiment categories proportionally. The dataset consisted of 16,955 Google Play Store reviews that were manually labeled into positive, neutral, and negative classes. Two augmentation strategies were compared, namely prompt-based augmentation and fine-tuning augmentation. The generated synthetic data were evaluated using novelty, diversity, duplication, and manual validation of sampled reviews before being incorporated into the training data. The IndoBERT-LoRA model was trained under four scenarios: baseline, class weighting, prompt-based augmentation, and fine-tuning augmentation. The results showed that fine-tuning produced better lexical-level quality indicators, as indicated by more stable novelty and diversity scores and a lower duplication rate. Both augmentation strategies improved macro recall and macro F1-score compared with the baseline and class weighting scenario. The largest improvement occurred in the neutral class, where recall increased from 0.5633 to 0.7801 with prompt-based augmentation and to 0.8133 with fine-tuning augmentation. These findings indicate that GPT-4o-based augmentation improved minority-class recognition, although the improvement involved a trade-off between recall, precision, and implementation cost.