Performance Comparison of Hybrid Indobert-Self-Paced Ensemble and Indobert- Rusboost for Imbalanced Text Classification (Case Study: Insecurity Detection on Social Media X)
TL;DR
It is confirmed that hybrid IndoBERT- RUSBoost provides more balanced and robust performance for imbalanced text classification for imbalanced text classification, offering practical implications for early insecurity detection systems on social media platforms.
Abstract
Insecurity detection on social media is crucial for early mental health intervention, yet imbalanced data distribution poses significant challenges for text classification. This study compares the performance of hybrid IndoBERT-Self-Paced Ensemble (SPE) and IndoBERT- RUSBoost for imbalanced text classification in insecurity detection on social media X. A dataset of 5,179 Indonesian tweets was collected using keywords "insecure" and "overthinking", preprocessed through text cleaning and slang normalization, and labeled automatically using a fine-tuned IndoBERT model as insecurity (positive) or non-insecurity (negative). Results demonstrated that RUSBoost significantly outperformed SPE across 8 of 13 evaluation metrics, achieving balanced accuracy of 0.8521, F1-score of 0.6763, and AUC-ROC of 0.9459 compared to SPE's 0.8465, 0.5171, and 0.9242 respectively. RUSBoost showed superior specificity (0.9460 vs 0.8381) and precision (0.6104 vs 0.3706), while SPE excelled in recall (0.8548 vs 0.7581). Gradient Boosting achieved the highest AUC-PR (0.7100) and specificity (0.9622). The findings confirm that hybrid IndoBERT- RUSBoost provides more balanced and robust performance for imbalanced text classification, offering practical implications for early insecurity detection systems on social media platforms.