Sentiment Analysis of Indonesian Political News using IndoBERT

Abstract

Sentiment analysis of political news plays a crucial role in understanding public opinion and political discourse. This study presents a comprehensive evaluation of six deep learning and machine learning architectures for three-class sentiment analysis (Negative/Neutral/Positive) of Indonesian political news, leveraging IndoBERT embeddings as a shared representation backbone. An expanded dataset of 25,322 annotated news articles from three major Indonesian news portals (CNBC Indonesia: 5,343; Detik: 9,999; Kompas: 9,980) was constructed using a structured two-stage annotation protocol. Six model architectures are compared: IndoBERT+GRU, IndoBERT+BiLSTM, IndoBERT+CNN, IndoBERT Fine-Tuned, IndoBERT+XGBoost, and IndoBERT+Random Forest. Class imbalance is addressed through Focal Loss (gamma=2.0). Experimental results validated through Explainable AI analysis (Integrated Gradients and SHAP TreeExplainer), Bootstrap Confidence Intervals, and McNemar statistical significance testing, show that IndoBERT Fine-Tuned achieves the highest Macro F1-Score of 0.8285, followed by IndoBERT+CNN (0.8255) and IndoBERT+GRU (0.8282). McNemar’s test confirms that these three models are statistically equivalent (p > 0.05), while all three significantly outperform the Machine Learning ensemble methods (p < 0.001). Inter-annotator agreement was measured using Fleiss’ Kappa, yielding κ = 0.6539, indicating substantial agreement among annotators.