Evaluating the Performance of the No Language Left Behind Encoder for Sentiment Classification Across Multiple Indonesian Regional Languages
Abstract
This study aims to evaluate the performance of the encoder from the No Language Left Behind (NLLB) model for sentiment classification tasks across several Indonesian regional languages. Originally developed for machine translation, the NLLB model is explored for its ability to generate contextual text representations that are relevant to sentiment classification. The dataset used in this study is NusaX, which comprises 12 languages, including Indonesian, English, and 10 Indonesian regional languages. Two training approaches were employed: fine-tuning, in which all model parameters were updated using sentiment classification data, and partial fine-tuning, in which only the upper layers were updated while the embedding layer was frozen to preserve the original lexical representations. Training was conducted using the AdamW optimization algorithm with CrossEntropyLoss as the loss function and mean pooling as the feature aggregation mechanism. Model performance was evaluated using accuracy and macro F1-score metrics at both the multilingual and per-language levels. The results indicate that the two approaches achieved comparable performance, with an accuracy of 81% and a macro F1-score of 80% on the multilingual dataset. Per-language analysis further revealed that the model performed better on languages that had been included in the original NLLB pretraining data, such as Acehnese, Balinese, Banjarese, Minangkabau, Javanese, and Sundanese, achieving accuracy scores ranging from 79% to 86%. In contrast, several languages that were not covered during NLLB pretraining, including Ngaju, Madurese, and Toba Batak, exhibited slightly lower performance, with accuracy scores ranging from 70% to 78%. These findings demonstrate that the NLLB encoder possesses strong adaptation capabilities for text classification tasks, even in the context of low-resource regional languages.