Translation-Aware Multilingual Sentiment Analysis: A Comparative Study of Machine Translation Systems and Transformer Ensembles

Abstract

Multilingual sentiment analysis remains challenging because of cross-linguistic differences, uneven data availability, and potential shifts in meaning introduced during translation. Rather than proposing a new model, this study examines the performance of translation-based pipelines and ensemble Transformer models in multilingual settings. The primary objective is to investigate differences in downstream sentiment classification performance between translation systems within a consistent evaluation framework. Texts in Arabic, Chinese, French, and Italian are standardized through label harmonization and basic preprocessing and subsequently translated into English using Google Translate and the open-source LibreTranslate. Sentiment classification is performed using three Transformer-based models: DeBERTa-v3, RoBERTa, and BERTweet. The experiments are formulated as a binary polarity-classification task comprising positive and negative instances, with neutral samples excluded because their annotations are not directly comparable across the heterogeneous source datasets. Predictions from the individual models are combined through majority voting without introducing a new ensemble method. Experiments on the resulting binary datasets demonstrate that the ensemble achieves the highest mean weighted F1-score, with low fold-to-fold variation under both translation pipelines. Google Translate yields higher and more consistent classification performance across the evaluated subsets, whereas LibreTranslate exhibits greater variation. Given that the datasets differ in domain and genre, these variations cannot be attributed solely to linguistic differences. Overall, the findings reveal differences in downstream sentiment classification performance between the two translation pipelines and indicate that the ensemble combines high mean performance with low fold-to-fold variation within the evaluated binary setting.