Manglish Code-Mixed Sentiment Analysis Using BERT-Base Algorithm

Abstract

In recent years, code-mixed texts have become a common thing in social media platforms. Especially in multilingual societies such as Malaysia, users frequently code-switch between Malay and English (referred to as Manglish) in their online discussions. Sentiment analysis of these kinds of contents usually pose significant challenges due to its known linguistic complexity, the lack of standardized grammar and a limited number of annotated datasets. This research aims to analyze the effectiveness of the BERT-base algorithm for multilingual and code-mixed sentiment analysis, develop a suitable prototype model and evaluate its performance using standard metrics. A dataset consisting of 32,256 YouTube comments focusing on entertainment was collected, preprocessed and annotated for multi-class sentiment classification (positive, neutral, negative). The proposed BERT-base model was further evaluated using three hold-out ratios (70:30, 80:20, and 90:10). The split 90:10 has achieved the best performance of 97.22% accuracy, a comparable training loss of (0.1131) and minimal overfitting between training and test loss. In this split, the model has attained a macro-average precision of 0.97, recall of 0.97, and F1-score of 0.97. The results demonstrated that BERT-base has effectively captured contextual relationships in code-mixed Manglish texts. However, the limitations known are the small size of dataset used due to time constraints and the difficulty in detecting sarcasm. Future works should explore larger datasets and fine-tuning larger transformer variants. This research contributes to the development of a more inclusive sentiment analysis, especially for the low-resource linguistic communities.