Lightweight Transformer-based Ai Model for Multilingual and Code-mixed Sentiment Analysis

Abstract

Social media users increasingly communicate across language boundaries, often combining two or more languages within the same sentence and frequently writing Indian languages in Roman script. This creates a difficult setting for sentiment analysis because conventional monolingual models must cope with spelling variation, transliteration, informal vocabulary, emojis, abbreviations, language switching, and context-dependent sentiment. Large multilingual Transformer models have improved contextual understanding, but their computational cost can make deployment difficult on resource-constrained systems. This thesis proposes a lightweight Transformer-based framework for multilingual and code-mixed sentiment analysis. The framework uses a multilingual teacher model, language-aware normalization, and knowledge distillation to transfer contextual knowledge into a smaller student model. The study focuses primarily on Hindi-English (Hinglish) and Spanish-English (Spanglish) social-media text, using the SemEval-2020 Task 9 SentiMix corpora as a reproducible benchmark. The proposed framework is evaluated against classical machine-learning, recurrent neural-network, and full-size Transformer baselines. Evaluation is designed to consider not only accuracy and macro/weighted F1, but also inference latency, parameter count, memory consumption, robustness to noisy and code-mixed text, and the consistency of predictions across language pairs. An explainability component is also included so that important tokens contributing to a prediction can be inspected. The central research question is whether a substantially smaller Transformer can retain useful multilingual sentiment knowledge while reducing computational requirements. The thesis therefore treats sentiment analysis as both a language-understanding problem and an engineering deployment problem. The resulting framework is intended for applications such as social-media monitoring, customer-feedback analysis, multilingual communication systems, and other AI systems where language diversity and limited computing resources must be considered together. Keywords: Transformer, multilingual NLP, code-mixed text, sentiment analysis, knowledge distillation, Hinglish, Spanglish, lightweight AI, explainable AI, social media.