A Transformer-Based Framework for Offensive Language Detection in Kannada-English Code-Mixed Text

Abstract

This paper explores the usefulness of multilingual transformer-based models in the detection of offensive language on Kannada-English code-mixed social media text. This work compares CNNTFIDF, XLMR, IndicBERT, and DistilBERT models, and suggests a better DistilBERT-related model using specific preprocessing and fine-tuning to low-resource code-mixed text. The experiments carried out using a trained Kannada-English dataset show that the proposed DistilBERT model has an F1-score of 0.846 and an accuracy of 85.93, which is better than heavier multilingual transformers and is computationally efficient. The findings point to the appropriateness of lightweight transformer architectures in detecting offensive language using low-resource and code-mixed languages. The work offers empirical information and baselines to be used in the creation of scalable and inclusive multilingual online community content moderation systems.