Zero-Shot Cross-Lingual Transfer for Hate Speech Detection Using Multilingual Models

Abstract

Detecting hate speech in low-resource and unseen languages remains challenging due to limited labeled data and linguistic diversity. This paper presents a comparative study of zero-shot cross-lingual transfer for hate speech detection using two multilingual transformer models: mDeBERTa-v3 and XLM-RoBERTa. To the best of our knowledge, mDeBERTa-v3 has not been previously used by researchers for zero-shot cross-lingual hate speech detection, making this the first study to evaluate its capabilities in this task. Furthermore, we introduce new unseen languages that have not been studied before in this context, including Hebrew, Amharic, and Swahili, alongside other languages such as Indonesian, Danish Portuguese, Turkish, French, and Russian. We evaluate model performance under three training scenarios: a single source language (Turkish), semantically similar language clusters, and multiple clusters from different language families. Experimental results show that mDeBERTa-v3 consistently outperforms XLM-RoBERTa in zero-shot settings. The most notable improvement is observed for Hebrew, where the macro F1 score increases from 0.39 (XLM-RoBERTa) to 0.71 (mDeBERTa-v3), a gain of 0.32. Substantial gains are also seen for Amharic (0.52 → 0.73, +0.21), Indonesian (0.57 → 0.71, +0.14), and Swahili (0.65 → 0.75, +0.10). Across all experimental conditions, mDeBERTa-v3 achieves average macro F1 gains ranging from 0.04 to 0.19, with statistical significance (p < 0.02). The model’s advantage is attributed to its disentangled attention mechanism, which enables better generalization across typologically distant languages. These findings establish mDeBERTa-v3 as a novel and more robust architecture for zero-shot cross-lingual hate speech detection, particularly for previously unexplored low-resource languages.