A Systematic Ablation Study of Multilingual Transformer Architectures for Sarcasm Detection: Architectural Design Versus Feature Engineering

Abstract

Sarcasm detection remains a challenging task in natural language processing due to the complex interaction between linguistic and pragmatic cues. While transformer-based models have shown encouraging performance, it is not yet clear whether improvements stem primarily from architectural design or explicit feature engineering. This work introduces a systematic ablation study of three multilingual transformer architectures (mBERT, mDeBERTa-v3, and XLM-RoBERTa) evaluated on HeteroSarc-47K, a curated multi-domain dataset containing 47,694 instances across six heterogeneous domains, including social media and dialectal Arabic text. We isolate the empirical impact of two feature augmentation techniques: emoji integration and contrastive pre-training. Our experiments reveal that while contrastive pre-training increases sarcasm recall for mBERT, it induces a significant drop in its overall accuracy, demonstrating architecture-specific sensitivities. In contrast, architectural upgrades alone consistently yield robust improvements; mDeBERTa-v3 achieves the highest performance (72.79% sarcasm recall and 87.20% accuracy) without requiring any feature augmentations, outperforming the baseline mBERT. Furthermore, per-class analysis reveals a persistent performance gap between non-sarcastic and sarcastic classes across all domains, highlighting that class imbalance remains a critical challenge independent of architecture. Ultimately, these findings provide a vital practical takeaway for practitioners: upgrading to inherently stronger multilingual architectures yields significantly higher and more robust performance returns compared to investing in extensive feature-engineering pipelines.