MEIR: Memory-Enhanced Incongruity-Aware Reasoning for Multimodal Sarcasm Detection and Explanation
Abstract
Multimodal sarcasm detection has emerged as a critical research direction, and recent multimodal large language models (MLLMs) have opened new possibilities toward deeper affective reasoning in conversational video scenarios. However, existing approaches still suffer from two key challenges: data-level annotation deficiency and architecture-level context entanglement, where the former provides only utterance-level binary labels while neglecting fine-grained reasoning annotations, and the latter entangles contextual history with target delivery, preventing explicit identification of multimodal conflicts. To address these issues, we first introduce Multimodal Understanding and Sarcasm Explanation DataSet (MUSE-Set), a bilingual video dataset with four-dimensional explanatory annotations decomposing sarcasm reasoning into character descriptions, situational contexts, incongruity analysis, and intended meanings. Building on this, we propose Memory-Enhanced Incongruity Reasoner (MEIR), a novel framework for multimodal sarcasm detection and explanation. MEIR employs a cognitively inspired dual-phase architecture: an episodic memory generator first compresses context videos into structured character-tracked memories, which an incongruity reasoner then contrasts with target utterance cues to explicitly identify conflicts and generate fine-grained explanations. To the best of our knowledge, MEIR is the first unified framework to jointly perform multimodal sarcasm detection and natural language explanation generation. Extensive experiments demonstrate that MEIR achieves best performance in both tasks, consistently outperforming strong MLLMs including Gemini-3-Flash and GPT-4o.