Skip to content

Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flow

May 2026 · arXiv.org · Vol abs/2605.21980 · 0 citations
Computer Science

TL;DR

This work constructs a specialized dataset to demystify the emotional circuits underlying the three-stage ``Adapt-Aggregate-Execute''mechanism and discovers a functional decoupling: visual emotional cues are aggregated in middle layers via sentiment-specific attention heads, but are subsequently translated into narrative generation in deep layers through emotion-general pathways.

Abstract

Large Vision-Language Models (LVLMs) represent a significant leap towards empathetic agents, demonstrating remarkable capabilities in emotion understanding. However, the internal mechanisms governing how LVLMs translate abstract visual stimuli into coherent emotional narratives remain largely unexplored, primarily due to the scarcity of visual counterfactuals and the diffuse nature of emotional expression. In this paper, we bridge this gap by introducing a steering-vector-based causal attribution framework tailored for descriptive emotional reasoning. To this end, we construct a specialized dataset to demystify the emotional circuits underlying the three-stage ``Adapt-Aggregate-Execute''mechanism. Crucially, we discover a functional decoupling: visual emotional cues are aggregated in middle layers via sentiment-specific attention heads, but are subsequently translated into narrative generation in deep layers through emotion-general pathways. Guided by these insights, we regulate the emotional information routing to strengthen attention flow and amplify the semantic activation to consolidate expression. Extensive experiments on the comprehensive MER-UniBench demonstrate that our methods significantly improve performance via inference-time intervention, effectively mitigating emotional hallucinations and corroborating the causal fidelity of the discovered circuits.

View source

Similar papers

#artificial intelligence Preprint Oct 2026

From Surface to Depth: Towards Cognitive Appraisal Reasoning in Multimodal Emotion Understanding

Recent multimodal large language models (MLLMs) increasingly incorporate explainable reasoning for emotion understanding. However, reasoning based mainly on observable affective cues can reduce emotion understanding to superficial cue-label associations, giving rise to the Clever Hans effect. Such shortcuts become unre...

Jia Li, Yi-Chao He, Yang-Chen Yu et al. · 0 citations
Sep 2026

Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning

A Hierarchical Emotional Reasoning Chain (HERC) is proposed that enhances the insufficient attribution by harmonizing fine-grained scene/object-level cues and performing a soft-gated reasoning, and a Contrastive Discriminative Visual Pruning (CDVP), which isolates discriminative visual tokens to reason the final emotio...

C. Ye, Wei-Dong Chen, Zhao-Bo Qi et al. · 0 citations
Open access Sep 2026

CogSent: Cognition-Driven Multimodal Sentiment Analysis Through Fast–Slow Thinking

This work proposes an image-text multimodal sentiment analysis framework (CogSent) grounded in the psychological Dual-System Theory and introduces an Intuition-Guided Cognition Refinement module that uses System-I-inspired intuitive representations as affective priors to retrieve and refine System-II-inspired contextua...

Guo-Guo Ye, Qi-Qi Chen, Li-Qi Yan et al. · 0 citations
Conference Aug 2026

Fusing Semantics and LLM-Driven Emotions by Gated Attention for Implicit Hate Speech Detection

Detecting implicit hate speech continues to be a significant challenge for natural language processing systems, as harmful intent is often concealed within polite, neutral, or coded language. Traditional text-based classifiers often struggle to identify these subtle forms of hostility because they lack the emotional co...

Yuan-Chih Yu · 0 citations

LongEmo: Towards Emotion Understanding and Reasoning in Long Videos

This work proposes LongEmo, a novel memory-augmented agentic framework designed to tackle the immense challenges of long-range affective reasoning, and achieves state-of-the-art performance, demonstrating the efficacy of its event-centric memory architecture.

Shuo Zhang, Yifan Zhou, Han-Yu Wang et al. · 0 citations
Preprint Aug 2026

LG-GER: Language-Guided Group Emotion Recognition via Multimodal Evidence Distillation

LG-GER is proposed, a language-guided distillation framework that uses a multimodal large language model (MLLM) to generate dense, spatially grounded evidence that achieves competitive or superior results compared to state-of-the-art methods that require detection and multi-stream processing at inference.

Ahmed-Shehab Khan, Zhiyuan Li, Yan Tong · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.