This work proposes LongEmo, a novel memory-augmented agentic framework designed to tackle the immense challenges of long-range affective reasoning, and achieves state-of-the-art performance, demonstrating the efficacy of its event-centric memory architecture.
Abstract
While recent Multimodal Large Language Models (MLLMs) have shown promise in affective computing, their reasoning capabilities are largely confined to short video clips with limited interactions. However, real-world emotions are not merely isolated instantaneous reactions but dynamic and cumulative processes deeply shaped by past experiences and ongoing events. To bridge this gap, we introduce LongEmoBench, a benchmark dedicated to emotion understanding and reasoning in long videos. It assesses progressive capabilities scaling from continuous scene interactions to complex episodic developments. Furthermore, we propose LongEmo, a novel memory-augmented agentic framework designed to tackle the immense challenges of long-range affective reasoning. LongEmo processes continuous video streams to construct an Event Memory Graph, explicitly modeling long-range dependencies and capturing emotional dynamics across discrete events. Given a question, the agent retrieves a query-relevant event stream from the graph, iteratively integrating multimodal memories and relational dependencies to deduce the final answer. Extensive evaluations of 17 representative methods reveal that they struggle significantly with emotion understanding and reasoning in long videos. In contrast, LongEmo achieves state-of-the-art performance, demonstrating the efficacy of its event-centric memory architecture.
Multimodal Empathetic Response Generation (MERG) integrates textual, visual, and acoustic signals to synthesize nuanced affective responses. While Large Language Models (LLMs) demonstrate strong capabilities in emotional reasoning, they still suffer from reasoning redundancy and contextual prolixity. Applying static, h...
Xinzehua Li, Cheng-Ji Wang, Wang-Da Zhu et al.· Proceedings of the 34th ACM...· 1 citation
Affective computing has progressed from categorical emotion recognition to open-ended affective analysis with large multimodal models. Yet affective science describes emotion as an unfolding process shaped by appraisal, regulation, and social interpretation, which remains underexplored computationally. We propose TRACE...
Recent multimodal large language models (MLLMs) increasingly incorporate explainable reasoning for emotion understanding. However, reasoning based mainly on observable affective cues can reduce emotion understanding to superficial cue-label associations, giving rise to the Clever Hans effect. Such shortcuts become unre...
Jia Li, Yi-Chao He, Yang-Chen Yu et al.· 0 citations
Implicit emotion understanding addresses a fundamental yet underexplored challenge: observable expressions do not always faithfully reflect underlying emotional states, requiring models to infer latent emotions from limited multimodal signals. Existing approaches largely assume direct correspondence between overt cues...
Ke-Han Wang, Huan Zhao, Guang-Hui Ye et al.· Proceedings of the 34th ACM...· 0 citations
Multimodal sarcasm detection has emerged as a critical research direction, and recent multimodal large language models (MLLMs) have opened new possibilities toward deeper affective reasoning in conversational video scenarios. However, existing approaches still suffer from two key challenges: data-level annotation defic...
Guo-Jian Li, Fei Ma, Hong-Bo Xu et al.· Proceedings of the 34th ACM...· 0 citations
Large Language Models (LLMs) increasingly operate in multi-turn interactions that require maintaining latent conversational states, not only explicit facts. We study affective state as an implicit state inferred from sparse contextual cues that remain relevant after topic shifts, and ask whether LLMs expose internal at...
Wen Tang, Zhen Wan, Yi Wang et al.· Volume 1· 0 citations
Known for his clear and elegant writing style, Bertsekas shaped fields from control and optimization to large-scale computation and artificial intelligence.