A Unified Perspective on Multimodal Emotion Recognition, Personality Trait Assessment, and Ambivalence/Hesitancy Recognition: Corpora, Methods, Evaluation, and Open Challenges
Oct 2026· Big Data and Cognitive Computing· 0 citations· 126 references
TL;DR
Findings indicate that many methodological trends remain supported primarily by within-benchmark evidence, and the synthesis identifies shared multimodal processing principles while preserving differences in target stability, annotation sources, temporal granularity, and evaluation.
Abstract
Multimodal analysis of human behavior has advanced rapidly, yet emotion recognition (ER), personality trait assessment (PTA), and ambivalence/hesitancy recognition (AHR) are usually reviewed separately. This review jointly examines these related but distinct tasks through a comparative analysis of corpora, computational methods, evaluation protocols, and ethical limitations. Structured searches in Scopus and Web of Science were followed by multi-stage screening and multi-reviewer assessment. The final methodological sample comprised 56 unique studies published between 1 January 2023 and 1 August 2026. One study addressed both ER and PTA, resulting in 57 task-specific entries. The reviewed methods include cross-modal interaction, temporal and contextual modeling, graph-based reasoning, reliability-aware fusion, learning under limited supervision, and semantic enrichment with large language models. The reviewed ER studies address contextual modeling, robustness-related objectives, and open-vocabulary prediction. PTA studies focus on structured behavioral modeling, temporal aggregation, target-aware assessment, and generalization-related strategies. AHR remains less explored and is concentrated on a narrow benchmark landscape with distinct frame- and video-level protocols. A post-inclusion quality and reproducibility assessment showed explicit repeated-run uncertainty reporting in only 10 of 56 studies. Explicit cross-corpus or external validation was reported in only 3 of 56 studies. These findings indicate that many methodological trends remain supported primarily by within-benchmark evidence. The synthesis identifies shared multimodal processing principles while preserving differences in target stability, annotation sources, temporal granularity, and evaluation. Key priorities include cross-corpus generalization, uncertainty, interpretability, cultural validity, privacy, and human oversight.
A reliability-sensitive framework has been proposed, in which recognition is driven by uncertainty management rather than feature accumulation, while preserving the modality-specific nature of different types of evidence.
Yi-Hang Chen· Applied and Computational En...· 0 citations
While Large Language Models (LLMs) demonstrate strong performance in textual emotion detection, their underlying reasoning mechanisms remain unclear. This study investigates the reasoning processes of GPT-5 by conducting a content analysis of its generated justifications for Weibo posts. Using the GPT API, we generated...
Fa Zhang, Qiu-Yue Wu· International Conference on...· 0 citations
Color may influence emotional cognition through perceptual salience, learned affective meaning, and response compatibility, yet task-based evidence has not been synthesized. Reported under the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for Scoping Reviews (PRISMA-ScR) and the...
Emotion regulation is a key construct in mental health, yet existing measures differ in conceptual coverage, with some focusing on specific strategies or regulatory difficulties rather than broader cognitive, behavioral, and contextual regulation processes. This study aimed to develop an Emotion Regulation Scale (ERS)...
I. Almohammadi, M. Alzahrani· BMC Psychology· 0 citations
BACKGROUND
Recent evidence suggests that many autistic individuals exhibit heightened emotional responsiveness to others' affect despite greater difficulty recognizing others' emotions and accurately inferring their perspectives. This study introduces and preliminarily evaluates the Emotion Regulation and Recognition A...
Jeffrey J. Wood, Stian Orm, Karen S. Wood et al.· Journal of Child Psychology...· 0 citations
Facial Emotion Recognition (FER) is a core component of social cognition, supporting theory of mind processes, empathy, and social reasoning. Deficits in FER are associated with functional impairments across multiple domains of typical and atypical development. Measurement instruments with sound psychometric prop...
F. de Lima Osório, Diego Alves Ferreira, Nelson Torro-Alves· Psicologia: Reflexão e Críti...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.