This systematic literature review brings together the findings of multimodal artificial intelligence (AI) in detecting psychological and affective states from nonverbal cues and examines whether it is ready for mental-health and counseling applications.
Abstract
The psychological assessment is limited to only relying on the patients' self-reports, as they may not even understand or have the desire or capacity to express the distress they show with their body language such as facial movements, voice, gaze, posture, and physiology. This systematic literature review brings together the findings of multimodal artificial intelligence (AI), mainly the use of such a system in detecting psychological and affective states from nonverbal cues and examines whether it is ready for mental-health and counseling applications. Using PRISMA 2020, we identified the records from a Scopus search, then screened them by the predefined eligibility criteria and evaluated them with the tool for Mixed-Method Appraisal. The original list of 669 articles was reduced first by removing the duplicates to 665 screened articles, and among them, we selected 20 articles released in 2024, 2025 that fulfilled the criteria. The themes identified in the five modality Channels facial & micro expression, speech & acoustic, text & linguistic, physiologic, and body movement together with their amalgamation were compared against the target states which were mainly depression, general emotion, and concealed affect. Multimodal approach generally yielded better results than unimodal methods and recognized concealed distress and suicidality, but the research is limited in scope, data sets being very small, external validation is missing and ethical precautions are not even considered yet.
Emotional expressions conveyed through body postures and movements play a central role in social communication and adaptive behaviour, yet they have received considerably less attention than facial expressions. This systematic review examines how emotional bodily expressions are induced, measured, and analyzed. Forty-n...
L. Rogez, Ali Oker, Stéphanie Caillies· Cognition & Emotion· 0 citations
A reliability-sensitive framework has been proposed, in which recognition is driven by uncertainty management rather than feature accumulation, while preserving the modality-specific nature of different types of evidence.
Yi-Hang Chen· Applied and Computational En...· 0 citations
Conversational AI may offer useful, supervised scaffolding for engagement and communication, but the evidence is heterogeneous, largely short term, and insufficient to support AI as an autonomous emotional caregiver or a substitute friend.
A. Bhatt, S. Nimbalkar, A. Nimbalkar· European Journal of Pediatri...· 0 citations
Not revealing psychologically distressing thoughts, symptoms, and experiences, that is, psychological non-disclosure, contributes to the failure of detection and treatment in many cases of mental illness. Since diagnostic methods mostly depend on self-reporting, such states often remain unrecognized. This article is a...
I. Ifdil, Nur Adila Wafiqah Zulvi, M. F. Zatrahadi· JRTI (Jurnal Riset Tindakan...· 0 citations
People with motor disabilities often display atypical emotional expressions that elude conventional recognition models, leading to communication difficulties, isolation, and restricted access to emotional support. Multimodal AI affective recognition systems offer promising avenues to overcome these barriers, but existi...
Maram Djebbi, Kirmene Marzouki, Fernando Marmolejo-Ramos et al.· Companion Publication of the...· 0 citations
Affective computing has become increasingly relevant in child–robot interaction (CRI), particularly in social robotics, emotion recognition, engagement assessment, and autism-related interventions. This systematic review with a critical and integrative synthesis analyzes 105 included studies to examine how affect is se...
Sandra Cano, J. Vásconez, Kiara Villarroel et al.· Italian National Conference...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.