Skip to content

Human-Inspired Social Engagement Analysis via Interpretable Mutual Visual Attention

Aug 2026 · arXiv.org · Vol abs/2608.24580 · 0 citations · 38 references
Computer Science

TL;DR

This work proposes an interpretable computational model of social engagement inspired by psychological theories of mutual visual attention that combines state-of-the-art head orientation estimation with lightweight geometric reasoning, producing explanations that remain accessible to non-technical users.

Abstract

Understanding social interactions from non-verbal visual data is important for behavior analysis and activity monitoring. We propose an interpretable computational model of social engagement inspired by psychological theories of mutual visual attention. Rather than learning interaction patterns end-to-end, our framework explicitly models dyadic visual attention and aggregates these cues into interpretable measures of individual and group engagement. The resulting modular framework combines state-of-the-art head orientation estimation with lightweight geometric reasoning, producing explanations that remain accessible to non-technical users. We evaluate the proposed approach on a variety of data through quantitative experiments and demonstrate its practical usefulness with qualitative visualizations designed to support teachers, caregivers, and social workers in understanding group interaction dynamics.

View source

Similar papers

#graph neural networks Book Open access Oct 2026

Interpretable Multimodal Engagement Prediction with Graph-based Mixture-of-Experts

A graph-based Mixture-of-Experts (MoE) framework that explicitly separates self and social influences to enable interpretable engagement modeling, which outperforms baselines by up to 64.1% while offering interpretable insights into engagement dynamics across language and gender groups.

Monisha Singh, A. Dhall · 0 citations
Book Open access Oct 2026

Multimodal Subjective Intention Modeling in Social Interactions

Understanding human intentions in complex social interactions is essential for developing socially intelligent systems, yet remains challenging due to their subjective, context-dependent, and multi-interpretable nature. Existing approaches typically reduce intention estimation to single-label outcome based prediction t...

Arthur Mercier · 0 citations
Book Oct 2026

Joint Attention Modeling in Naturalistic Interactions with Social Gaze and Proximity Cues

Joint attention (JA) is a core mechanism of social interaction, supporting coordination, knowledge transmission and early language development; deficits in JA are also a key early indicator of autism. While prior computer vision work often operationalizes JA as frame-based co-attention to a shared target, cognitive sci...

M. Dickerman, Michael Villamizar, Sabine Stoll et al. · 0 citations
Book Open access Oct 2026

Self-Supervised Representation Learning for Heterogeneous Behavioral Expressions of Social Engagement

A self-supervised framework that learns visual representations of interaction dynamics directly from paired videos of two people interacting, suggesting that diverse behavioral markers of engagement share a common foundation in interaction dynamics, and that this foundation can be learned without engagement-specific su...

Naga Venkata Sai Raviteja Chappa, L. Yankowitz, Gokul M. Nair et al. · 0 citations
Open access Sep 2026

Behaviorally-relevant features of observed actions dominate cortical representational geometry in natural vision

We effortlessly extract behaviorally relevant information from dynamic visual input to understand the actions of others. Here, we compare behavioral judgments of action meaning against other models capturing semantic and visual features. We measured brain activity using fMRI while participants viewed 90 video clips dep...

Jane Han, Vassiki S. Chauhan, Rebecca Philip et al. · 1 citation
Review Sep 2026

The social side of social attention: New perspectives and implications.

Visual attention is highly sensitive to spatial cues provided by others, with eye-gaze direction playing a primary role in shaping how we orient attention within different environments. While gaze-mediated attentional orienting can often be considered automatic, as it reflects our spontaneous tendency to look where oth...

Mario Dalmaso, Anna Lorenzoni · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.