Skip to content
Open access

Behaviorally-relevant features of observed actions dominate cortical representational geometry in natural vision

Sep 2026 · Nature Communications · Vol 17 · 1 citation · 205 references
Medicine

Abstract

We effortlessly extract behaviorally relevant information from dynamic visual input to understand the actions of others. Here, we compare behavioral judgments of action meaning against other models capturing semantic and visual features. We measured brain activity using fMRI while participants viewed 90 video clips depicting social and nonsocial actions in real-world contexts. Using arrangement tasks, we developed two behavioral models capturing judgments of action meaning (transitive purpose, social content), as well as three control models capturing judgments of visual content (people, objects, scene). We compared these models with semantic models based on word embeddings and visual models based on gaze and motion energy. Behavioral models of action meaning outperformed models of visual content, semantics, and low-level visual features across much of cortex. Transitivity and sociality captured large portions of unique variance throughout the action observation network, extending into regions not typically associated with action perception, such as ventral temporal cortex. Han et al. show that behavioral judgments of action meaning better capture brain activity than models of semantic and visual content across much of cortex, including the action observation network and extending into ventral temporal cortex.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.