Initial evidence is provided that RMET can be studied as a process-aware assessment task in controlled VR and desktop settings and that VR received higher usability ratings and was also rated as more effortful.
Abstract
Social intelligence, the ability to interpret others'emotions, beliefs, and intentions, is often assessed with the Reading the Mind in the Eyes Test (RMET), in which participants infer mental states from images of the eye region. Yet RMET is typically presented on paper or desktop displays, where viewing geometry can vary across participants, and it rarely includes immediate feedback. We investigated whether presentation medium and brief trial-level feedback influence RMET behavior. We implemented RMET in Unity for both desktop and Virtual Reality (VR), using VR to hold stimulus distance and field of view constant without changing the items. We conducted a 2x2 mixed study with 20 participants, with device (VR vs. desktop) manipulated between subjects and feedback (immediate correctness cue vs. none) manipulated within subjects. Eye-tracking and EEG data were recorded and synchronized with behavioral logs. We analyzed fixation-based gaze measures, EEG signals, response time, accuracy, and subjective measures. Immediate feedback was associated with longer fixation durations and higher EEG-based engagement, while no significant differences were observed in task completion time or total correct answers. Presentation medium did not produce reliable differences in the objective measures, but VR received higher usability ratings and was also rated as more effortful. These results provide initial evidence that RMET can be studied as a process-aware assessment task in controlled VR and desktop settings.
Virtual reality (VR) interaction remains largely dependent on explicit motor input, limiting seamless and adaptive interaction. This study investigated whether electroencephalography (EEG)-based passive brain–computer interfaces (BCIs), combined with eye gaze, can decode interaction intent directly from its underlying neurophysiological correlates during dynamic VR gameplay. We operationalized interaction intent as comprising two components: affordance-related evaluation, indicating whether an attended object affords interaction, and approach–avoidance evaluation, indicating the directional tendency of interaction toward desirable or undesirable outcomes. Twenty-three participants completed a VR game with two calibration sessions and one online BCI session. Offline analyses showed above-chance decoding of the binary approach–avoidance decision classification across all actionable trials, with a grand-average accuracy of 66.28% across participants. This decoding transferred to online closed-loop gameplay, where grand-average accuracy remained above chance at 69.64%. Category-level analyses further revealed substantial variability in classification separability. For approach–avoidance-related classifications, accuracy reached 80.84% for the most distinct pairing between clearly valenced reward and punishment categories, but dropped to near chance at 59.03% for the more context-dependent pairing with ambiguous motivational valence. Affordance-related classifications between non-actionable and actionable item categories were consistently high, ranging from 77.76% to 83.50%. User Experience questionnaire results showed that, despite limitations leading to perceived loss of control and reduced ease of use, participants found the BCI-based interaction paradigm itself more fun than the controller baseline. To our knowledge, this is the first demonstration of real-time EEG decoding of interaction intent during dynamic VR gameplay, contributing toward intuitive user-adapted interfaces driven by physiological signals in immersive environments.
Yanzhao Pan, Lea Rabe, Thorsten O. Zander et al.· bioRxiv· 0 citations
The present study examined whether changing presentation context can restore sustained attention during monotonous tasks. Participants (N = 164) completed a sustained attention task (i.e., the visual metronome response task) with stimuli presented either on a computer monitor or through a virtual reality (VR) headset. Across two task blocks some participants switched devices (monitor to VR, or VR to monitor), while others used the same device throughout (monitor to monitor, or VR to VR). We observed a sustained attention decrement characterized by increasing response time variability and mind-wandering over time. Participants who switched presentation contexts experienced reductions in mind-wandering compared to those who maintained the same device, demonstrating contextual changes can effectively restore subjective attentional engagement. The type of device switched to (and from) had minimal impact, suggesting the benefit stems from the contextual change rather than the specific presentation mode. Contrary to expectations, overall performance was poorer in VR than on the monitor. However, the effects of switching on task performance were not significant. The results suggest changes in context surrounding a persistent task can be beneficial by temporarily reducing mind-wandering.
Maxime LeDrew, A. Safati, Daniel Smilek· Virtual Reality· 0 citations
It is concluded that combining eye tracking and Thinking Aloud is feasible without compromising data quality and meaningful for comprehensive HCI evaluation.
Tina Frenzel, Inka Schmitz, Lisa Kerwien et al.· Proceedings of Mensch und Co...· 0 citations
Perception is a continuous, active, and dynamic process of inference shaped by current goals, past experience, and hard biological constraints. The physiological state of the body is arguably the most fundamental one. Continuous inference about interoceptive states colors perception with emotions, which in turn affect basic perceptual and attentional processes. How, then, do we really look at the world when experiencing malaise? Here we leverage Galvanic Vestibular Stimulation (GVS) – a non-invasive brain stimulation technique that induces mild dizziness, motion sickness, and nausea – in conjunction with high-resolution eye tracking, pupillometry, and computational modelling. We demonstrate that mild vestibular perturbations disrupt the coupling between gaze and informative low-level visual features. Crucially, however, this effect depends on individual susceptibility and on stimulus valence. Particularly susceptible participants experiencing stronger nausea increase their reliance on visually salient features, and particularly so for threatening (negatively valenced) images. Pupillometry reveals larger tonic pupil size in these participants, consistent with heightened arousal, as well as disrupted phasic responses to affective pictures. Altogether, these results unveil the continuous and dynamic interplay between interoceptive and exteroceptive processes, showing that how we perceive the environment fundamentally resonates with how we feel. Significance statement Feeling unwell – whether from illness or motion – is a universal experience. Bodily signals profoundly shape cognition, emotion, and decision-making. We therefore asked how nausea changes visual exploration of the environment. We electrically stimulated the vestibular system of healthy volunteers to induce dizziness and nausea, while monitoring gaze during free viewing of affective images. The stronger the nausea, the more visual exploration was guided by objectively salient visual features, especially for highly arousing images. Pupillometry revealed larger pupils, consistent with heightened arousal, and disrupted pupillary responses to affective pictures, showing less room for affective processing and increased cognitive control. We conclude that nausea may dull perception, while strengthening its grip on physical reality. Highlights GVS caused variable degrees of dizziness and nausea GVS led to spatial biases in visual exploration which were, however, further dependent on image valence and content During GVS eye fixations landed, on average, at locations that were less salient based on their mere physical properties, provided participants did not experience strong interoceptive perturbations Highly susceptible participants showed the opposite pattern: their fixations landed on visually more salient and informative locations, especially for negative images Participants who felt stronger effects, including nausea, also showed the largest increase in tonic pupil size; concurrently, the signature phasic dilation associated with negative emotions was disrupted Oculomotor behavior and perception adjust dynamically to ongoing interoceptive processes; GVS can be an effective tool to study this dialogue, provided that inter-individual differences are accounted for
Irene Petrizzo, F. Hadj-Bouziane, E. Blini· bioRxiv· 0 citations
While 360° videos have shown promise for enhancing perceptual-cognitive skills in sport, the mechanisms underlying its advantages over traditional video-based training remain unclear. This study tested whether 360° videos promote different visual search strategies compared to traditional fixed screens (FS), and whether these strategies vary based on expertise level. Thirty-six participants (17 highly-trained basketball players, 19 novices) viewed the same 25 basketball video clips in 360° and FS modes while their eye and head movements were recorded by an eye tracker embedded in a head-mounted display. Gaze events were categorized (fixation, saccade, smooth pursuit, visual scanning) and compared using two-way ANOVAs with viewing modality and expertise as factors. Compared to FS, 360° videos elicited more smooth pursuit and visual scanning behaviors (p < 0.001), alongside fewer and shorter visual fixations and saccades (p < 0.001). No main effect of expertise was observed, except novices exhibited larger saccade amplitudes than highly-trained players in the 360° videos modality (p < 0.008). These results indicate that the FS modality promoted fixation-dominant behavior likely driven by the constrained visual environment, whereas the 360° modality afforded a more fluid search strategy, potentially reflecting real-world visual behavior and supporting its higher ecological validity for perceptual-cognitive training.
Maxime Trempe, E. Charbonneau, Virginie Lévesque et al.· Jurnal sport science· 0 citations
The classification and validity scoring schemes presented in this study contribute to the theory-driven application of eye tracking in IVR, and offer guidance for improving measurement validity in educational psychology.
J. Y. Lee, Jewoong Moon, Fred Paas et al.· Educational Psychology Revie...· 0 citations