Skip to content

H-GIM: a human-centered generative immersive multimedia framework for adaptive immersive multimedia systems

Nov 2026 · Multimedia tools and applications · Vol 85 · 0 citations · 46 references
Virtual Reality Applications and Impacts

TL;DR

A synthetic data generation pipeline that automatically produces high-fidelity 3D assets using procedurally guided generative models and demonstrates the potential of generative and multimodal AI to enable scalable, adaptive, and human-centered immersive multimedia systems.

Abstract

Immersive multimedia systems based on Extended Reality (XR) are increasingly adopted in education, entertainment, and collaborative environments. However, traditional multimedia production pipelines rely heavily on manual content creation and static interaction models, limiting scalability and real-time adaptation to user behavior. This paper proposes H-GIM (Human-Centered Generative Immersive Multimedia), a framework powered by generative and multimodal artificial intelligence. The proposed architecture integrates multimodal foundation models capable of processing textual, audio, visual, and spatial signals through a router-based orchestration layer. To address the lack of large-scale XR training datasets, we introduce a synthetic data generation pipeline that automatically produces high-fidelity 3D assets using procedurally guided generative models. Importantly, 3-D asset generation is hybrid: geometry is produced via procedural seeds with physics validation, while SDXL-Turbo is used only for texture maps, ensuring physical plausibility without claiming end-to-end diffusion-based mesh generation. Furthermore, an affective interaction module dynamically adapts multimedia complexity based on user cognitive load and behavioral signals. Experimental evaluation on a functional prototype demonstrates that H-GIM achieves an average asset-generation time of 1.1 ± 0.2 min, a mean interaction latency of 42 ms, and a user-presence score of 0.81 ± 0.04. Results from a user study (n = 50) further indicate statistically significant improvements in immersive experience compared with conventional XR workflows (p < 0.001). These results demonstrate the potential of generative and multimodal AI to enable scalable, adaptive, and human-centered immersive multimedia systems.

Read PDF

Similar papers

Conference Open access Oct 2026

Research on cross-media intelligent image generation models for immersive interactive experiences

Immersive interaction technologies and cross-media content generation demands continue to grow and grow in the wake of booming optical display technologies and immersive visual presentation systems. The traditional image generation approaches have limitations in semantic consistency and interactive responsiveness for h...

Hua-Xin Ye, Lin Dong · 0 citations
#artificial intelligence Open access Oct 2026

SPHERE: Adaptive VR Indoor Scene Generation via LLM-Enhanced Spatial Preference Learning and Human-in-the-Loop RL.

SPHERE demonstrates how capturing demonstrated spatial logic enables controlled spatial adaptation, establishing a reliable, governed human-AI collaboration framework for immersive authoring.

Hyeonmin Lee, Zheng Wei, Kyungmin Kwon et al. · 0 citations
Open access Oct 2026

Hear What You See: Depth-Aware Spatial Audio Rendering for Immersive Panoramic Video.

Panoramic video has emerged as a core medium for virtual reality (VR) and immersive media, yet its audio typically remains stereo, lacking spatially coherent soundfield reproduction consistent with the visual content and severely compromising immersion. Although multi-loudspeaker spatial audio systems can reconstruct r...

Di Zhang, Zi-Shan Peng, Yi-Nuo Du et al. · 0 citations
Book Open access Oct 2026

Human-Centered Digital Twins: A Bidirectional WebXR-Based Architecture for Immersive Interaction

Digital twin technology has emerged as a profound paradigm for bridging physical systems and their virtual counterparts through real-time data synchronization and interactive three-dimensional visualization. However, existing digital twin platforms are predominantly domain-specific, proprietary, and reliant on dedicate...

Hayette Hadjar, Zineb Hadjadj, A. Meziane · 0 citations
Open access Oct 2026

The Impact of Point-Cloud Rendering on User Perceptions of Human- and AI-Controlled Avatar Embodiment.

Avatars for social VR are increasingly driven by imperfect tracking or AI-generated motion, where mismatches between appearance fidelity and behavioral realism can harm user perception. We investigate whether a simulated point-cloud rendering (PCR) can improve social perception and increase tolerance to motion artifact...

Tian-Yu Song, Baran Gursoy, Felix Pabst et al. · 0 citations

Related blog posts

Google DeepMind Blog Nov 18, 2026

Genie 4 is now available

We are rolling out a new Project Genie experience to eligible Google AI Ultra subscribers, opening access to our new Genie 4 world model.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.