Skip to content
Preprint

Ego-Forge: Text and Geometric-Attention Free Exo-to-Egocentric Video Generation

Sep 2026 · 0 citations · 54 references
Computer Science

TL;DR

Ego-Forge is a caption-free and bias-free framework for exo-to-egocentric video generation that achieves state-of-the-art performance on Ego-Exo4D, runs faster end-to-end, requires no external annotation at inference, and generalizes to in-the-wild scenes, including cases where over-reliance on geometry blocks appearance inference.

Abstract

Exo-to-egocentric video generation aims to synthesize what a person sees from their own viewpoint given third-person footage and a target head trajectory. The task requires transferring appearance and semantics across large viewpoint changes while hallucinating content never observed by the exocentric camera. Existing approaches either impose additional input requirements, such as a ground-truth initial egocentric frame or multiple synchronized exocentric views, or remain limited to category-specific settings. EgoX is the first to address cross-activity and in-the-wild generalization, but requires a human-provided caption of the non-existent egocentric view at inference and introduces a computationally expensive geometry-guided attention bias that can propagate reconstruction errors and suppress textual and visual context. We therefore propose \textbf{Ego-Forge}, a caption-free and bias-free framework for exo-to-egocentric generation. It introduces \textit{Dynamic Captioning}, which derives conditioning tokens directly from the model's hidden states and adapts them to the diffusion timestep and network depth, replacing external text conditioning. By scaling training by an order of magnitude and using all available exocentric viewpoints, Ego-Forge learns cross-view correspondence implicitly and eliminates the need for geometry-guided attention, requiring only a lightweight depth prior. Ego-Forge achieves state-of-the-art performance on Ego-Exo4D, runs faster end-to-end, requires no external annotation at inference, and generalizes to in-the-wild scenes, including cases where over-reliance on geometry blocks appearance inference. Our model and source code will be made publicly available.

View source

Similar papers

Preprint Aug 2026

Grounded-Exo2Ego: Structured Semantic Grounding for Robust Exocentric-to-Egocentric Video Generation

Generating egocentric video from a single exocentric video is an emerging and important topic for AR/VR and physical AI. Compared with conventional novel view synthesis, exo-to-ego generation is a significantly harder task because the standard geometric conditioning becomes highly unreliable under extreme view changes...

Shengze Wang, Michael Stengel, Tianye Li et al. · 2 citations
#artificial intelligence Preprint Sep 2026

Exo2EgoHOI: Hand-Object-Interaction Aware Exocentric-to-Egocentric Video Generation

Egocentric videos of human manipulation provide valuable visual experience for embodied intelligence, yet collecting such data at scale is costly. Exocentric-to-egocentric video generation offers a scalable alternative by transforming abundant third-person manipulation videos into first-person observations. However, ex...

Hong-Jia Zhai, Xi-Yu Zhang, Hao-Ran Zhang et al. · 0 citations
Preprint Oct 2026

LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation

Generating an egocentric video from a single exocentric recording is a challenging case of novel view synthesis, as the two cameras share little overlap and much of the target view is unobserved. Current state-of-the-art methods reconstruct the scene explicitly by estimating depth, lifting the video into a point cloud,...

Suhwan Cho, Yonwoo Choi, S. Kim et al. · 0 citations
Preprint Aug 2026

ACE-Ego-Hand: Repurposing Video Diffusion Models for Occlusion-Robust Egocentric 3D Hand Motion Recovery

ACE-Ego-Hand is introduced, an offline clip-level framework that extracts features via a Deterministic Clean-Latent Encoder and decodes them with a Bidirectional Spatiotemporal Decoder, offering a scalable path from everyday human video to robot manipulation data.

Yu-Fei Liu, Xixi Wang, Hao Li et al. · 4 citations · ⚡2
#small language model Preprint Aug 2026

G3Ego: Gaze-Guided Graphs for Egocentric Action Understanding

G3Ego, a graph-based framework for egocentric action understanding that uses gaze as a structural cue to identify action-relevant entities in the scene, achieves competitive performance compared with video-based approaches and consistently improves Macro-F1 under class-imbalanced evaluation, while avoiding reliance on...

Marko Haralović, Akash Ramakrishnan, E. T. Martínez · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.