Seokju Cho

Can Learning to Answer Spatial Questions Improve Novel-View Generation?

This work introduces Ask4NV, a pretrained unified multimodal model with a Mixture-of-Transformers (MoT) architecture that incorporates camera geometry into both VAE and ViT representations and leverages understanding features during generation through joint self-attention with a modality-aware attention masking strateg...