Aug 2026· International Journal of Computer Vision· Vol 134· 0 citations· 54 references
TL;DR
This work proposes a Scale-wise Conditional Decoding mechanism, which projects semantic signals from a frozen vision encoder into scale-specific layers of the VAR backbone, and introduces a Unified Control Alignment strategy (SCALAR-Uni) to handle diverse control modalities within a single projection space.
Modern computer vision pipelines remain fragmented, with tasks such as text-to-image generation, editing, restoration, and classical perception handled by separate models. We study Unified Visual Generation (UVG), where a single model produces diverse image-valued outputs through a unified multimodal interface. While diffusion-based systems dominate UVG due to strong quality and controllability, their iterative sampling incurs substantial inference latency, limiting practical deployment. To address these limitations, we propose UniGen-AR, a framework that pairs a general-purpose multi-modal language model (MLLM) with an efficient next-scale visual auto-regressive (VAR) decoder. This design retains the flexibility of MLLM-based conditioning while leveraging the sampling efficiency and latent unification properties of VAR models. In our framework, the MLLM encodes free-form instructions and control signals into a unified sequence, which guides the VAR decoder to generate image-valued outputs for over 15 tasks spanning four families. Empirically, UniGen-AR achieves up to $19 \times$ lower inference latency than diffusion-based baselines while maintaining or improving output quality. Our ablations further reveal that VQ-VAE tokenizer design, particularly codebook size and hierarchy, is a critical factor for VAR scalability in UVG. These results establish visual auto-regressive modeling as a compelling and efficient backbone for unified visual generation. Our project page is at https://zpbao.github.io/projects/unigenar.
Zhipeng Bao, Zhen Zhu, Nupur Kumari et al.· 0 citations
. Abstract Visual AutoRegressive (VAR) modeling employs a next-scale decoding paradigm that progresses from coarse structures to fine details. While enhancing fidelity and scalability, this approach challenges two fundamental assumptions of conventional dynamic inference: semantic stability (intermediate outputs approximating final results) and monotonic locality (smooth representation evolution across layers), which renders existing dynamic inference methods ineffective for VAR models. To address this challenge, we propose FreqExit , an integrated loss design that enables dynamic inference in VAR without altering its architecture or compromising output quality. FreqExit is based on a key insight: high-frequency details are crucial for perceptual quality and tend to emerge only in later decoding stages. Leveraging this insight, we design targeted mechanisms that guide the model to learn more effectively through frequency-aware supervision. The proposed framework consists of three components: (1) a curriculum-based supervision strategy with progressive layer dropout and early exit loss; (2) a wavelet-domain high-frequency consistency loss that aligns spectral content across different generation steps; and (3) a lightweight self-supervised frequency-gated
Large-scale 4D scene generation aims to synthesize dynamic 3D environments and provides a critical intermediate representation for downstream tasks such as autonomous driving simulation, embodied agent training, and scene forecasting. Existing methods typically adopt a two-stage latent diffusion paradigm, which improves computational efficiency by modeling and generating scenes in a compressed latent space. However, this paradigm suffers from a reconstruction-generation trade-off: increasing the latent dimensionality improves reconstruction fidelity, but substantially increases the computational burden of diffusion modeling and makes generative optimization more challenging. This issue becomes particularly pronounced in 4D occupancy generation, where complex spatial layouts and long-range temporal dynamics must be jointly preserved within compact representations. To alleviate this problem, we advance a central principle: low-dimensional latent spaces should not rely solely on unconstrained compression, but should instead be structurally regularized to preserve sufficient 4D spatio-temporal information while maintaining compactness. To instantiate this principle, we propose Compact Autoregressive Latent Prior (CALP), which regularizes low-dimensional latent variables with a history-conditioned autoregressive prior, achieving compact, long-range temporally coherent, and spatially structured latent representations through a more reasonable latent capacity allocation and a latent-space organization that is better suited for diffusion-based 4D generation. Built upon CALP, we further introduce 4DWeaver, a compact 4D scene generation framework that enables high-quality spatio-temporal occupancy synthesis in a low-dimensional latent space. Extensive experiments on multiple large-scale 4D occupancy benchmarks demonstrate that 4DWeaver achieves superior reconstruction and generation performance while substantially reducing memory consumption and computational cost.
Ruosen Hao, Tianyu Shen, Jing-Yuan Gao et al.· IEEE Transactions on Pattern...· 0 citations
VISTA is the first gradient-based test-time alignment framework for next-scale autoregressive image generation, and introduces the mechanisms needed to make such optimization stable across scales, together with an extensible objective space that any differentiable constraint on cross-attention can plug into.
Hossein Shahabadi, Niki Sepasian, M. Baghshah· 0 citations
We introduce DenseAR, a new generative paradigm that reformulates autoregressive image generation as coarse-to-fine next-dense-stride prediction using a compact single-scale tokenizer. Our key insight is that traversing a single-scale latent grid with progressively denser strides naturally captures the transition from global structure to fine detail. This addresses two limitations of existing autoregressive models at once: the slow inference of raster-order autoregression, which DenseAR avoids by predicting multiple tokens in parallel, and the heavy cost of multi-scale approaches, which need long, multi-resolution token sequences to achieve coarse-to-fine prediction. Building on our efficient framework and the flexibility of autoregressive modeling, we further extend DenseAR to a unified model that handles multiple modalities and imaging tasks within a single backbone. We validate DenseAR on both medical and natural images. On multi-contrast brain MRI, a single DenseAR model unifies cross-modal translation, modality-conditioned generation, and tumor segmentation, while remaining competitive with task-specific methods. On ImageNet, DenseAR improves class-conditional generation quality (FID and IS) over both a single-grid baseline without stride ordering and a multi-scale tokenizer-based baseline.
Chicago Y. Park, Jialin Mao, Xiaojian Xu et al.· 0 citations
MotionCraft is presented, a controllable VSR framework that formulates restoration as motion-aware latent state prediction inspired by world models and integrates adaptive sparse attention with an explicit user-accessible control interface to deliver temporally consistent, high-quality reconstructions under streaming constraints.
Rong Fu, Chunlei Meng, Yangcheng Zeng et al.· 0 citations
