Skip to content

On the Diffusibility of High-Dimensional Latents

Sep 2026 · 0 citations · 87 references
Computer Science

TL;DR

It is shown that using the standard velocity prediction in flow matching in this high-dimensional space requires the model to fit orthogonal noise directions outside the low-dimensional signal manifold, making optimization inefficient, and motivated using the clean data parameterization ($\boldsymbol{x}_{0}$-prediction) instead, which focuses learning on the underlying signal manifold.

Abstract

Representation Autoencoders (RAEs) enable diffusion models to operate in the feature spaces of pretrained visual encoders. However, many off-the-shelf encoders are not optimized for faithful reconstruction, discarding fine-grained visual details. As expected, finetuning these encoders for image reconstruction recovers such details. However, perhaps counterintuitively, this procedure reduces the effective dimensionality of the resulting representation, and the altered geometry has downstream effects on generation. Specifically, we show that using the standard velocity prediction in flow matching in this high-dimensional space requires the model to fit orthogonal noise directions outside the low-dimensional signal manifold, making optimization inefficient. This motivates using the clean data parameterization ($\boldsymbol{x}_{0}$-prediction) instead, which focuses learning on the underlying signal manifold. Across experiments with multiple strong-reconstruction encoders, we show that $\boldsymbol{x}_{0}$-prediction consistently improves text-to-image generation performance.

View source

Similar papers

Conference Open access Sep 2026

Isotropic Embedding Perturbations for Robust Vision Language Encoders

Aether is introduced, a simple plug-in method that applies diffusion-style random perturbations in the embedding space via controlled alpha-mixing, specifically designed to provide isotropic regularization that remains semantically consistent.

Hyesong Choi, Daeun Kim, Song Park et al. · 0 citations
Preprint Aug 2026

XYZFlow:Scaling Multi dimensional Shortcut Flows for Efficient Generative Modeling

High-fidelity image generation faces a trade-off between speed and quality. Diffusion models produce strong visuals but require costly iterative sampling. Existing efficient methods mainly distill pretrained models into few-step samplers, a challenging process that depends heavily on teacher-model quality. In this pape...

Jin-Xiu Liu, Xuan Liu, K. Mei et al. · 0 citations
Sep 2026

Masked Swingers: Harnessing Data Augmentation to Advance Autoencoders for Self-Supervised Learning

Self-supervised learning (SSL) removes the need for annotations and makes models that are capable across more domains than supervised learning. The autoencoder SSL framework learns by reconstructing its own input after information loss through a bottleneck or noise injection. Masked autoencoders (MAE) are the most succ...

A. Fuller, Scott C. Lowe, Daniel G. Kyrollos et al. · 0 citations
Sep 2026

Adversarial Training for Pixel Diffusion

This work identifies direct output access to the image statistics being corrected as a key factor governing when adversarial post-training succeeds, the first systematic study of adversarial post-training for pixel diffusion.

Xin Lin, Zhi-Fei Zhang, Yu-Qian Zhou et al. · 0 citations
#machine learning Preprint Sep 2026

Neural Residual Modeling for Scientific Data Compression under Guaranteed Error Bounds

This work proposes a post-processing pipeline that augments an RVQ-based compressor with a U-Net trained to predict and correct pixel-space residuals between the original volume and its RVQ reconstruction, and is the first pipeline to combine latent-space RVQ, explicit pixel-space residual correction via a deep spatial...

Surya Majumder, Liang-Ji Zhu, Sanjay Ranka et al. · 0 citations
#artificial intelligence Preprint Sep 2026

D-Scope: Decomposing and Steering Diffusion Transformers with Sparse Autoencoders

Sparse autoencoders (SAEs) reveal visual structure in diffusion transformers (DiTs), but interpreting a feature does not establish whether it can be used to control generation. We introduce D-Scope (Diffusion Scope), a framework that connects feature interpretation to generation control through shared visual evidence....

Xinyue Xu, Jiahao Zhang, Li-Jie Hu et al. · 1 citation

Related blog posts

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

Microsoft Research Blog Aug 11, 2026

Introducing CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement

Radiology AI is evolving beyond report generation. CARE-X explores a unified approach that combines flexible reasoning, calibrated predictions, and measurement-based tools for chest X-ray interpretation. The post Introducing CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.