Skip to content

Unifying Distributional Training for One-Step Visual Generation

Sep 2026 · 2 citations · 74 references
Computer Science

TL;DR

A unified theoretical framework is introduced that separates distribution modeling from matching discrepancy and connects global objectives to pointwise feature updates through Wasserstein gradient flow and motivates MGFlow, which models feature distributions with Gaussian mixtures at an adjustable granularity between global moments and sample-based representations.

Abstract

Distributional training provides collective supervision for one-step visual generation by matching real and generated features in frozen representation spaces. We introduce a unified theoretical framework that separates distribution modeling from matching discrepancy and connects global objectives to pointwise feature updates through Wasserstein gradient flow. Under this framework, FD-Loss and Gaussian-kernel Drifting are recovered through Gaussian optimal transport and kernel-density-based KL matching, respectively. The framework motivates MGFlow, which models feature distributions with Gaussian mixtures at an adjustable granularity between global moments and sample-based representations. MGFlow supports both optimal transport and score-based matching, and couples mass-constrained sample assignment with paired component updates to address mode collapse that mixture expressivity alone does not resolve. On ImageNet $256\times256$, MGFlow substantially surpasses the FD-Loss baseline, achieving state-of-the-art results with 1.45 $\mathrm{FDr}^6$ on pMF-H and 1.64 on JiT-H. For text-to-image generation, MGFlow post-trains FLUX.2 [klein] 4B into a one-step generator that outperforms the original four-step model on both GenEval and PickScore.

View source

Similar papers

Data Unlearning via Inverse Distillation

Multi-step matching models, including flow and diffusion models, produce high-quality outputs but incur substantial inference costs and may reproduce unwanted components of their training datasets. We introduce Inverse Distillation Unlearning (IDU), a unified framework that simultaneously distills a teacher multi-step...

Aleksei Leonov, N. Kornilov, Zhen-He Zhang et al. · 0 citations

Learning a Flow to Self-Supervised Representations

Flow-Based Distribution Matching is introduced, a non-adversarial framework that learns this reference-directed geometry through spherical conditional velocity regression that achieves performance nearly on par with DM and remains competitive with existing SSL methods.

Yu-Ling Jiao, Wen-Sen Ma, Hou-Duo Qi et al. · 0 citations
Conference Open access Sep 2026

Isotropic Embedding Perturbations for Robust Vision Language Encoders

Aether is introduced, a simple plug-in method that applies diffusion-style random perturbations in the embedding space via controlled alpha-mixing, specifically designed to provide isotropic regularization that remains semantically consistent.

Hyesong Choi, Daeun Kim, Song Park et al. · 0 citations
Sep 2026

Generative Uncertainty as a Self-supervised Signal for Semantic Similarity Learning

A novel self-supervised approach that leverages generative uncertainty from text-to-video (T2V) diffusion models to learn semantic similarity without human annotations to identify stable semantic features within existing pretrained representations, such as VideoMAE and V-JEPA.

Enrico Pallotta, Sina Raoufi, Lars Doorenbos et al. · 0 citations
Preprint Aug 2026

Geometric Regularization for Long-Tailed Semi-Supervised Learning via Gaussian Feature Bridges

This work introduces a novel framework, Gaussian Bridge Consistency (GBC), to address challenges of semi-supervised learning by constructing semantic interpolation paths between unlabeled samples and high-quality class anchors, and proposes BridgeMix, a confidence-aware feature mixing strategy that interpolates both sa...

Hong-Yang He, Xin-Yuan Song, Yan Zhong et al. · 1 citation
Sep 2026

Training-Free Spectral Transductive Refinement for Cross-Domain Few-Shot Classification

Spectral Transductive Refinement (STR), a training-free transductive inference rule that exploits the geometry of the complete support-query episode, operating entirely at inference time, attains the highest 1-shot average among compared methods and remains competitive at 5-shot, rivalling approaches relying on heavy s...

F. Rahman, S. Rohan, Mahmound Sayed et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.