Skip to content

FluxLite: Inference-Time Proposal Control for Discrete Diffusion Models

Sep 2026 · arXiv.org · Vol abs/2609.35947 · 1 citation · 190 references
Computer Science Mathematics

TL;DR

FluxLite is introduced, a lightweight, training-free proposal-control framework for discrete diffusion, identifying a tilted-path coverage factor that governs robustness to score error, together with finite-particle convergence for a fixed controlled Feynman-Kac recursion.

Abstract

Many inference-time tasks for pretrained discrete diffusion models and diffusion language models reduce to drawing samples from a tilted version of the pretrained distribution. Feynman-Kac sequential Monte Carlo (SMC) makes this correction exact in principle, but its prescribed weights routinely degenerate when the proposal dynamics are misaligned with the tilt, capping the practical gains from additional particles. We introduce FluxLite, a lightweight, training-free proposal-control framework for discrete diffusion. On the sparse directed graph of pretrained reverse rates, any sparse jump-rate perturbation can be exactly compensated by a $q_t$-weighted graph-divergence term in the Feynman-Kac potential; the target path is therefore preserved while the residual reweighting variance becomes a local convex objective. We instantiate this principle as two practical samplers: a one-hop local reallocation rule (HEU) and a small nonnegative quadratic program over pretrained-rate bases (D-VCG). We further prove population stability under the standard score-entropy training loss, identifying a tilted-path coverage factor that governs robustness to score error, together with finite-particle convergence for a fixed controlled Feynman-Kac recursion. Empirically, FluxLite improves over standard Feynman-Kac SMC baselines by up to two orders of magnitude in terminal KL on an analytically tractable finite-state CTMC benchmark, and reduces row-correlation MSE on 2D Ising sampling by 5-7x in geometric mean and up to 55x at peak.

View source

Similar papers

#machine learning Preprint Oct 2026

Error-Corrected Inference-Time Scaling for Imperfect Diffusion Models

Inference-time scaling adapts pretrained diffusion models to new sampling tasks without additional training. Existing methods rely primarily on Monte Carlo sampling with more particles, yet are premised on the pretrained model being exact. In practice, data and training limitations make the model imperfect, and these m...

Zuo-Kai Wen, Louis Grenioux, E. Weinan et al. · 0 citations

Simplex Diffusion Models

Simplex Diffusion Models (SDMs) are proposed, a framework that lifts the diffusion process to the probability simplex to represent beliefs over categories and can carry uncertainty across denoising steps, which mitigates information collapse.

Justin Deschenaux, Alexandre Galashov, Andrew Campbell et al. · 2 citations

Acceleration of Diffusion Language Model through Discrete Average Generator

Discrete diffusion models and flow matching have emerged as powerful frameworks for generative modeling over discrete state spaces, yet efficient few-step generation remains a fundamental challenge. In this work, we introduce the Discrete Average Generator, a principled extension of MeanFlow to Continuous-Time Markov C...

Yi-Dong Ouyang, Zheng-Yan Wan, Themistoklis Haris et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Risk-Calibrated Proposal Transport for Finite-Particle Diffusion Steering

Inference-time steering combines pretrained diffusion experts or rewards without retraining by changing the dynamics that transport noise to data. Feynman-Kac correction compensates for proposal mismatch through importance-weighted sequential Monte Carlo (SMC), whose finite-particle behavior depends on the proposal. Va...

Ziseok Lee, JaeHyeong Kim, Seungwon Kim et al. · 0 citations
#machine learning Preprint Oct 2026

IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models

IDRF, a framework for reward fine-tuning of few-step masked discrete diffusion generators, view few-step generation as a finite-horizon Markov decision process and optimize reward with a clipped policy-gradient objective over the student's trajectories.

V. Gromadskii, David Li, Samson Gourevitch et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.