Skip to content

Adversarial Training for Pixel Diffusion

Sep 2026 · arXiv.org · Vol abs/2609.38170 · 0 citations · 55 references
Computer Science

TL;DR

This work identifies direct output access to the image statistics being corrected as a key factor governing when adversarial post-training succeeds, the first systematic study of adversarial post-training for pixel diffusion.

Abstract

Pixel diffusion models generate RGB images directly, avoiding the bottleneck of an autoencoder, yet their outputs still systematically underrepresent fine-scale natural-image statistics. We show that adversarial learning provides an effective post-training correction for this deficiency. Starting from a pretrained model, we retain its original diffusion or flow-matching objective and add an adversarial loss to the predicted output at non-high-noise timesteps, leaving the model architecture and sampling procedure unchanged. To our knowledge, this is the first systematic study of adversarial post-training for pixel diffusion. Across two pixel backbones, the method jointly improves distribution fidelity, coverage, prompt alignment, and perceptual quality. We further investigate why it works. Frequency-band and power-law analyses show that the original models systematically underproduce natural-image high-frequency content, while adversarial post-training restores this missing spectral power. In contrast, perceptual loss also increases high-frequency content but sacrifices distribution fidelity and prompt alignment. Nearest-neighbor, recall, and matched no-GAN SFT controls further rule out memorization, mode dropping, and additional optimization as simple explanations. Finally, we examine the boundary of this effect. Under the tested latent diffusion configurations, the same procedure does not produce comparable joint gains and adds almost no decoded high-frequency power. These results identify direct output access to the image statistics being corrected as a key factor governing when adversarial post-training succeeds.

View source

Similar papers

#artificial intelligence Preprint Aug 2026

GAN-Diff : Coupling Pretrained WGAN-GP Features with Conditional Diffusion U-Nets

A hybrid GAN-guided diffusion framework that uses a pretrained Wasserstein GAN with gradient penalty (WGAN-GP) as a feature prior for conditional diffusion-based image restoration that consistently improves the quality of both degraded and low-resolution images.

Saif Ahmed, A. Galib, S. Antu et al. · 0 citations
Sep 2026

Boosting cross-architecture adversarial transferability by enhanced deformation attack.

The Enhanced Deformation Attack (EDA), which estimates attack gradients over stochastically transformed views of the current adversarial image, achieves the highest mean attack success rate (ASR) on ViT targets for all four CNN sources, outperforming the strongest source-specific baseline.

Jia-Cheng Wang, He-Gui Zhu, Yi-Fan Zhang et al. · 0 citations
#machine learning Open access Oct 2026

PhaseAT: Fourier Phase Adversarial Training for Medical Image Domain Generalization

Reliable clinical deployment of deep medical image models is hindered by distribution shifts across scanners, sites, and acquisition protocols. Existing domain generalization (DG) methods often focus on style or intensity diversification, but they can still leave networks dependent on domain-specific texture correlatio...

Ahmed Sharshar, Asif Hanif, Naveen Kumar Kummari et al. · 0 citations
Preprint Aug 2026

PixRestore: Unified Image Restoration via Pixel Diffusion Transformer

PixRestore is presented, a VAE-free pixel-space Diffusion Transformer (DiT) for UIR, where the diffusion backbone is trained entirely from scratch, without relying on T2I pretraining.

Ling-Chen Sun, Rong-Yuan Wu, Xiang-Tao Kong et al. · 2 citations
Preprint Aug 2026

Fast Test-Time Refinement for Robust Learned Image Compression

This study reveals an Asymmetric Adversarial Trajectory (AAT) property in LIC systems: transitioning from adversarial to benign regions is significantly easier than the reverse process, where adversarial examples can often be roughly recovered within only 1-2 steps.

Jia-Ming Liang, Chi-Man Pun, Weisi Lin · 0 citations
Sep 2026

DMA2: Pixel-space Distribution Matching with Adversarial and Anchor Losses

Distribution matching distillation (DMD) provides a general framework for few-step diffusion generation, but its modern text-to-image instantiations have been developed primarily around latent diffusion. It therefore overlooks key properties and design opportunities of native RGB. We revisit two DMD interfaces for pixe...

Xin Lin, Zhi-Fei Zhang, Yu-Qian Zhou et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.