Native Binary-Weight Training Outperforms Zero-Shot Sign Projection in MNIST Diffusion

Abstract

Does severe failure after projecting a diffusion model to one-bit weights show that its binary representation is unusable? We test that implication, rather than propose a new quantizer, in a small MNIST residual U-Net. We compare floating-point training, zero-shot per-channel sign/scale projection of a trained model, binary-weight training from random initialization, floating-point warm-started quantization-aware training (QAT), and a binary-weight/binary-activation core. The primary FP32, native W1A32, and projected W1A32 comparison fixes the parameterization, data, training budget where applicable, and 1,000-step ancestral sampler, with three independent training seeds. Native W1A32 training obtains a domain-feature Fréchet distance of 46.33 ± 17.24, which is 28.4-fold lower than matched zero-shot projection (1314.20 ± 314.96), but remains substantially worse than FP32 (3.15 ± 0.38). Warm QAT also recovers much of the projection loss, while the W1A1 core retains a large distributional gap. An elementary feasible-set argument explains why one projected floating-point solution is not an impossibility certificate for the binary hypothesis class. The supported claim is deliberately local: on MNIST, optimizing binary forwards reaches a much better W1A32 solution than this zero-shot projection baseline. This does not establish full W1A1 success, floating-point parity, or transfer to natural images. This record is an unreviewed technical report. The accompanying source archive contains the LaTeX manuscript, bibliography, and manuscript figures. Code, checkpoints, evaluation artifacts, and the Diffuino demonstration are available from the related project resources.