A target-centric survey of QAT is provided, aimed at clarifying both its theoretical foundations and its evolving implementation landscape and synthesizing cross-target differences in error characteristics, numerical formats, and strategy transferability.
Abstract
The rapid development of LLMs incurs prohibitive memory footprints and intensive computational demands. Quantization-Aware Training (QAT) techniques have emerged as a promising solution to address these challenges by explicitly simulating quantization effects during model training, yielding low-bit models that achieve accuracy comparable to their full-precision counterparts. In this work, we provide a target-centric survey of QAT, aimed at clarifying both its theoretical foundations and its evolving implementation landscape. We systematically review existing QAT methods through a target-centric taxonomy and synthesize cross-target differences in error characteristics, numerical formats, and strategy transferability. We further summarize QAT evaluation paradigms and discuss challenges in optimization and deployment, outlining potential directions for future research.
A cross-modal structural sensitivity asymmetry in VLMs is revealed and SeGO is proposed, a unified structural sensitivity-aware sparse optimization framework that achieves the balance among model parameter amount, quantization accuracy and scaling factors’ search efficiency on InternVL2 and LLaVA series.
Tian-Qi Zhao, Xin-Rui Cheng, Yang Su et al.· Proceedings of the Thirty-Fi...· 0 citations
Large language model (LLM) inference is increasingly moving toward lower precision to realize the throughput of hardware accelerators, but aggressive post-training quantization (PTQ) can degrade model quality. We present QATFactory, an open-source framework for deployment-aligned quantization-aware distillation (QAD) a...
Wei-Li Xu, Ji-Sen Li, Yu-Qing Jian et al.· 1 citation
A comprehensive PTQ framework is presented that addresses the problem of compressing LLM weights through three core innovations: a calibration process guided by Kullback-Leibler divergence minimization to preserve the original weight distribution, a learnable codebook optimization mechanism employing noise substitution...
Bao Tan Duy Huynh, Takashi Tsunakawa, Masafumi Nishida· International Conference on...· 0 citations
This paper introduces Gradient Enhancement Task Aware Post-training Quantization, i.e., GTAQ, to address the generalization issue of Large Language Models, and extensively evaluates the LLaMA family of language models on WikiText, C4, and MMLU.
Yi-Hua Shao, Yang-Yang Gu, Min-Xi Yan et al.· Proceedings of the Thirty-Fi...· 1 citation
Real-Q (Real-time E2E-loss Aligned LLM Quantization), a novel PTQ paradigm that breaks this compromise: instead of diluting the objective for the sake of analytic tractability, REAL-Q targets an end-to-end-aligned surrogate of the global loss and refines it via fine-grained, dynamic Block-wise Gradient Descent applied...
Qian Zhang, Yao-Ming Li, Zheng Tan et al.· 0 citations
This work presents a framework for quantizing VLMs for efficient inference on resource-constrained hardware, which combines a quantization pipeline that uses the model itself to generate training data and does not require access to the training setup, with a novel 2.7-bit-per-parameter format.
Luka Ribar, Jeevan Bhoot, Douglas Orr· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 14, 2026
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.
MIT News · Artificial Intelligence· news.mit.eduOct 13, 2026
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
