Sep 2026· 2026 11th International Conference on Computer Science and Engineering (UBMK)· pp. 770-775· 0 citations· 16 references
TL;DR
HaRP-ViT (Hadamard Rotation for Post-Training Quantization of Vision Transformers), a training-free randomized Hadamard transform applied exclusively after the GELU activation is proposed, making the post-GELU tensor better suited to standard per-tensor uniform quantization.
Abstract
Vision Transformers (ViTs) demonstrate state-of-the-art performance on computer vision tasks such as image classification, object detection and semantic segmentation. However, their large number of parameters and high memory requirements make them difficult to deploy on edge devices. Post-training quantization (PTQ) addresses this issue by efficiently compressing pretrained models using only a small set of calibration images, without retraining. While PTQ performs well on ViTs at higher bit-widths, low-bit quantization remains challenging. In this work, we investigate a calibration-only baseline to identify the dominant source of residual quantization error. Through a component-wise sensitivity analysis on RepQ-ViT, we find that post-GELU activations exhibit the highest sensitivity at W4A4 across both DeiT and ViT model variants. Our experiments show that restoring this component to 8-bit precision yields more than twice the top-1 accuracy recovery obtained by restoring any other component. Motivated by this finding, we propose HaRP-ViT (Hadamard Rotation for Post-Training Quantization of Vision Transformers), a training-free randomized Hadamard transform applied exclusively after the GELU activation. This output-preserving rotation spreads channel-wise outliers, making the post-GELU tensor better suited to standard per-tensor uniform quantization. Experiments on five ViT/DeiT models evaluated on ImageNet-1k, averaged over five runs with different calibration samples, show that HaRP-ViT improves the W4A4 RepQ-ViT baseline by approximately 2.5 to 6.7 percentage points in top-1 accuracy.
Vision Transformers (ViTs) achieve strong performance on image recognition and mobile vision applications, but their high-dimensional linear projections and attention computations still impose substantial storage and inference costs. Extremely low-bit quantization is a promising solution, yet ViTs often suffer severe a...
Meng-Yuan Fan, Bo-Kai Huang, Jia-Ming Pan et al.· 0 citations
Deploying vision-transformer backbones on edge devices requires low-bit quantization, and mixed-precision methods must assign each layer a bit-width under a memory budget. Allocation is usually driven by per-layer sensitivity estimated from loss curvature (Hessian traces), following HAWQ. We propose CASQ (Configuration...
Luka Kričković, Vijay Kakani· Electronics· 0 citations
Extensive experiments across representative ViT architectures demonstrate that GLF-Q with standard uniform quantizers substantially outperforms state-of-the-art methods under 3-bit quantization on image classification and exhibits strong out-of-domain calibration robustness and achieves speedups under 8-bit GPU deploym...
Microscaling (MX) formats are emerging as a hardware-supported approach to efficient training and inference. They combine low-precision elements with shared block scales, but their impact on vision models remains underexplored. We systematically investigate post-training MX quantization across vision models and tasks....
Elad Cohen, Ofir Gordon, Lior Dikstein et al.· 0 citations
Post-training quantization (PTQ) compresses the weights of a trained model with limited compute and data. Down to 4 bits it is near-lossless, but at 2 bits scalar PTQ configurations fall to chance-level accuracy. We study three choices usually treated as implementation details that matter little at 4 bits but decide 2-...
R. Sukthanker, Riccardo Grazzi, James Hensman et al.· 0 citations
ThinQuant is introduced, a data selection procedure which reduces the required number of calibration data points and an exact reduction of the associated optimization on this reduced calibration set using an efficient ADMM algorithm that iteratively employs thin matrix updates at every step, hence the name ThinQuant.
Mehdi Makni, Ryan Lucas, R. Mazumder· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.