Skip to content
Log in

Gradient Enhancement Task Aware Post-training Quantization

Yihua Shao Yangyang Gu Minxi Yan Siyu Chen Haiyang Liu Ziyang Yan Yongjian Li Yan Wang Qun Song Hao-Lin Tang Haotong Qin Jingcai Guo Nicu Sebe Hong Kong
· 0 citations · 43 references

TL;DR

This paper introduces Gradient Enhancement Task Aware Post-training Quantization, i.e., GTAQ, to address the generalization issue of Large Language Models, and extensively evaluates the LLaMA family of language models on WikiText, C4, and MMLU.

Semantic Scholar
View source

Similar papers

Jul 2026

P4Q: Learning to Prompt for Quantization in Low-Bit CLIP

Large-scale pre-trained Vision-Language Models (VLMs) have demonstrated remarkable performance across various visual and multimodal tasks. However, deploying these models on downstream application platforms remains challenging due to computational demands and domain gaps. Quantization offers a promising solution by significantly reducing these costs, making VLMs more feasible for deployment in such environments. There are two prevailing paradigms: Quantization-Aware Training (QAT), which preserves model performance but incurs substantial training costs; and Post-Training Quantization (PTQ), which offers greater efficiency but introduces multimodal gaps and leads to performance degradation on downstream tasks. To reduce computational costs and bridge domain gaps, we propose the “Prompt for Quantization” (P4Q) by integrating PTQ with Parameter-Efficient Fine-Tuning (PEFT) techniques. P4Q compresses model parameters and activations via PTQ, introducing learnable prompts and low-bit adapters to enhance performance on downstream tasks. The learnable prompts embed downstream knowledge to mitigate domain gaps, while the low-bit adapters realign the distributions of image and text features, thereby mitigating multimodal gaps. We also introduce a distillation loss based on cosine similarity predictions to distill the quantized model using a full-precision teacher model. Extensive experiments on thirteen datasets demonstrate that P4Q significantly enhances the performance of low-bit CLIP while reducing deployment costs. For instance, an 8-bit P4Q compressed CLIP-ViT/B-32 achieves 66.94% Top-1 accuracy on ImageNet, surpassing the prompt fine-tuned full-precision counterpart by 2.24% while reducing model size by 4 \(\times\) . The source code is publicly available at https://github.com/HuixinSun/P4Q_official.

H. Sun, Runqi Wang, Yanjing Li et al. · 0 citations
Conference Open access 2026

ACBQ: Adaptive Cross-Block Quantization of Large Language Models

ACBQ is presented, a simple yet effective framework that simultaneously addresses weight–activation joint quantization and extreme low-bit weight quantization and an adaptive cross-block quantization strategy that explicitly accounts for cross-layer dependencies by encouraging consistency across blocks.

Hailing Wang, Jianglin Lu, Yitian Zhang et al. · 0 citations
Conference Open access 2026

PE-QAT: Parameter-Efficient Quantization-Aware Training for Large Language Models

PE-QAT is introduced, a parameter-efficient framework targeting per-channel 4-bit weight-activation quantization of LLMs, which aims to preserve model accuracy while significantly reducing resource requirements and mitigate the impact of severe activation outliers.

Shresth Mishra · 0 citations
Preprint Jul 2026

C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

C-PTQ is proposed, a unified channel-wise PTQ method that harmonizes task-specific loss perturbation and quantization error and achieves state-of-the-art performance without auxiliary modules like LoRA, thereby maintaining high efficiency.

Jiameng Li, Han Zhou, M. Blaschko · 0 citations
Book Open access Jul 2026

Scaling and Stabilizing Large-Scale Embedding-Based Retrieval

A unified pipeline deployed at Walmart that addresses both signal quality and model evolution is presented, and a Warm-Start Distillation technique that transfers domain-specific expertise from the legacy model to the new backbone is introduced.

Zhen Yang, Juexin Lin, Hongwei Shang et al. · 1 citation
Preprint Aug 2026

From Specialization to Generalization: Instruction-tuned LLMs for Robust Harmful Content Mitigation

By thoroughly unifying 36 English hate speech datasets spanning multiple labeling schemes, this work fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate speech mitigation, demonstrating not only state-of-the-art performance on in-domain benchmarks but also substantial improvements in cross-domain and cross-lingual generalization--areas where encoder-based specialist classifiers often struggle.

Lukas Edman, Daryna Dementieva, Alexander Fraser · 0 citations