This work proposes Residual Fallback Quantization (RFQ), a lightweight activation reconstruction framework that supplements the primary ulta-low-bit activation representation with an auxiliary quantized residual pathway that improves activation fidelity while preserving the efficiency advantages of ultra-low-bit comput...
C-PTQ is proposed, a unified channel-wise PTQ method that harmonizes task-specific loss perturbation and quantization error and achieves state-of-the-art performance without auxiliary modules like LoRA, thereby maintaining high efficiency.
This work presents SCULPT (Statistical Clipping and Uniform Loss for Post-Training), a training-time method that improves PTQ readiness during ordinary FP32 fine-tuning and combines a topology-aware activation regularizer that suppresses quantization-hostile skewness and kurtosis with a stable percentile-based clipping...
Vision-language pre-training (VLP) serves as a cornerstone for medical multimodal representation learning. However, existing medical VLP frameworks are often constrained by the limited context windows and shallow representational capacities of lightweight text encoders when processing lengthy, terminology-dense clinica...
The Channel-wise Token-balanced Output-Aware Clipping (CTOAC) method, which learns per-input-channel clipping bounds by minimizing a token-balanced reconstruction loss on the corresponding linear outputs, is proposed.
The underlying causes of miscalibration in VLMs and Medical-VLMs are investigated and a simple train-time calibration method, Multi-Class Margin (MCM) regularization, is proposed, which achieves lowest ECE on 10 out of 12 settings in in-domain and remains competitive under domain shifts.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
