PRISM a diagnostic framework for parametric recall degradation in quantized large language models

Abstract

Post-training quantization (PTQ) is the standard method for deploying large language models (LLMs), though it is widely assumed to degrade model quality uniformly. Although, this assumption masks how quantization uniquely affects different mechanisms, particularly the balance between a model’s internal parametric memory and its ability to utilize external context. Existing benchmarks primarily measure overall accuracy, failing to isolate these specific sources of knowledge degradation when bit-rates are reduced. To address this, we introduce PRISM (Precision-Resolved Inspection of Stored Memory), an efficient diagnostic framework that separates and measures these distinct capabilities on a single 8GB consumer GPU. PRISM isolates these effects by leveraging counterfactual knowledge-conflict probes, component-wise mixed-precision quantization, and leak-free comprehension probes. We evaluate this framework across multiple model families (Qwen2.5, Llama-3.2, Gemma-2) and diverse quantization schemes (round-to-nearest, NF4, GGUF, GPTQ). Our results demonstrate that open-ended parametric recall collapses under low-bit PTQ while verbatim copy-from-context remains robust; furthermore, component-wise probing reveals that MLP/FFN quantization drives this recall collapse far more than attention layers. Ultimately, these insights reveal that while closed-book benchmarks overstate quantization harm for simple retrieval tasks, complex multi-hop reasoning remains vulnerable, providing crucial nuance for deploying compressed LLMs. Clinical trial Not applicable.