When Keywords Drop but Classifiers Hold: Soft Refusals under KV Cache Compression
KV cache compression is widely used for long context LLM inference under memory constraints, while deployed systems typically score refusals after generation with keyword filters or learned classifiers. Such monitors are intended to indicate whether a model declined a harmful request under the serving regime actually u...
Kang Chen, Xiu-Ze Zhou, Hong Chen et al.
· 0 citations