Preprint
Aug 2026
ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads
ARCHead is presented, a packed LM-head compressor that combines a quantized low-rank core, group-wise INT4 residuals, and a low-rank correction fitted in an activation-derived metric to complements block quantizers by compressing the large output projection they can leave untouched.
Suayp Talha Kocabay, Talha Rüzgar Akkus, K. Yuksel
· 0 citations