CLEAR: Water-Filling Rank Allocation with Sparse Dictionary Representations for Training-Free LLM Compression
TL;DR
CLEAR (Convex-optimal Layer Energy Allocation and Representation), a training-free compression framework that combines an activation-whitened structured-dictionary representation with a provably optimal, globally coordinated budget allocation across layers, is proposed.
Abstract
Training-free LLM compression avoids fine-tuning by approximating weight matrices from a small unlabelled calibration set; however, existing methods assign each layer an independent target rank with no global budget coordination, leaving parameter distribution across layers systematically suboptimal. We propose CLEAR (Convex-optimal Layer Energy Allocation and Representation), a training-free compression framework that combines an activation-whitened structured-dictionary representation with a provably optimal, globally coordinated budget allocation across layers. Ranks are distributed across all layers simultaneously via a convex water-filling optimization, whose KKT solution is provably optimal and achieves <10−3 pp precision on the dictionary-path parameter budget (the total realized retention ratio, including EoRA and outlier bypass parameters, deviates from the target by at most 0.02 pp in practice). Each layer is then compressed using activation-whitened structured sparse dictionaries (W≈DC, k-sparse columns), supplemented by an EoRA low-rank residual correction and an outlier input-channel bypass. Evaluated on five LLMs spanning 600 M to 8 B parameters across four model families, CLEAR outperforms the current state of the art on three of four evaluated models at compression ratio (CR) = 0.2. On LLaMA-3.2-1B, average accuracy across eight zero-shot benchmarks reaches 50.4% versus 42.7% for CoSpaDi and 37.6% for SVD-LLM, with perplexity reduced from 63.7 to 22.4; on LLaMA-3-8B, 65.5% versus 61.8% for CoSpaDi. Component ablations confirm +5.22 pp from sparse dictionaries and +1.12 pp from global rank allocation. KFAC Fisher covariance proves counterproductive in this setting, degrading accuracy by 8.1 pp due to numerical overflow in SiLU gating layers and consequent budget misallocation. CLEAR completes in 8–30 min on a single GPU, requires no gradient computation at any stage (forward-pass only, including the cascade activation refresh in Phase 3), and reduces peak deployment memory by approximately 20% at BFloat16 precision.