zip2zip++: Towards Inference-Time Token Compression with Minimal Quality Loss
Dynamic vocabularies built on-the-fly via Lempel-Ziv-Welch (LZW) compression let a language model represent the same text in fewer tokens, but prior work shows that this efficiency comes at a cost to reasoning and perplexity. We show that this quality gap is not a fundamental price of compression but can be substantial...
Sai-Bo Geng, A. Gentilini, Xin-Xian Ma et al.
· 0 citations