By coupling QAD's stable low-bit initialization with OPD's on-policy reasoning recovery, the framework provides a comprehensive sub-3-bit solution that preserves broad capabilities while restoring long-form reasoning.
Yuan-Teng Chen, Zhi-Lei Liu, Peisong Wang et al.· 0 citations
Fine-grained mixture-of-experts (MoE) architectures have become a mainstream design for open-weight LLMs, with hundreds of experts and increasingly many selected per token. This shift makes dynamic expert pruning an attractive route to cheaper inference. Yet existing evidence comes largely from coarser architectures an...
Yuan-Teng Chen, Qi-Wei Lai, Tian-Qi Chen et al.· 0 citations
BiSCo-LLM is presented, a codebook-free binary spherical coding framework for extreme low-bit LLM weight compression and its reported storage budget includes binary codes, neural decoders, protected-channel payloads, LoRA adapters, and metadata.
Yuantian Shao, Peisong Wang, Zhilei Liu et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.