#artificial intelligence
May 2025
A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
Low-Rank Clone (LRC), an efficient pre-training method that constructs SLMs aspiring to behavioral equivalence with strong teacher models, and maximizes knowledge transfer while removing the need for explicit alignment modules.
Jitai Hao, Qiang Huang, Hao Liu et al.
· Neural Information Processin... · 17 citations
· ⚡2