Fine-tuning language models on consumer GPUs is constrained by computation and memory. Although LoRA reduces trainable parameters and optimizer state, frozen projections remain expensive, and activations retained for backward can dominate memory usage. We present PACT, a LoRA fine-tuning system that accelerates frozen...
Lei Gao, Hossein Entezari Zarch, Chao-Yi Jiang et al.· 0 citations
Results show that HeteroFold enables efficient cross-family KV reuse without receiver prefill, and achieves the best cache-transfer performance on all four long-context benchmarks and most short-context settings.
OptR is proposed, an output-aware rotation method that minimizes post-W_O$ attention-output error and applies an attention-equivalent key reparameterization to reduce large channel-wise offsets without changing the softmax distribution.
This work proposes ASPIRE, a non-synchronized batched self-speculative decoding framework built on three components, which achieves speedup in decoding throughput over autoregressive baselines and improves average speedup by approximately $27\% over the strongest prior self-speculative baselines.
Amir Ziashahabi, Hossein Entezari Zarch, Lei Gao et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.