Preprint
Oct 2026
When Does Better Acceptance Pay? A Controlled Study of Speculative Decoding
The throughput gains of speculative decoding depend on the trade-off between acceptance length and execution latency. We ask when additional acceptance is worth the cost of obtaining it. We establish a comparison protocol spanning training, inference, and evaluation, remeasuring released checkpoints and retraining six...
Yong-Chao Long, Chun-Rui Zheng, Yuxuan Zhou et al.
· 0 citations