Give the Drafter a Second Chance to Guess: Parallel Reconsideration for Speculative Decoding
Speculative decoding accelerates large language model inference with a small drafter, whose per-guess acceptance rate determines the achievable speedup. Feature-level drafters share a design choice: every token, easy or hard, is drafted under a single forward pass. Many of the tokens the target rejects are ones the dra...