Skip to content
Preprint

Alignment Drift in Single-Model Speculative Decoding for ASR: Mechanism, Correction, and Cost

Aug 2026 · 1 citation · ⚡ 1 influential · 30 references
Computer Science

TL;DR

Results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost, and AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph, improves end-to-end speed at both tested target scales.

Abstract

Speculative decoding speeds up generation by letting a cheap draft propose several tokens that a target model checks in one pass. In the single-model form, the draft is a lightweight module attached to the target rather than a separate model. Applying this design to Automatic Speech Recognition (ASR) introduces an extra problem. The draft can read the whole audio at every step, yet its proposals get worse as it runs on its own. Access is not localization. The accepted text keeps the transcript position explicit, but the draft must also track the changing audio position. In the primary matched comparison, per-step audio access changes the first proposal modestly but roughly doubles later-proposal acceptance. Fixed-width windows show that the audio position explains part of this gap. A correctly placed window recovers continuation, while an equally narrow window at the wrong position reduces it. Late-draft median error reaches 21 frames in the hardest reported condition, while target attention during verification stays within a 2-frame median. We test two ways to reduce this drift. The first reads the audio position from verification attention and uses it to guide the next draft round. It saves time only when the extra accepted tokens offset the readout cost. The second is AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph. The trained draft improves end-to-end speed at both tested target scales. These results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost.

View source

Similar papers

#natural language process... Preprint Aug 2026

Verification-Aware Training for Speculative Decoding

Verification-Aware Training is introduced, a plug-in framework that simulates verification at every training step and turns the resulting accept and reject patterns into supervision and improves average acceptance length and wall-clock speedup across math, code, and chat benchmarks.

Geonmo Gu, Byeongho Heo, HeeJae Jun et al. · 6 citations · ⚡1
#artificial intelligence Preprint Sep 2026

LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

This work introduces LongSpark, a block-diffusion drafter that achieves state-of-the-art end-to-end efficiency across multiple model scales and realistic serving conditions by extracting fixed-size, multiscale views from the target's verification pass, thereby eliminating the need for a growing persistent state.

Hao-Yuan He, Peng-Fei Liu, Si-Shi Shen et al. · 0 citations
#natural language process... Preprint Aug 2026

LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding

LiLiCorr, a Lightweight Likelihood-based model that Correlates the per-position marginal distributions a drafter already produces, is introduced, a Lightweight Likelihood-based model that Correlates the per-position marginal distributions a drafter already produces.

M. Rusanovsky, Yoav Miron, Roy Uziel et al. · 4 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.