Skip to content

DEdit: Iterative Draft Editing for Speculative Decoding

Sep 2026 · arXiv.org · Vol abs/2609.38510 · 0 citations · 42 references
Computer Science

TL;DR

DEdit is introduced, a diffusion-based drafter that can not only draft by conventional parallel unmasking but also iteratively edit its draft through token-to-token predictions, and ProposalMix, a training scheme that mixes draft predictions with ground-truth tokens based on first-pass confidence during training.

Abstract

Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusion-based drafters further reduce drafting latency by proposing multiple tokens at once. However, these tokens are predicted independently, so a single early error causes prefix verification to discard the rest of the draft, even when it contains useful downstream predictions. We introduce DEdit, a diffusion-based drafter that can not only draft by conventional parallel unmasking but also iteratively edit its draft through token-to-token predictions. Through editing, later predictions can serve as bidirectional context for repairing earlier errors and extending the accepted prefix. To teach the model to repair errors while preserving correct predictions, we propose ProposalMix, a training scheme that mixes draft predictions with ground-truth tokens based on first-pass confidence during training. Across seven benchmarks on Qwen3-4B and Qwen3-8B, DEdit achieves the highest macro-average token acceptance and speedup among the evaluated drafters, reaching macro-average speedups of $5.72\times$ and $5.97\times$ over autoregressive generation under greedy decoding, respectively. Further analysis shows that acceptance improves with more editing passes and wider drafting windows, and that ProposalMix halves harmful edits that shorten the accepted prefix. Moreover, restricting the editor to causal attention lowers acceptance, especially on highly predictable outputs, indicating that future context is a key source of these gains.

View source

Similar papers

#natural language process... Preprint Oct 2026

Contextual Correction of Parallel Drafts for Speculative Decoding

Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this paradigm particularly attractive: deep attention-based neural backbones provide strong predi...

Roy Eisenstadt, Harel Lidar, T. Gonen et al. · 2 citations
#natural language process... Preprint Oct 2026

DLoop: Looped Speculative Decoding

Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft models, we find that the target model frequently accepts all tokens produced in a drafting s...

Geonmo Gu, Byeongho Heo, HeeJae Jun et al. · 0 citations
#natural language process... Preprint Oct 2026

D-Loop: Looped Diffusion Drafting for Speculative Decoding

D-Loop is proposed, which introduces \emph{intra-block causal conditioning} within the original diffusion drafter without additional model components, and can beat DFlash and DSpark on Qwen3-4B and Qwen3-8B with obvious gains.

Ke-Cheng Chen, Yu-Yang He, Cheng Gong et al. · 0 citations
#artificial intelligence Preprint Sep 2026

LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

This work introduces LongSpark, a block-diffusion drafter that achieves state-of-the-art end-to-end efficiency across multiple model scales and realistic serving conditions by extracting fixed-size, multiscale views from the target's verification pass, thereby eliminating the need for a growing persistent state.

Hao-Yuan He, Peng-Fei Liu, Si-Shi Shen et al. · 0 citations
#machine learning Preprint Sep 2026

Acceptance-Aware Draft Model Training for Speculative Decoding

This work proposes acceptance-length-aware training losses that directly optimize the expected number of accepted tokens within a speculative window and shows that directly optimizing the acceptance objective is more effective than conventional distribution-matching objectives.

Tian-Hua Xia, M. Ganesan, Yi-Fei Feng et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

A theoretical framework for training and evaluating parallel and semi-autoregressive drafters by representing speculative decoding as a Markov reward process, which yields the Expected Decoding Rounds (EDR) objective, which weights local rejection costs by state occupancies and exactly equals the expected number of dec...

Yun-Xiao Zhao, Chang-Xiao Cai · 3 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.