Yun-Xiao Zhao

Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

A theoretical framework for training and evaluating parallel and semi-autoregressive drafters by representing speculative decoding as a Markov reward process, which yields the Expected Decoding Rounds (EDR) objective, which weights local rejection costs by state occupancies and exactly equals the expected number of dec...