Yepeng Weng

Tsubame: Tree Replay for Diffusion-Based Speculative Decoding

Context-aware dynamic trees allocate the speculative decoding budget according to draft path probabilities, adapting their depth and branching to the current context. Under stochastic decoding, however, we find that this structural advantage does not always compensate for the acceptance gains of random sampling paired...