Characterizing self-speculative decoding approaches for accelerating LLMs

TL;DR

Five self-speculative decoding techniques are characterized across three model sizes and three datasets and recommendations for future research in this area are provided.