Taking Tokenization off the Host for Agentic LLM Serving
GraphTok is presented, a GPU tokenizer that runs the entire encoding pipeline, from pre-tokenization through byte-pair encoding (BPE) merging to token-ID assembly, as one replayable CUDA graph.
Mohammad Siavashi, Pascal Schwinte, A. Banaei et al.
· 1 citation