AMEND: Audited Margins Enable Nonblocking Drops in GPU-PIM LLM Decoding
AMEND predicts each block's BLASST verdict from margins audited at earlier steps, so the GPU fetches only predicted survivors while near-bank PIM units in high-bandwidth memory (HBM) concurrently score the omitted complement.
Zu-Xiong Tan, Will Wei-Jen Wang, Wei Shao et al.
· 0 citations