TRACE: Transformer Residual Activation Codebook Encoding for Communication-efficient Distributed Edge Inference
Transformer Residual Activation Codebook Encoding (Trace), a framework that eliminates the received data downstream computation and enables communication--computation overlap and reduces a key latency bottleneck in distributed Transformer inference.
Muhammad Azlan Qazi, Alexandros Iosifidis, Qi Zhang
· 0 citations