KABUMI: Communication-Avoiding Batched GEMM on GPUs
Batched general matrix multiplication (GEMM) operations in latency-sensitive inference workloads often expose limited cooperative thread array (CTA)-level parallelism within each GEMM, making efficient GPU utilization challenging. Independent GEMMs can run concurrently, but this creates a tradeoff: each GEMM should exp...