SL²OT: Scaling LLM Locality on Tile with Distributed 3D-DRAM as Unified Memory
Large language models (LLMs) are increasingly limited by memory bandwidth and latency rather than compute throughput. Our modeled 3D-DRAM system offers 4× the raw bandwidth of the HBM reference. However, high remote-access latency (~3× local latency) and redundant NoC traffic reduce its effective bandwidth below that o...