Torch-PIM: Automated Profile-Guided PIM Offloading for PyTorch
Torch-PIM is presented, a compiler framework that uses profile-guided optimization (PGO) to decide host-versus-PIM placement over the loop nests that progressive lowering materializes, and its offloading decisions yield speedups of up to 8.6x on tensor operators, 2.9x on MLP, 4.4x on Attention, and 3.6x on LLaMA-7B ove...