Sep 2026· International Conference on Parallel Processing· 0 citations· 31 references
Computer Science
TL;DR
TileGEMM is proposed, a high-performance GEMM implementation on AMX that systematically enhances data reuse across the memory hierarchy, and achieves average speedups of 3.27 × and 1.96 × over AVX-512-based implementations TVM and MKL, respectively.
Abstract
General Matrix Multiplication (GEMM) is the cornerstone of high-performance computing and deep learning. Its efficiency significantly influences the performance of applications ranging from large language models to scientific simulations. Intel Advanced Matrix Extensions (AMX) significantly boost matrix operations throughput, yet existing implementations underutilize its potential. The core bottleneck is that AMX’s unique microarchitectural constraints break the classic Advanced Vector Extensions 512 (AVX-512) GEMM paradigm, causing critical data reuse loss across the memory hierarchy: inefficient vector register reuse during mandatory layout transformation, poorly reconciled trade-off between Tile register reuse for partial sums and L1 cache locality, and severe buffer reuse degradation due to fine-grained update strategies. To address these challenges, we propose TileGEMM, a high-performance GEMM implementation on AMX that systematically enhances data reuse across the memory hierarchy. First, a dual Tile register layout transformation strategy eliminates redundant memory accesses, maximizing vector register reuse. Second, a multi-level cache- and Tile register-aware tiling strategy adopts a K-dimension-centric model under strict L1 capacity constraints, improving Tile register reuse for intermediate partial sums while preserving L1 cache locality. Third, a coarse-grained buffer update strategy with loop reordering extends transformed data residency, boosting buffer reuse and reducing transformation overhead. Extensive experiments in various matrix configurations show that TileGEMM achieves average speedups of 3.27 × and 1.27 × over AMX-based implementations LIBXSMM and OneDNN, and 10.30 × and 7.96 × over AVX-512-based implementations TVM and MKL, respectively.
This work proposes the segmented fiber tree (SFT) data structure, which extends the conventional fiber tree through further partitioning to better support the dataflow paradigm while enhancing data reuse, and decouples the multiplication and merging phases.
Sheng-Bai Luo, Sheng Ma, Bo Wang et al.· ACM Transactions on Architec...· 0 citations
Sparse Matrix-Vector Multiplication (SpMV) is a critical computational kernel in scientific computing, graph processing, and machine learning. Despite its importance, SpMV achieves only a small fraction of peak performance, varying substantially across matrices and architectures. This paper comprehensively analyzes SpM...
Panagiotis Mpakos, Dimitrios Galanopoulos, Petros Anastasiadis et al.· ACM Transactions on Architec...· 0 citations
This work proposes FIBER, a new architecture that extends the GPU SIMT (single instruction, multiple thread) model, and extends the ISA, microarchitecture, and compiler to realize shared-register addressing, conflict-free operand delivery, and fiber-based program mapping.
Zi-Han Liu, Jingwen Leng, Yang-Jie Zhou et al.· 0 citations
A multi-cluster architecture that replaces inter-cluster communication with localized data replication within ScratchPad Memories (SPMs) is introduced, creating a critical architectural trade-off between the data replication overhead required by overlapping-footprint kernels and the routing delays of larger clusters.
Chanon Khongprasongsiri, P. Tanguy, Kevin J. M. Martin et al.· IEEE International Conferenc...· 0 citations
Graph Neural Network (GNN) inference involves two successive matrix operations per layer: a sparse neighbor aggregation (SpMM) followed by a dense linear transformation (GeMM). The conventional two-step execution materializes a large intermediate matrix Z in main memory, incurring significant memory traffic that domina...
Xiao Yan, Haodong Bian, Xiao-Ying Wang et al.· International Conference on...· 0 citations
A hand-optimized GEMV kernel for AMD Zen 3, implemented using AVX2 (Advanced Vector Extensions 2) and FMA (fused multiply-add) intrinsics is presented, achieving up to 1.37× speedup over OpenBLAS and 1.20× over AOCL-BLAS.
Rafał Lenart, B. Bylina, J. Bylina· Conference on Computer Scien...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.