Skip to content
Log in
Book Open access

Parametric Mappings for Distributed-Memory Tensor Computations

Botao Wu Martin Kong
Jul 2026 · International Conference on Supercomputing · 0 citations · 43 references
Computer Science

TL;DR

A new method for automatically generating parametric mappings agnostic of the input problem sizes for the class of DAGs of tensor computations is introduced, based on the insight that in parametric DAGs not all parameters are equally important.

Abstract

Tensor computations are an important class of operations widely used in domains such as computational chemistry, machine learning, and various types of physical simulations that demand distributed-memory clusters. Recent work has shown that generating efficient mappings for multi-operator Directed Acyclic Graphs of distributed-memory tensor computations is possible by leveraging non-linear formulations underpinned by Satisfiability Modulo Theories (SMT) solvers. However, this approach is sensitive to the problem size, grid shape, and count of Processing Elements (PEs) given. We introduce a new method for automatically generating parametric mappings agnostic of the input problem sizes for the class of DAGs of tensor computations. The approach relies on three building blocks: the insight that in parametric DAGs not all parameters are equally important, an offline algorithm to generate a small bounded number of mapping classes – forming a family of mappings – and a sampling space that enables the generation of sufficiently large and diverse family of mapping classes. We show the effectiveness of our parametric mappings on a cluster of multi-GPUs, comparing against NVIDIA cuTensorNet, the Cyclops Tensor Framework (CTF) and PyTorch. Our mappings achieve speedups up to 22.1 × and 2.35 × over NVIDIA cuTensorNet, on matrix-based and tensor-based DAGs, respectively. When benchmarked against PyTorch, we observe speedups of 3.41 ×.

DOI Semantic Scholar
Read PDF

Similar papers

#artificial intelligence Preprint Aug 2026

Iterative tensor network transformations for element-wise evaluation of elementary and filtering functions

Tensor networks are powerful formats for compressing large-scale data. However, their application to general data processing has been limited by the difficulty of performing nonlinear operations. Here, we introduce iterative tensor network transformations (ITNTs), a general algorithmic framework for the element-wise evaluation of elementary and nonlinear filtering functions on data encoded as tensor trains (TTs), a class of tensor networks. Our approach operates entirely in the compressed domain, enabling efficient computation on exponentially large datasets while maintaining a controlled computational cost. We demonstrate its power in two key areas: (I) evaluating highly nonlinear elementary and filtering functions on a 3D reactive flow field, enabling high-fidelity reaction rate computation and region filtering, and (II) finding extrema in complex optimization problems, such as solving Max-SAT instances on spaces up to $2^{70}$ configurations. These results establish ITNT as a foundational tool that provides tensor network methods with the capability for general-purpose data science and large-scale optimization.

Xiao Wang, Tomohiro Hashizume, Pia Siegl et al. · 2 citations
Open access Oct 2025

Efficient Tensor Completion Algorithms for Highly Oscillatory Operators

A novel reformulation of the butterfly structure as a compact tensor network is proposed, leading to speedups of orders of magnitude and significant accuracy improvements for large matrices compared to state‐of‐the‐art low‐rank matrix and QTT tensor completion algorithms.

Navjot Singh, Edgar Solomonik, X. Li et al. · 0 citations
Open access Dec 2024

Randomized Algorithms for Streaming Low‐Rank Approximation in Tree Tensor Network Format

This work presents the tree tensor network Nyström (TTNN), an algorithm that extends recent research on streamable tensor approximation to the more general tree tensor network format, enabling a unified treatment of various existing methods.

Alberto Bucci, Gianfranco Verzella · 3 citations
Book Open access Jul 2026

Optimizing Streaming Tensor Decomposition on GPU

ASTRA is a framework that optimizes the holistic streaming CPD process on GPU that integrates the memory pool and asynchronous strategy with compact storage format to reduce idle cycles and execution latency and devises a similarity-aware updating mechanism to reduce the overall workload.

Wenqing Lin, Jianuo Sheng, Shuqin Feng et al. · 0 citations
Preprint Aug 2026

Structured Learning on Mapper Representations

This work develops a framework for learning over Mapper induced structured representations, and studies mathematical properties of these representations, including invariance under relabeling, a distance functional on the space of Mapper representations, structural complexity of multiscale decompositions, and learning oriented stability under representation perturbations.

G. Babus, Farzana Nasrin · 0 citations
Preprint Aug 2026

Tensor Field Models

TFMs are introduced, realization-level Mathematical Structures in which a learned Operator maps a product of admissible component-section families to a prescribed family of time-dependent tangent sections on a Generative State Manifold.

A. Strunk, Roland Assam · 0 citations