Skip to content

UltraGNN: A Sparse-Operator-Aware Framework for Accelerating Graph Neural Networks on Tensor Cores

Nov 2026 · IEEE Transactions on Parallel and Distributed Systems · Vol 37, pp. 2508-2523 · 0 citations · 90 references
Computer Science

Abstract

Graph Neural Networks (GNNs) have achieved widespread success from social networks to AI-for-Science. Most existing GNN frameworks adopt scatter-first (edge-centric) or gather-first (vertex-centric) scheduling paradigms for message passing. However, these paradigms are closely tied to traditional CUDA-core execution model in terms of data access patterns, scheduling granularity, and compute mapping, limiting their ability to fully leverage the superior matrix computation capabilities of Tensor Core Units (TCUs) on modern GPUs. To this end, we propose UltraGNN, the first sparse-operator-aware framework designed to efficiently accelerate both topology-driven and dynamic-weight GNN models on TCUs. UltraGNN introduces a novel subgraph-centric scheduling paradigm, which reformulates message generation and aggregation as TCU-friendly sparse matrix multiplications. To enable efficient support for dynamic-weight GNN architectures on TCUs, UltraGNN incorporates a transpose-friendly sparse storage format explicitly designed to facilitate fast gradient computation and memory-efficient training and inference. UltraGNN adopts PyTorch as its frontend, delivering an out-of-the-box user experience. Extensive experimental results on the H100 and RTX 4090 GPUs demonstrate that UltraGNN sets a new state-of-the-art in GNN acceleration, achieving geometric mean speedups of 2.47× and 5.46× over DGL on GAT and GATv2 models, respectively.

Read PDF

Similar papers

#artificial intelligence Preprint Jul 2024

The Llama 3 Herd of Models

It is found that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks, and performs competitively with the state-of-the-art on image, video, and speech recognition tasks.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri et al. · 21,082 citations · ⚡3,832
#computer vision Review Sep 2017

Agile Software Development Methods: Review and Analysis

This publication proposes a definition and a classification of agile software development approaches and analyses ten software development methods that can be characterized as being "agile" against the defined criterion.

P. Abrahamsson, O. Salo, Jussi Ronkainen et al. · 727 citations · ⚡54
#computer vision Jun 2008

The impact of agile practices on communication in software development

The study shows that agile practices improve both informal and formal communication, but indicates that, in larger development situations involving multiple external stakeholders, a mismatch of adequate communication mechanisms can sometimes even hinder the communication.

M. Pikkarainen, Jukka Haikara, O. Salo et al. · 401 citations · ⚡48
#machine learning Review Open access Oct 2014

Software development in startup companies: A systematic mapping study

The results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context.

Nicolò Paternoster, Carmine Giardino, M. Unterkalmsteiner et al. · 394 citations · ⚡54

Trajectory Balance: Improved Credit Assignment in GFlowNets

It is proved that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution, and empirically demonstrate the benefits of the trajectories balance objective for GFlowNet convergence, diversity of generated samples, and robustness to long action sequenc...

Esmeralda S. Whitammer, Moksh Jain, Emmanuel Bengio et al. · 302 citations · ⚡60

Related blog posts

Microsoft Research Blog Oct 21, 2026

Solving a 31-year-old grand challenge in mathematics – the KLS conjecture

For decades, a fundamental question about the difficulty of computing in high-dimensional spaces has resisted proof. New research resolves this question, removing a major theoretical obstacle to understanding the efficiency of randomized algorithms used across science and computing. The Kannan–Lovász–Simonovits (KLS) conjecture was proposed in 1995. It asks whether, for a broad class of high-dimensional […] The post Solving a 31-year-old grand challenge in mathematics – the KLS conjecture appear…

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.