Over the Memory Wall, Into the Instruction Wall: The New Bottleneck in GPU Data Processing

FitFloat: Read/Write Random-Access Compressed Floating-Point Arrays for GPUs

GPU implementation of a resource-constrained virtual machine

One of the main reasons compute hardware becomes obsolete is software bloat: resource requirements increase for every iteration of a software product. Resource constrained VMs are one way to combat software bloat as they post a hard limit on the resources and so force the programmer to be frugal. In this paper we explore the deployment of one such resource constrained VM, Uxn, on GPU. We show that for competitive performance it is essential to make use of the GPU data parallelism. We present an OpenMP-style parallelism API for Uxntal, the stack-based assembly-style language for the Uxn platform. We demonstrate that exemplar code using our API can run at comparable performance even on an integrated GPU. Specifically, our evaluation results show that using this approach improves performance on the compute-intensive Stencil benchmark with 19x and frame rate on the graphics-intensive Bunnymark benchmark with 7x. In practice, all laptops and desktops and even mobile devices have a GPU and our work shows that they can be used to execute frugal workloads effectively.

Concurrency Response of Plain Global Loads on the NVIDIA H100

The main result concerns the plain-load path: attained LDG bandwidth peaks at a small offered per-thread load (K ~ 2) and then declines, by about 35% from K=2 to K=8 at the authors' primary configuration.

Comparative Performance Analysis of Workload on Enterprise GPUs with Consumer Platforms Accelerated by CUDA Graphs

This work investigates the feasibility of reproducing benchmarks originally run on datacenter GPUs such as the NVIDIA A100 and RTX 8000 using consumer-grade graphics cards, focusing on the NVIDIA GeForce RTX 3050 and GTX 1060 with CUDA Graphs support. Seven NAS Parallel Benchmarks (BT, LU, SP, EP, IS, MG, and CG) are evaluated across problem classes W, A, B, and C. Results show that the RTX 3050 delivers stable performance, typically 6×–12× slower than the A100, while VRAM limitations severely constrain the GTX 1060 for larger-scale problems. Although enterprise GPUs remain essential for massive, memory-bound workloads, modern consumer hardware combined with CUDA Graphs enables economical reproduction of moderate scientific experiments, supporting the democratization of high-performance computing research.

SAI: Virtualizing Shared Memory of GPU for AI workload acceleration

This work proposes SAI, a mechanism that virtualizes shared memory into the L2 cache to improve GPU performance for AI applications and introduces an L2 cache management strategy that integrates associativity-based virtual page allocation and a replacement information table, reducing page-swapping overhead while preserving L2 cache performance.

Experience with NVIDIA GPUDirect Storage (GDS) in Academic HPC Environments: Challenges, Pitfalls, and Practical Limitations

The use of general-purpose GPUs has become essential in modern computing, particularly for workflows that are throughput-bound, e.g., deep learning workflows, LLMs, image segmentation, etc. Because GPUs function as coprocessors, data transfer between the CPU and GPU is constrained by the bandwidth of the interconnect. To mitigate this bottleneck, GPUDirect Storage (GDS) was introduced by Nvidia to enable more efficient data movement and improve overall system performance. GDS allows direct data transfers between storage and GPU memory, bypassing the CPU entirely. Vendors claim significant bandwidth improvements with minimal code changes. In this work, we report our experience deploying GDS on OSCAR, Brown University’s heterogeneous HPC cluster. We deployed and tested GDS across three storage configurations: VAST Data (pNFS over 200G HDR InfiniBand), IBM Spectrum Scale (GPFS over NDR InfiniBand), and local NVMe drives on DGX systems. We executed benchmarks both vendor-provided and a production workload using a Scientific Machine Learning (SciML) benchmark. Our results show that GDS performance depends heavily on file sizes, access patterns, and storage backends. The vendor benchmarks showed improvements in specific scenarios, but these gains did not always translate to the SciML benchmarks. Beyond the performance results, the deployment itself consumed months of staff time, multiple support tickets across NVIDIA, VAST, and IBM, and significant unplanned expenditure on dedicated optical components and cables that vendor planning documents never mentioned. The gains, where they existed, were modest. More critically, application-level support remains immature: PyTorch lists GDS integration as experimental, and at the time of writing, their own tutorial code has been removed from the documentation website. We discussed this issue with PyTorch developers via GitHub, but no clear solution was provided. The feature is currently labeled as experimental and does not appear to be under active development. By sharing our experience, we hope to give other HPC centers realistic expectations for GDS deployments. The technology works, but the path from benchmark to production is longer and more expensive than marketing materials suggest.