Skip to content
Preprint

Adaptation Fidelity of SPEC CPU2026

Aug 2026 · 0 citations · 24 references
Computer Science

TL;DR

This work offers data-driven validation of SPEC's methodology, showing that the fidelity gap is not a flaw but a quantifiable consequence of enforcing portability, determinism, and CPU-centric measurement.

Abstract

Standardized benchmarks are often criticized for not being"real workloads,"but this critique is rarely backed by data. This paper provides the first systematic, quantitative analysis of the"fidelity gap"between the SPEC CPU2026 suite and its original, upstream open-source counterparts. We compile both the SPEC benchmarks and their upstream applications and execute them with official input workloads under two scenarios: a single-copy latency run and a 192-copy throughput run. Our findings show that most benchmarks exhibit high fidelity in single-copy runs, while a few outliers reveal the impact of SPEC's adaptation process. The multi-copy results further highlight the necessity of this adaptation: several benchmarks become significantly more efficient than their upstream versions under heavy load, underscoring the importance of I/O reduction. This work offers data-driven validation of SPEC's methodology, showing that the fidelity gap is not a flaw but a quantifiable consequence of enforcing portability, determinism, and CPU-centric measurement.

View source

Similar papers

Preprint Sep 2026

Performance Characterization of SPEC CPU 2026 on AMD EPYC 9755 Processor

SPEC CPU 2026 is the first major update to the industry-standard CPU benchmark suite since 2017. This paper presents the first microarchitecture based performance characterization of the new suite, conducted on AMD EPYC"Zen 5", also the first SPEC CPU characterization study on this microarchitecture. Using a multi-lens...

Kunal Kashyap, Rajiv Ramanathan, S. Bhattacharya · 0 citations

T 𝒆𝒙𝑩𝒆𝒏𝒄𝒉 : A Unified Benchmarking Suite for Shifting Workloads

A unified key-value benchmarking suite that enables benchmarking key-value stores against dynamically shifting and production-like workloads and comparing their performance side by side and allows users to benchmark multiple databases and perform an apples-to-apples comparison under the same workload readily within a s...

Abhishek Chanda, Shubham Kaushik, A. Lavrov et al. · 0 citations
#artificial intelligence Preprint Sep 2026

PerfReasoning: How Well Do LLMs Reason on Hardware Performance?

PerfReasoning is introduced, a benchmark that evaluates LLMs both as direct performance reasoners and as generators of analytical performance-model code, exposing the gap between plausible architectural reasoning and reliable performance-model construction.

Da Zhao, K. Sankaralingam, Christos Kozyrakis et al. · 0 citations
Open access Sep 2026

Tools-CC-Bench: a Benchmark Suite for Collective Communication with Compression in HPC and AI Workloads

CC-Bench is presented, a lightweight, extensible, and application-oriented benchmark suite for evaluating communication compression under realistic execution conditions, and uses declarative application-environment modeling to decouple profiling logic from communication libraries, datasets, and fidelity metrics, enabli...

Hao Fan, Wei Wang, Xing-Chen Liu et al. · 1 citation
Open access Aug 2026

Investigating Parallel Scaling Bottlenecks Across Rust, Julia, Haskell, and Python: Workload–Runtime Signatures

A decomposition-based diagnostic framework built on three measured components: slowest-worker computation (Cp), algorithmic merge/combine work (Bp), and residual parallel overhead (Rp) is proposed, which forms a Workload–Runtime Scaling Signature (WRSS).

Muhammad Hassam Aslam Khan, Daniel Stapleton, Medha Kulkarni et al. · 0 citations
Preprint Aug 2026

ClusterBench: A Framework for Cluster-Wide Continuous Benchmarking and Regression Testing

ClusterBench is presented, a framework for cluster-wide continuous benchmarking that ships with a benchmark collection targeting each component: CPU, GPU, memory, interconnect, and I/O, and collects data across space and time.

A. Ujeniya, Jan Eitzinger, Thomas Gruber et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.