Benchmarking Python–Rust Integration for High-Performance Computing Tasks

Abstract

High-performance computing (HPC) requirementshave dramatically increased in recent years for a wide range oftasks, including data analytics, machine learning and system optimization. Although Python is a favored language in science and analysis, its dynamic features can cause performance bottlenecks.On the other hand, Rust is a new systems language that guarantees memory safety while offering near-native execution performance, and as such is a prime contender for high-performanceapplications. In this paper we conduct a benchmarking study of Python-Rust interoperability: We benchmark pure Pythonversions against Python code with embedded Rust routines (usingthe PyO3 package) for various tasks like arithmetic calculations,string operations, list operations, file operations and conditionalstatements. Our benchmarking experiments reveal that the Rust enhanced versions systematically execute faster than pure Pythoncode- up to three times faster in some cases. We also analysethe impact of effective language integration and reduced run-timeon more sustainable software engineering practices: by reducingthe overall number of CPU cycles, memory use and energy consumption, hybrid approaches enable more energy-efficient HPCapplications. Finally, we discuss when and how the Python–Rust co-development can be used to create high-performance andenergy-efficient solutions in scientific computing.