It is demonstrated that LLMs provided with specific optimization goals achieve better measured performance and validity rates when generating C code compared to creating computation pipelines and optimization schedules with established frameworks, suggesting that future development should explore alternative approaches for verifiable LLM-guided code optimization.
Abstract
Code performance optimization is a vital aspect of modern software development, as it enables faster response times and reduced resource usage. These optimizations require a deep understanding of low-level hardware details and the intricacies of parallel processing, making them challenging even for experienced developers. With the advent of Large Language Models (LLMs), which are increasingly capable of generating and understanding code, there is growing interest in incorporating these models into automated code optimization processes. Traditionally, this automation involves transcribing the source code into a domain-specific representation that can be auto-tuned using grid search or machine learning algorithms, while adhering to strict rules and a limited set of feasible transformations to ensure verifiability. LLMs incorporate high-level code semantics and can thus perform transformations that go beyond verifiable automated optimizations. This paper investigates whether the traditional abstractions used in automated code optimization improve the performance and correctness of LLM-guided optimizations of parallel HPC applications. We evaluate this using the PolyBench benchmark suite and demonstrate that, in our evaluated setting, LLMs provided with specific optimization goals achieve better measured performance and validity rates when generating C code compared to creating computation pipelines and optimization schedules with established frameworks, suggesting that future development should explore alternative approaches for verifiable LLM-guided code optimization.
Recent advances in Large Language Models (LLMs) have opened opportunities to apply high-level code transformations to the field of code optimization, and it has since emerged as one of the most fundamental tasks for LLMs to perform; however, at present, LLMs struggle to apply wide-ranging code optimization tasks due to both the complexity of the code and the inability to independently verify the correctness of the transformations. In this paper, we present the Trusted LLM (T-LLM) Compiler, which proposes an advancement in compiler technology through a collaborative effort involving high-level LLM code transformations, traditional compilers, and verification tools. Experimental results reveal that it can significantly improve code correctness when tested on a set of PolyBench/C benchmarks. Our approach facilitates iterative code optimization efforts with verification strategies that enable corrective actions. Through this approach, T-LLM Compiler achieves code optimization accuracy of up to 83.3% and a speedup of up to 16.1\% on the PolyBench/C benchmarks, with the transformed code reaching an average of 26.7% speedup wrt standard baselines. Additionally, we release the project's source code to the open-source community.
Zahra Fazel, Sunanda Gamage, Shayan Shirahmad Gale Bagi et al.· 0 citations
Runtime efficiency is a critical factor that impacts both software quality and user satisfaction. There are many approaches proposed for code optimization to improve runtime efficiency. Traditional code optimization methods operate on intermediate representations (IRs) during compilation for static languages. They are effective but struggle to handle dynamic languages that do not require compilation. Recently, large language models (LLMs) have been leveraged to directly optimize source code in dynamic languages. However, these methods fail to identify suitable optimization targets and usually conduct incomprehensive single-level optimization. To address these challenges, we propose Optimo, a multi-level LLM-based code optimization approach built on a novel Mixture-of-Prompts (MoP) architecture. In the MoP architecture, Optimo identifies time-critical code structures as performance bottlenecks via differential profiling. These structures are then routed to some optimization strategies, akin to expert models in MoE, each tailored to optimize specific code patterns. Unlike traditional approaches that focus only on statement-level optimizations, Optimo operates at four levels of abstraction, ranging from coarse-grained algorithmic improvements to fine-grained optimizations in API usage. We evaluate Optimo on two code efficiency benchmarks, COFFE and Effibench. Our results demonstrate that Optimo achieves an up to 57.48% opt%, i.e., the percentage of optimized programs that are correct and at least 10% faster than the original programs, and an up to 3.97x speedup when optimizing human-written code, and it consistently outperforms the best baseline by up to 96.51% in terms of opt%. Furthermore, Optimo achieves an up to 42.42% opt% and an up to 13.51x speedup when optimizing LLM-generated code.
Over the past two years, the rapid evolution of Large Language Models (LLMs) and the relatively slower progress in hardware development have led to the emergence of many operator-level optimization techniques and theories. These software-based methods accelerate model inference by improving memory management and computational efficiency, and they have demonstrated empirical effectiveness. However, in current computer architecture research, hardware designers often focus on microarchitectural optimizations or the performance of individual operations. They tend to adopt existing software optimizations passively, without actively leveraging them as design principles. Constrained by this limited perspective, researchers may overlook opportunities to systematically integrate insights from software into hardware design, potentially hindering more efficient and flexible architectural innovations. To combine operator-level optimization methods with hardware design, we introduce a hardware evaluation platform called LLMSGHD (Large Language Model Software-Guided Hardware Design), which focuses on operator-level optimized LLM inference workloads. LLMSGHD simulates hardware inference behavior while aiming for broad applicability and high efficiency. LLMSGHD integrates advanced software optimization techniques to offer more insightful analysis for hardware design, particularly regarding computational density variations in inference and their interaction with software-level optimizations. Based on LLMSGHD, we develope a heterogeneous LLM inference platform targeting high throughput and low cost.
Qingshan Xue, Sheng Ma, Rui Xu et al.· ACM Transactions on Architec...· 0 citations
To improve a program’s performance and avoid excessive binary-size growth, compiler optimizations can be applied on only the most frequently executed parts of the program, i.e. hot code. Ahead-of-time (AOT) compilation can benefit from profiles to determine the hot code, and to improve optimization decisions with respect to code hotness. In this article, we present a novel technique for reducing the size of AOT compiled programs by using fully-context-sensitive profiles to concentrate the optimizations on the hot code. These profiles enable capturing different behaviors of the same subroutine when invoked from different calling contexts. We use this information to identify those subroutines that contribute significantly to the program’s performance and compile them differently according to their calling contexts. We implemented this technique inside GraalVM Native Image, a state-of-the-art AOT compiler for Java, and performed an evaluation on 29 benchmarks from DaCapo, Scalabench, and Renaissance benchmarking suites. With this selective subroutine specialization, we achieved significant binary-size reductions, ranging from \(11\% \) to \(38\% \) compared to the Enterprise version of GraalVM Native Image with PGO, while retaining similar performance as the O3 optimization level with PGO.
Maja Vukasović, Petar Đekanović, B. Spasojević et al.· ACM Transactions on Architec...· 0 citations
The rise of heterogeneous computing systems has intensified the need for performance-portable programming models and effective autotuning methodologies. Although compiler and runtime tuning are known to significantly influence application performance, it remains unclear how such optimizations transfer across different hardware architectures, particularly within single-source models such as SYCL. This work investigates the transferability of compile-time and runtime autotuning decisions across CPUs and GPUs, focusing on AdaptiveCpp, a SYCL implementation built on LLVM. We introduce an automated framework that jointly explores compiler flags and runtime parameters using both Bayesian optimization and a tabu-search–based strategy. The tool orchestrates compilation, execution, and measurement while also providing statistical attribution via ridge regression to quantify the impact of individual tuning parameters. Through an extensive evaluation of CPUs and GPUs from multiple vendors, we demonstrate that autotuning can deliver substantial performance gains—up to 3 × on CPUs—yet the influence of specific compiler flags often diverges across different architectures. For example, flags such as -fno-builtin yield large improvements on CPUs but have negligible effect on GPUs. We also demonstrate that runtime-level choices, such as thread-placement policies, can significantly affect performance on CPUs. Our findings highlight the challenges and opportunities of autotuning in heterogeneous, single-source programming ecosystems. They also underline the importance of architecture-aware autotuning strategies and motivate further exploration of cross-device performance modeling.
Hari Abram, Nikela Papadopoulou, Jens Domke et al.· International Conference on...· 0 citations
Pointer analysis is a cornerstone of numerous static analysis applications, including compiler optimizations, slicing, bug detection, and verification. While offline simplification is a common approach to boosting performance, existing methods are often tightly coupled to specific analysis algorithms and limited to a set of simplification rules. This paper explores a new perspective: applying semantic-preserving compiler optimizations directly to intermediate representation (IR) before pointer analysis. This strategy is modular, analysis-agnostic, and easily integrates with existing tools. We conduct an empirical study using diverse programs and three pointer analyses. The results show substantial performance gains---up to 3.14x speedup and 1.94x memory reduction---while precision remains largely unchanged. We also analyze the trade-offs between optimization overhead and analysis speedup, quantify changes in IR structure, assess the characteristics of optimization configurations, and identify promising directions for future research.
