FPGAgent is the first task-specification-to-executable HLS generation framework experimentally validated on a well-established benchmark, and the value of end-to-end validation is demonstrated.
Abstract
Large language models (LLMs) have shown substantial promise for high-level synthesis (HLS) code generation, but most existing approaches validate only simulation or synthesis results. Because of timing and place-and-route constraints, \emph{HLS code that passes simulation and synthesis may still fail to produce deployable, runnable designs on real FPGA platforms}. Moreover, the lack of public benchmarks has limited many evaluations to small, self-curated test suites. We propose FPGAgent, a multi-agent framework tailored to real FPGA environments for autonomous HLS coding with end-to-end executability validation. To the best of our knowledge, FPGAgent is \emph{the first task-specification-to-executable HLS generation framework experimentally validated on a well-established benchmark}. Given a natural-language task specification, FPGAgent injects HLS-specific knowledge and employs evolutionary search to iteratively derive reliable HLS kernel implementations. It then generates a C++ validation program to verify functional correctness, diagnoses potential defects, and guides targeted repairs. Finally, it synthesizes host code for compilation and board-level execution on FPGA hardware. We comprehensively evaluate FPGAgent with five established LLMs on HLS-Eval, a benchmark containing 78 tasks across multiple domains, and verify board-level executability on a real FPGA platform. Compared with existing baselines, FPGAgent improves the synthesizable rate by 16.9% on average, executability by 26.7%, and functional correctness by 30.6%. These results show that FPGAgent substantially improves the practical usability of LLM-based HLS generation and demonstrates the value of end-to-end validation.
FPGA architectures increasingly incorporate domain-specific in-fabric hardblocks to accelerate DL inference, particularly GEMM, which dominates DL computation. To realize the performance gains of these hardblocks, manual RTL design is required: the programmer must understand the hardblock microarchitecture, instantiate them in RTL, and manage tiling and control logic. While programming in C/C++ and using HLS tools has increased the abstraction level and productivity of FPGA engineers, HLS tools do not support code generation for custom hardblocks natively. Prior work has demonstrated that blackbox mechanisms in HLS tools can be used to target custom hardblocks, but this still requires explicit function calls in user-written HLS C and manual creation of RTL IP libraries, significant effort that must be repeated for every layer in a DL model. Furthermore, for DL, an even high-level programming interface, e.g., Pytorch/Keras instead of C/C++, is desirable for improved programmability and user adoption. We present ATLAS, a fully automated flow from a high-level DL model description to a hardware implementation on an FPGA with custom in-fabric DL-optimized hardblocks, requiring no manual RTL design or explicit hardblock instantiation from the end user. Our approach uses GEMM as a universal abstraction layer and comprises two components: (1) hls4ml-GEMM, a compiler frontend that transforms DL layers into HLS C code with architecture-agnostic GEMM function calls, and (2) a GEMM IP Generator, an architecture-aware backend that produces hardblock-based RTL wrappers with tiling logic, control FSMs, and scheduling metadata. We evaluate the flow across 11 DL designs, including individual fully connected, convolution, and attention layers, as well as full CNN, MLP, and Transformer models targeting an FPGA architecture with Tensor Slices using Catapult for HLS and VTR for implementation.
Application-specific FPGA accelerators offer substantial performance and energy-efficiency gains across many application domains, but developing them is costly, often requiring months of specialized effort. Even with high-level synthesis (HLS), designers still need extensive hardware expertise to build high-performance accelerators. Although large language models (LLMs) have demonstrated strong software-generation capabilities, even frontier models lack the hardware intuition and procedural knowledge needed to reliably translate baseline C/C++ programs into high-performance HLS designs: they struggle to identify effective architectures, follow the optimization processes used by HLS experts, and apply hardware transformations consistently across diverse kernels. We present HLSmith, an expert-guided framework for translating C/C++ programs into optimized HLS accelerators. HLSmith combines three components: an HLS optimization expertise library that encodes guarded transformation recipes, their applicability and prerequisite conditions, and unsafe cases to avoid; a staged, feedback-driven orchestration flow modeled on expert HLS development practice that guides agents through synthesis, bottleneck analysis, and optimization; and a tool-grounded model-adaptation pipeline that converts optimization trajectories from commercial frontier models into training data for fine-tuning open-weight LLMs. We evaluate HLSmith on PolyBench against ChatHLS, a leading prior agent-orchestration framework for HLS accelerator development. HLSmith achieves a geometric mean speedup of 4.24x over ChatHLS while producing functionally correct designs, in both software and RTL simulation, for every benchmark, compared with ChatHLS's 57% valid-design rate. It further reaches speedups of up to 252x and 138x with commercial frontier models and open-weight models, respectively.
Yuebo Luo, Ahmad Sedigh Baroughi, Philip Stachura et al.· 0 citations
As FPGA design complexity increases, the correctness and reliability of logic synthesis tools are critical to ensuring correct hardware implementation. These tools translate hardware description languages (e.g., Verilog) into gate-level netlists, where latent faults may introduce functional errors or performance degradation during synthesis. Existing approaches rely on automatically generated Verilog test cases to find these latent faults. However, their effectiveness depends heavily on generator configurations and is typically guided by input diversity, which fails to accurately capture differences in synthesis behavior. Moreover, the high-dimensional configuration space of generator further hinders efficient exploration. To address these challenges, we propose SynaSpace, a behavior-driven configuration optimization framework for fault detection in logic synthesis tools. SynaSpace focuses on synthesis behavior coverage to guide configuration search, by constructing behavioral representations through joint analysis of synthesis logs and gate-level netlists. The framework comprises four components: (1) configuration space modeling for unified parameter representation; (2) Bayesian optimization–based configuration search for efficient exploration; (3) synthesis behavior characterization and coverage evaluation for capturing and quantifying behavioral differences; and (4) fault detection and utility modeling for extracting effective feedback via differential testing and deduplication. These components are integrated into a unified optimization framework to enable efficient configuration exploration and improved testing effectiveness. We evaluate SynaSpace on two established logic synthesis tools (i.e., Vivado and Yosys). SynaSpace identifies 18 faults across four categories, all of which have been confirmed and fixed by vendors and the open-source community.
Peiyu Zou, Xiaochen Li, Yijia Meng et al.· ACM Transactions on Design A...· 0 citations
Modern FPGA computer-aided design (CAD) flows have grown increasingly complex, integrating numerous stages, configuration parameters, timing constraints, and physical implementation specifications. As designs scale, failures often arise from subtle interactions across command-line options and constraint files, making debugging time-consuming and heavily dependent on expert knowledge. Identifying the root cause of such failures and determining the appropriate corrective action remains a major productivity bottleneck in CAD workflows. This paper presents VTR-LLM, a fully automated, multi-agent framework for diagnosing and resolving failures in the Verilog-to-Routing (VTR) CAD flow. VTR-LLM leverages large language models (LLMs) in combination with retrieval-augmented generation (RAG) and specialized agents that target distinct sources of errors, including command-line invocations, timing constraints (i.e., Synopsys Design Constraints or SDC), and floorplanning specifications. A Classification Agent dynamically assigns each failure to the most appropriate agent and supports sequential resolution for compound failures involving multiple error sources. The system leverages LLMs without requiring fine-tuning, enabling use of the latest models such as GPT-OSS-120B, by using RAG and intelligent agents to add domain-specific context and behaviours. We evaluate VTR-LLM using a dataset of 92 distinct VTR failure cases spanning multiple error patterns and levels of complexity. VTR-LLM can resolve 95% of all failures fully automatically, with robust performance across error categories. Additional studies demonstrate the impact of documentation retrieval scope, tool iteration, and LLM model choice on resolution accuracy and inference cost.
Mohamed A. Elgammal, Jamie Wu, Lynne Liu et al.· ACM Transactions on Reconfig...· 0 citations
Rapid advancements in LLMs have driven the adoption of generative AI in domains like Electronic Design Automation (EDA). Within the field of software development, EDA presents unique challenges derived from specific requirements of generated RTL code; RTL code must not only be syntactically correct and functionally accurate, but also synthesizable by hardware generators, while matching performance, power and area (PPA) constraints. These additional requirements introduce complexities that existing code-generation benchmarks often fail to capture, limiting their effectiveness in evaluating LLMs for RTL generation. To address this gap, we propose TuRTLe, a unified evaluation framework designed to systematically assess LLMs across key RTL generation tasks. TuRTLe integrates multiple existing benchmarks and automates the evaluation process, enabling a comprehensive assessment of LLM performance in syntax correctness, functional correctness, synthesis, PPA optimization, and exact line completion. Using this framework, a diverse set of forty open LLMs are assesed, tracking their strengths and weaknesses in EDA-specific tasks. Our results identify the best match for specific tasks (e.g., base models are better in module completion tasks, instruct-tuned models are better in specification-to-RTL tasks), while finding that recent models with autoregressive reasoning chain perform the best overall. We also analyze common compiler and runtime failures, study correlations between benchmarks and evaluation goals, and investigate potential training-data contamination in existing RTL datasets. These analyses provide further insight into the capabilities and limitations of current benchmarks for RTL generation.
Miquel Albertí-Binimelis, Cristian Gutierrez-Gomez, D. García-Gasulla et al.· ACM Transactions on Design A...· 0 citations
A reproducible benchmarking platform that evaluates open-source LLMs on Verilog RTL generation across 50 curated tasks consisting of combinational, sequential, finite state machine (FSM), and mixed designs, enabling reproducible evaluation of generative AI for hardware design workflows.