Sep 2026· IEEE Transactions on Very Large Scale Integration (vlsi) Systems· Vol 34, pp. 2807-2817· 0 citations· 25 references
Abstract
Graph convolutional networks (GCNs) have demonstrated strong capability in modeling graph-structured data and have been widely adopted in various industrial applications. With the rapid growth of edge computing, designing GCN accelerators that maintain high performance under strict resource constraints has become increasingly important. This article presents SPP-GCN, a hardware–software co-optimized accelerator that significantly improves memory efficiency (ME) while maintaining competitive performance. SPP-GCN introduces four hardware–software co-optimization strategies—shuffle, z-merge, concatenation, and preplanning cache—to leverage software-side preprocessing and enhance the graph awareness of the hardware execution pipeline. These techniques effectively improve processing-element utilization and reduce on-chip memory footprint. Evaluated on both GCN and GraphSAGE models across multiple benchmark datasets, SPP-GCN achieves up to $86.9\times $ and $18.85\times $ latency reductions compared with CPU and GPU implementations, respectively. Compared with a prior lightweight accelerator, LW-GCN, SPP-GCN further reduces on-chip memory consumption by 58.5% while achieving up to $1.27\times $ higher relative performance (RP) under the adopted CE/RP-normalized comparison methodology. In addition, although several existing field-programmable gate array (FPGA) accelerators (e.g., BoostGCN, FP-GNN, and Graph-OPU) provide competitive latency, they require significantly larger on-chip memory resources. To better characterize performance density under resource constraints, we further introduce a ME metric that quantifies the RP achieved per unit of on-chip memory. Experimental results show that SPP-GCN consistently achieves the highest ME under the proposed ME metric among the evaluated FPGA-based GCN accelerators. These results demonstrate that the proposed hardware–software co-optimization framework effectively balances performance and resource utilization, making SPP-GCN particularly suitable for memory-constrained edge platforms.
Edge intelligence applications demand high-throughput and energy-efficient hardware accelerators under strict power, latency, and resource constraints in real-time environments. Conventional compute-centric architectures rely heavily on sequential multiply-accumulate (MAC) operations and frequent off-chip memory access, resulting in high energy consumption, increased latency, and bandwidth bottlenecks. These limitations become more pronounced as neural network models scale in size and complexity. While Ternary Content-Addressable Memory (TCAM) offers an alternative through parallel associative search and constant-time pattern matching, it suffers from excessive power consumption and limited adaptability due to continuous match-line activation. This work proposes a hybrid SRAM-TCAM memory-centric deep neural network (DNN) architecture that integrates three tightly coupled subsystems. First, a two-layer autoencoder with ReLU activations compresses 128-bit input feature vectors to 32-bit latent codes, achieving a compression ratio η=4×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\eta = 4\times $$\end{document} and a 75%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$75\%$$\end{document} BRAM saving. Second an on-chip dual-port BRAM data-locality buffer exploits the compressed representation to eliminate inter-layer off-chip DRAM accesses in the critical inference path. Third, a 256×32\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$256 \times 32$$\end{document}-bit Ternary Content-Addressable Memory array provides O(1) parallel associative inference. A software-only Genetic Algorithm (GA) with Roulette Wheel Selection (RWS) optimizes TCAM configuration offline prior to FPGA deployment; the converged 8,480-bit chromosome is loaded once at startup and remain static during inference, adding zero dynamic power overhead. The proposed architecture, implemented on a Xilinx Artix-7 FPGA, achieves a 42.3%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$42.3\%$$\end{document} reduction in power compared to a conventional MAC-based baseline (120.5 mW vs. 208.7 mW), an end-to-end throughput of 108.47 Mops reflecting steady-state pipelined operation at 108.47 MHz (2.42×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2.42\times $$\end{document} speedup), and an inference accuracy of 95.3%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$95.3\%$$\end{document} – a 1.5%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1.5\%$$\end{document} degradation attributable to quantization, evaluated on the benchmark dataset. Power figures are obtained from the Vivado Power Estimator tool (post-implementation) at nominal supply voltage of 1 V. These results demonstrate an efficient and scalable solution for next-generation edge AI inference systems.
Vandana Thakur, V. More, Abhishek Bhatt· Journal of King Saud Univers...· 0 citations
Domain-specific hardware accelerators provide significantly higher performance and energy efficiency for deep neural network (DNN) workloads than general-purpose processors, but often lack adaptability to evolving model architectures. In contrast, general-purpose ISA-based solutions, such as RISC-V-based accelerators, improve programmability at the cost of efficiency. This work addresses this tradeoff by introducing a machine-learning-oriented instruction set architecture (ISA) and a reconfigurable hardware platform that combine high efficiency with flexibility. The proposed ISA enables fine-grained control over data movement, dynamic precision, and decoupled execution across data-fetching, tensor processing, and post-processing domains. The corresponding architecture employs lightweight programmable cores and SIMD units to maintain high processing-element utilization with low control overhead, while remaining independent of the underlying numerical representation. We demonstrate the approach using a Residue Number System (RNS) instantiation supporting 3-8-bit dynamic precision. A 22-nm implementation achieves 5.12-10.47 TOPS/W for a typical workload and up to 1.2x higher energy efficiency than its fixed-point counterpart, while preserving model accuracy. It also outperforms state-of-the-art and mixed-precision accelerators. These results show that the proposed design effectively bridges the gap between efficiency and programmability in modern DNN accelerators.
Vasilis Sakellariou, Vassilis Paliouras, I. Kouretas et al.· 0 citations
Graph convolutional networks (GCNs) excel in many applications, but scaling to large graphs is bottlenecked by heavy data movement. Existing in-storage processing (ISP) solutions offload I/O-intensive operations to the SSD controller to reduce PCIe traffic, but limited parallelism and flash bandwidth still constrain energy efficiency, even with accuracy-degrading neighbor sampling. We propose Tris-GCN, a 3D NAND-based ISP design that executes core GCN computations in situ by leveraging inherent flash computing capabilities, reducing channel traffic without accuracy loss. It incorporates mapping and scheduling optimizations for energy efficiency, alongside wear-leveling with selective recomputation for reliability. Results show that Tris-GCN achieves average 11.7× speedup (up to 41.0×) and 99.7% energy savings over CPU baselines, and 2.45× speedup and 63.8% energy savings over the SOTA ISP on the Amazon dataset.
Yi-Wa Wu, Jia-You Li, Chi-Jung Chen et al.· Proceedings of the ACM/IEEE...· 0 citations
Large language models (LLMs) place unprecedented and still-growing demands on the hardware that trains and serves them. This review surveys the full landscape of AI hardware accelerators for LLMs, including general-purpose GPUs, custom ASICs such as TPUs, Trainium, Groq, and Cerebras, reconfigurable FPGAs, processing-in-memory and near-memory architectures, and emerging neuromorphic and photonic approaches across cloud and edge deployment. Using the transformer's computational structure and roofline analysis as a common framework, we show that the decisive constraint on LLM acceleration is not arithmetic but memory: the autoregressive decode phase is bandwidth-bound, the key-value cache can rival the model weights in size, and data movement dominates energy. Comparing platforms on compute, memory, energy, programmability, and scalability, we find that no single architecture is optimal across workloads: GPUs remain the flexible default and the workhorse of training; domain-specific ASICs win at scale for stable, high-volume workloads; processing-in-memory is the most promising near-term response to the memory wall, entering systems as a heterogeneous complement; and neuromorphic and photonic computing, while promising, are not yet production-ready at frontier scale. Future progress depends on hardware-algorithm co-design and heterogeneous, memory-centric systems: for large language models, the memory system has become the computer.
Memory row conflicts (MRCs) continue to be a major bottleneck that results in higher latency, ineffective double data rate (DDR) usage, and decreased effective bandwidth in field-programmable gate array (FPGA-based) convolutional neural network (CNN) accelerators. The majority of current effort focuses on computational optimization, frequently ignoring inefficient memory access. In order to reduce memory reference codes (MRCs), this research suggests a hardware-level data layout technique using a memory-centric accelerator architecture. In order to improve hit rates and row buffer locality, the architecture incorporates a dynamic cursor-based address mapping method that adjusts to different feature map sizes across CNN layers and a dual-DDR setup for concurrent data access. The experimental results on VGG16, YOLOv2, and AlexNet show an 18% reduction in MRCs, a 40% increase in throughput, and a 23% decrease in latency compared to state-of-the-art techniques. The design uses a Xilinx Kintex-7 FPGA with low power usage of 1.52 W. The suggested method improves memory performance in FPGA-based CNN accelerators in a scalable and hardware-efficient manner without requiring a large computational burden.
S. Prasad, Suman Jayakumar, Bellary Kursheed et al.· Bulletin of Electrical Engin...· 0 citations
In recent years, Large Language Models (LLMs) have exhibited remarkable capabilities, driving advancements in real-world applications. However, training LLMs on increasingly long input sequences imposes significant challenges due to high GPU memory and computational demands. Existing solutions face two key limitations: (1) memory reduction techniques, such as activation recomputation and CPU offloading, compromise training efficiency; (2) distributed parallelism strategies require excessive GPU resources, limiting the scalability of input sequence length. To address these gaps, we propose Adaptive Sequence Pipeline Parallel Offloading (SPPO), a novel framework that optimizes memory and computational resource efficiency for long-sequence LLM training. SPPO introduces adaptive offloading, leveraging sequence-aware offloading, and two-level activation management to reduce GPU memory consumption without degrading the training efficiency. Additionally, it develops an adaptive pipeline scheduling approach with a heuristic solver and multiplexed sequence partitioning to improve computational resource efficiency. We perform comprehensive experiments to demonstrate the superiority of SPPO: it achieves up to 3.38 × throughput improvement over state-of-the-art methods like Megatron-LM, DeepSpeed and TeraPipe, realizing efficient training of a 7B LLM with sequence lengths of up to 4M tokens on only 128 A100 GPUs.
Qiaoling Chen, Shenggui Li, Wei Gao et al.· International Conference on...· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduSep 2, 2026
What if pathology foundation models could do more with less? GigaPath-Flash and GigaTIME-Flash cut computational demands while maintaining strong performance, opening the door to larger studies and broader exploration. The post GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduAug 31, 2026
With millions of users across the world, Julia has been used to conduct cutting-edge research and to design new drugs, jet engines, heat pumps, and more.
MIT News · Artificial Intelligence· news.mit.eduAug 27, 2026
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
