Jul 2026· Frontiers in High Performance Computing· Vol 4· 0 citations· 37 references
TL;DR
It is shown that one can use a small number of bits to store most of the values in the output layer without negatively impacting accuracy by carefully quantizing the weights that fall outside a given range.
Abstract
Binary Neural Networks (BNNs) perform competitively in practical image classification tasks. In particular, their accuracy is similar to that obtained by full-precision models tailored to edge devices. BNNs are amenable for devices with constrained resources, since they employ just one bit to store inputs and weights; hence, their storage requirements are low. Moreover, BNN computations utilize XNOR operations, which are implemented efficiently using simple hardware structures. However, supporting BNNs efficiently on edge devices is challenging since they perform many memory accesses to load weights, hindering their benefits. In BNNs, weights and inputs are stored using one bit, and several of them are packed together as a sequence of bits. In this work, the number of unique sequences representing a set of weights is observed to be low. In addition, a small group of unique sequences is seen to be used more frequently than others. Accordingly, this observation is exploited by using variable length encoding to represent the most commonly used sequences and decompressing them during inference. Commonly, BNNs employ floating-point values to store the weights for the output layer, aiming to obtain competitive accuracy. This further magnifies the storage requirements and significantly affects performance. In this work, it is observed that using traditional quantization schemes to quantize these weights negatively affects network accuracy. Then, it is shown that one can use a small number of bits to store most of the values in the output layer without negatively impacting accuracy by carefully quantizing the weights that fall outside a given range. Finally, an efficient quantization scheme for the output layer is proposed. A previously proposed FPGA accelerator is extended with the required hardware to implement the proposed schemes. The proposal is evaluated on a Pynq Z1 device. Our techniques reduce memory requirement by 1.5 × and improve performance by 2.2 × on average.
On edge devices, convolutional neural network (CNN) inference is bottlenecked mainly by memory bandwidth, owing to the frequent memory accesses to feature maps and parameters. To address this challenge, we propose a memory-efficient hardware accelerator for depthwise separable convolution that minimizes off-chip memory traffic and parameter storage. The proposed architecture employs three key techniques: (1) a 64-bit run-length coding (RLC) packet compression that exploits feature-map sparsity after ReLU, (2) a mixed-precision scheme that represents feature maps and weights at different precisions, and (3) separated depthwise and pointwise convolution units. In particular, feature maps are transferred in RLC-compressed form, which reduces the amount of data exchanged with the host. The compressed data are decoded row by row, so the on-chip buffers hold only the rows required for computation rather than a complete feature map. In software simulation on ImageNet, the mixed-precision scheme reduced the parameter storage by 49.22% at a cost of 6.24 percentage points (pp) in Top-1 accuracy, and the RLC reduced the data by up to 56.07% in the deeper layers. Implemented on a Xilinx ZCU-104 FPGA, the proposed accelerator performs the depthwise separable convolution with a small number of logic resources and on-chip memory, confirming its feasibility for resource-constrained edge devices.
Jaeseong Kim, Taehong Min, Chaebin Lee et al.· Electronics· 0 citations
The authors introduce the Edge Suitability Score (ESS), a composite metric that combines normalized accuracy, model size, and inference speed into a single value, weighted at 0.40, 0.35, and 0.25 to reflect their relative importance for microcontroller deployment.
Experimental results demonstrate that effective compression significantly reduces model size and computational cost with minimal performance loss, highlighting the importance of compression-aware design and concluding as a valuable reference for building efficient and scalable AI systems.
Daniel Rodríguez· International Journal of App...· 0 citations
Deep neural network (DNN) weights dominate memory consumption during inference, making their efficient numerical representation particularly important for resource-constrained Edge AI systems. Widely used 8-bit floating-point (FP8) formats, such as E5M2 and E4M3, provide representable ranges that are significantly wider than required for typical DNN weights, which exhibit low variance and a narrow effective support. Consequently, a substantial portion of the available representation range remains unused. Motivated by the observation that exponent bias scales representable values, we establish a general analytical framework linking FP representation with quantization theory by interpreting FP formats as piecewise-uniform quantizers, enabling systematic analysis of exponent bias effects on representation efficiency. The framework formalizes encoding and decoding rules for low-bit FP representations and provides analytical estimates of representation accuracy through derived expressions for mean squared error and signal-to-quantization-noise ratio (SQNR) as functions of data variance. The analysis shows that the maximum achievable SQNR, primarily determined by mantissa precision, is sustained over a variance range whose width is governed by the exponent–mantissa allocation, while its position is controlled by the exponent bias. This enables increased mantissa precision within a fixed bit budget by compensating for reduced variance coverage caused by fewer exponent bits through an appropriate bias selection that aligns the high-SQNR region with the statistical properties of the data. Based on these insights, we propose Bias-Aware FP8 for Weights (BA-FP8-W). Experimental results on ResNet-18 and DenseNet-121 demonstrate that BA-FP8-W achieves higher SQNR than E4M3 and E5M2 and delivers inference accuracy close to the FP32 baseline.
A. Jovanovic, Stefan S. Tomic, Jelena R. Nikolić et al.· IEEE Access· 0 citations
Findings confirm that combining complementary compression strategies yields substantially better performance-efficiency trade-offs than any single technique applied in isolation.
Upma Sharma Archana· International Journal of Res...· 0 citations
Designing energy-efficient and high-performance arithmetic hardware remains a major challenge for modern
computing systems, particularly for the multipliers and compressors that dominate multiply–accumulate (MAC) workloads. This
paper presents an efficient approximate compressor architecture designed to achieve balanced error accumulation in MAC units
used for error-tolerant applications such as image processing and machine learning (ML). The proposed architecture selectively
simplifies the compressor stages that reduce partial products of least numerical significance, while preserving exact computation
for the bits that dominate accuracy, so that approximation error accumulates in a controlled and balanced manner rather than
growing unpredictably across operand widths. Building on this compressor, a configurable multiply–accumulate (MAC) unit for
ML hardware is developed, capable of switching between an exact mode and 2-bit and 4-bit approximate compression modes at
runtime according to instantaneous power and accuracy requirements. A modified, error-compensated activation function is
further proposed to offset the systematic bias introduced by the approximate compressor at the neuron output. The proposed
architecture is implemented in SystemVerilog and synthesized with Synopsys Design Compiler for UMC 90 nm CMOS
technology, with functional validation on FPGA platforms. A Gaussian blur filter is used to evaluate the compressor in an
image-processing pipeline, while a multilayer perceptron (MLP) trained on standard ML datasets evaluates the configurable
MAC unit and the compensated activation function. Experimental results show that the proposed approximate compressor
architecture achieves substantial reductions in power, area, and energy per operation relative to an exact Wallace-tree baseline,
while keeping error metrics such as NMED, MRED, and ER low and well balanced across configurations, making it a strong
candidate for energy-constrained, error-tolerant computing systems
Tirupati Yaswanth, Dr. Suresh Babu Potladurty· International Journal for Re...· 0 citations