It is shown that CGLA-offloaded BitNet C++ execution measures 2.52 tokens/s, and the mapping adds OP_SMA4 as a reusable signed-int4 multiply-accumulate instruction rather than a BitNet-only datapath.
Abstract
Large language model (LLM) inference transfers model weights and activations for every generated token, making memory traffic and its energy cost part of the decode path. BitNet b1.58 represents its low-bit weights by ternary values and uses integer activations. However, this arithmetic does not match conventional int8 or floating-point general matrix multiplication, and existing BitNet accelerators implement it in specialized datapaths. We instead map this operation to a CPU-Grounded Linear Array (CGLA), a programmable ASIC with explicit direct memory access, local memories, and reusable compiler-visible integer lanes. The mapping adds OP_SMA4 as a reusable signed-int4 multiply-accumulate instruction rather than a BitNet-only datapath. Each ternary weight occupies one signed 4-bit lane. Each int8 activation is split into two signed-int4 fragments and reconstructed by shift-and-add. Frequency scaling of the 145 MHz FPGA measurement to an 840 MHz 28 nm CGLA achieved 0.390 ns per signed-int4 product. We showed that CGLA-offloaded BitNet C++ execution measures 2.52 tokens/s.
This research presents an optimized, reduced-clock-cycle approach for performing 128-bit floating-point calculations on Field-Programmable Gate Arrays (FPGAs) using the SRMA architecture, which not only reduces execution time but also decreases energy consumption on Kintex-7 FPGA boards.
N. P, Thirumalaiswamy V., H. S. et al.· International Journal of Com...· 0 citations
An eight-stage cyclic radix-2 driver over the ML-KEM modulus is implemented by splitting each twiddle into 8-bit and 4-bit parts before modular reduction, differs from the standardized seven-layer incomplete negacyclic NTT and does not implement the full ML-KEM polynomial multiplication path.
This work presents a hardware–software co-design whose 16-bit tag uses odd parity and fail-safe class encoding, which combines a variable-precision extent, an aligned CRC checker, and an exact-bounds micro-cache.
Dan Toderici, T. Enache, R. Rughiniș et al.· Computers· 0 citations
Complex scalar kernels often contain instruction fragments whose computation is simple but whose operand demand exceeds the traditional two-input–one-output custom-instruction model. This operand interface bottleneck limits instruction merging in tightly coupled RISC-V acceleration. This paper proposes a multi-operand...
Peng Lu, Mei-Jiao Yu, You-Ping Mao et al.· Electronics· 0 citations
The Proposed scheme achieved the lowest BF16-referenced Layer NRMSE and Logit NRMSE among the evaluated Baseline, DIMC-S-derived, LSAC OR+SXAFA-derived, and Proposed schemes, while the Top-1 accuracy difference relative to the Baseline remained within −0.02%p to +0.12%p.
This work combines compiler device plans, synchronized memory-controller measurements, and compute-unit exclusion controls for a fixed single-token forward workload and establishes encoding-dependent placement in the measured deployment stack and shows that compression comparisons require joint measurement of backend s...
A. ShahirM· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.