Skip to content

Implementation and Evaluation of BitNet Inference on a CGLA by Signed-Int4 Instructions

Sep 2026 · arXiv.org · Vol abs/2609.27453 · 0 citations · 16 references
Computer Science

TL;DR

It is shown that CGLA-offloaded BitNet C++ execution measures 2.52 tokens/s, and the mapping adds OP_SMA4 as a reusable signed-int4 multiply-accumulate instruction rather than a BitNet-only datapath.

Abstract

Large language model (LLM) inference transfers model weights and activations for every generated token, making memory traffic and its energy cost part of the decode path. BitNet b1.58 represents its low-bit weights by ternary values and uses integer activations. However, this arithmetic does not match conventional int8 or floating-point general matrix multiplication, and existing BitNet accelerators implement it in specialized datapaths. We instead map this operation to a CPU-Grounded Linear Array (CGLA), a programmable ASIC with explicit direct memory access, local memories, and reusable compiler-visible integer lanes. The mapping adds OP_SMA4 as a reusable signed-int4 multiply-accumulate instruction rather than a BitNet-only datapath. Each ternary weight occupies one signed 4-bit lane. Each int8 activation is split into two signed-int4 fragments and reconstructed by shift-and-add. Frequency scaling of the 145 MHz FPGA measurement to an 840 MHz 28 nm CGLA achieved 0.390 ns per signed-int4 product. We showed that CGLA-offloaded BitNet C++ execution measures 2.52 tokens/s.

View source

Similar papers

Aug 2026

Branch-Free Binary128 Multiplication: SRMA Implementation with SSE2 Instructions and Kintex-7 Validation

This research presents an optimized, reduced-clock-cycle approach for performing 128-bit floating-point calculations on Field-Programmable Gate Arrays (FPGAs) using the SRMA architecture, which not only reduces execution time but also decreases energy consumption on Kintex-7 FPGA boards.

N. P, Thirumalaiswamy V., H. S. et al. · 0 citations
Sep 2026

Implementation and Evaluation of NTT Arithmetic for ML-KEM on a CGLA

An eight-stage cyclic radix-2 driver over the ML-KEM modulus is implemented by splitting each twiddle into 8-bit and 4-bit parts before modular reduction, differs from the standardized seven-layer incomplete negacyclic NTT and does not implement the full ML-KEM polynomial multiplication path.

Takuto Ando, Yasuhiko Nakashima · 0 citations
#software testing Open access Sep 2026

Hardware-Assisted Spatial Memory Safety for Shared GPU Accelerators: A Pointer Tagging Co-Design with Analytical and Synthesis-Level Evaluation

This work presents a hardware–software co-design whose 16-bit tag uses odd parity and fail-safe class encoding, which combines a variable-precision extent, an aligned CRC checker, and an exact-bounds micro-cache.

Dan Toderici, T. Enache, R. Rughiniș et al. · 0 citations
Open access Sep 2026

Design and FPGA Evaluation of a Multi-Operand Extension Mechanism for Tightly Coupled RISC-V Processors

Complex scalar kernels often contain instruction fragments whose computation is simple but whose operand demand exceeds the traditional two-input–one-output custom-instruction model. This operand interface bottleneck limits instruction merging in tightly coupled RISC-V acceleration. This paper proposes a multi-operand...

Peng Lu, Mei-Jiao Yu, You-Ping Mao et al. · 0 citations
Open access Aug 2026

Joint Multiplier–Adder Approximation with Flag-Based Error Recovery for BF16 Digital Compute-in-Memory

The Proposed scheme achieved the lowest BF16-referenced Layer NRMSE and Logit NRMSE among the evaluated Baseline, DIMC-S-derived, LSAC OR+SXAFA-derived, and Proposed schemes, while the Top-1 accuracy difference relative to the Baseline remained within −0.02%p to +0.12%p.

Yuhyeon Jin, Munhyeon Kim · 0 citations
Preprint Aug 2026

How Weight Encoding Affects Language Model Placement and Performance on the Apple Neural Engine

This work combines compiler device plans, synchronized memory-controller measurements, and compute-unit exclusion controls for a fixed single-token forward workload and establishes encoding-dependent placement in the measured deployment stack and shows that compression comparisons require joint measurement of backend s...

A. ShahirM · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.