Hardware-Aware Reinforcement Learning-Based State of Charge Estimation for Lithium-Ion Batteries: A Cross-Platform Evaluation of Fixed- and Floating-Point Implementations
TL;DR
A three-stage pipelined SoC estimation framework incorporating a reinforcement learning primary stage, least squares boosting (LSB) secondary residual corrector, and ultimate linear three-point interpolation (3p-InT) stage that can be generalized across both profiles without requiring dataset-specific re-tuning is proposed.
Abstract
Accurate state of charge (SoC) estimation is of primary importance in terms of safe and efficient management of energy storage systems (ESSs). In this regard, data-driven frameworks offer the advantage of rapid execution during online operations. Nevertheless, their deployment on resource-constrained embedded systems is often hindered by the strict memory and processing limitations of low-cost hardware. This article proposes a three-stage pipelined SoC estimation framework incorporating a reinforcement learning (RL) primary stage, least squares boosting (LSB) secondary residual corrector, and ultimate linear three-point interpolation (3p-InT) stage. The RL phase utilizes a twin deep delayed deterministic policy gradient neural network (TD3NN)-based agent along with a customized reward function. The inference part of the framework is deployed on two separate embedded platforms, i.e., an STM32F411RE microcontroller (MCU) and Digilent Nexys A7-100T FPGA through automatic C code and hardware description language (HDL) code generation features in MATLAB/Simulink, respectively. The efficacy of the proposed framework is evaluated using a Panasonic 18650 lithium-ion battery (LiB) and a battery-powered drill load profile (BPD-LP). Across the four hardware scenarios, the accuracy of the proposed framework is preserved, with the maximum %RMSE deviation not exceeding 0.08 percentage points. The RMSE value remains within 1.80–1.82% for the LiB dataset and within 0.76–0.84% for the BPD-LP, irrespective of the platform or the arithmetic format. As for the resource footprint, the fixed-point implementation more than halves the FPGA logic with respect to the floating point (27.33% against 65.55% of the LUTs), at the cost of a comparatively higher DSP usage (15% against 7.08%). On the MCU, it trades additional flash memory (31.51% against 25.23%) for a 2.6-fold smaller RAM footprint. The framework’s reward function, hyperparameters, and architecture are kept unchanged across both datasets, indicating that the same configuration can be generalized across both profiles without requiring dataset-specific re-tuning. Moreover, the detailed hardware deployment findings provide a practical insight into the hardware and arithmetic format selection for an accurate embedded SoC estimation framework.