Deep Reinforcement Learning-Based Active Damping for Engine Torque Ripple Reduction in 48V Mild HEVs

Abstract

Torque ripple of internal combustion engine causes a decline in the combustion stability, increase of cyclic variation and more emissions especially for 48V-mild hybrid electric vehicle (HEV) under transient conditions. Standard damping methods do not have a straightforward solution for the nonlinear time-varying dynamics associated with the engine-crankshaft system. This paper develops approximately a new deep reinforcement learning (DRL)-based control approach that uses the 48V active damping controller to immediately cancel engine torque ripple online as well. The DRL agent, trained on soft actor-critic (SAC) architecture, optimizes a reward function that explicitly penalizes both torque ripple amplitude and the coefficient of variation (COV) of indicated mean effective pressure (IMEP). Simulation results on a validation mild HEV model demonstrated that the proposed method reduced peak-to-peak torque ripple from 76.0 N·m to 22.0 N·m (71% reduction) and COV of IMEP from 4.8% to 2.1% across transient driving cycles compared with conventional PID control on average. The results contribute to increased combustion stability. This will allow the engine to operate more cleanly and efficiently with no changes to hardware.