This is the first study in which a fully three-dimensional computational fluid dynamics simulation has been employed to train a DRL model for active flow control in wings, suggesting promising directions for extending DRL-based strategies to higher Reynolds numbers and more complex wing configurations where prior physical knowledge may be limited.
Abstract
Abstract Content of image described in text. This study investigates the application of deep reinforcement learning (DRL) for active flow control in a three-dimensional NACA0012 wing section with periodic spanwise boundary conditions at low Reynolds number ( italic Re Subscript c Baseline equals 1000 Rec=1000
$ \textit{Re}_c=1000$
) and high angle of attack ( AoA equals 20 Superscript ring AoA=20∘
$\text{AoA}=20^\circ$
), where the flow exhibits massive separation, strong vortex shedding and a chaotic three-dimensional wake. The baseline configuration is validated against previous numerical studies, showing the accuracy of the computational set-up. Two DRL control policies are trained with different reward functions, leading to fundamentally different control strategies. The DRL set-up uses a multi-agent reinforcement learning framework to coordinate distributed actuators along the wing span. In the first case study, when the reward prioritises drag reduction, agents delay the leading-edge separation and stabilise the wake, reducing drag by 21.1 percent sign 21.1%
$21.1\,\%$
and lift fluctuations by 54.1 percent sign 54.1%
$54.1\,\%$
while maintaining baseline lift. In the second case, when the reward targets lift enhancement instead, agents synchronise actuations with vortex shedding, forming leading-edge instabilities that increase lift by 42.3 percent sign 42.3%
$42.3\,\%$
, albeit at the cost of higher fluctuations and slightly increased drag. A comparison with classical open-loop control further demonstrates that DRL autonomously determines physically meaningful and interpretable strategies consistent with established mechanisms in the literature. To the best of the authors’ knowledge, this is the first study in which a fully three-dimensional computational fluid dynamics simulation has been employed to train a DRL model for active flow control in wings, suggesting promising directions for extending DRL-based strategies to higher Reynolds numbers and more complex wing configurations where prior physical knowledge may be limited.
A framework based on the twin-delayed deep deterministic policy gradient algorithm is coupled with a validated numerical simulation of a two-bladed VAWT, aimed at maximising power generation.
Jarno Platenburg, Brice Martin, Thierry Jardin et al.· Journal of Fluid Mechanics· 0 citations
This work investigates Deep Reinforcement Learning (DRL) as a tool for model-free closed-loop active separation control in a fully turbulent wind tunnel flow over a one-sided diffuser. The agent controls an array of magnetic valves (on/off) that eject compressed air into the boundary layer, while the environmental state is reduced to the signal from a single wall-shear-stress sensor placed near the natural transitory detachment point. The control law is learned in real time using Proximal Policy Optimization. Compared to the standard learning design based on the weighted sum of all rewards following an action, we demonstrate that a horizon aligned with the convective time of the flow leads to faster convergence and a more robust control strategy. The resulting control law corresponds to a low-duty-cycle actuation pattern that yields a forward-flow fraction of approximately $53\%$. This compares favorably with conventional and optimized periodic open-loop control ($\sim 40\%$ and $\sim 51\%$, respectively). The findings of this article indicate that, when embedded into an online experiment, DRL represents an efficient tool to identify robust and interpretable active separation control strategies.
Sofia Avdiiv, Andre Weiner, B. Steinfurth· 0 citations
HydroGym is introduced, a solver-independent reinforcement learning platform providing more than 60 validated, openly available flow control environments spanning from canonical laminar flows to complex turbulent flows, with systematic progression in the Reynolds number up to Re = 4 × 105, and Mach number variations in two and three dimensions.
Christian Lagemann, Sajeda Mokbel, Miro Gondrum et al.· Nature· 1 citation
Closed-loop wall controllers learnt by multi-agent reinforcement learning are usually trained on periodic boxes far smaller than the flows they are meant to drive, and a large part of their drag reduction is lost when they are carried across. Retraining on the target domain is not an affordable remedy: the centralised critic that assigns credit to each wall patch degrades as patches are added, the zero-net-mass constraint couples the patches it is asked to separate, and the episodes must be collected in sequence at a cost that grows with the domain. We propose instead a short gradient-free refinement stage, applied to the transferred policy on the domain it will drive. An evolution strategy scores whole flow episodes against a regularised objective, so no credit has to be assigned to individual patches, and the candidates in a generation are independent and run in parallel. Applied to a recurrent multi-agent policy trained on a minimal flow unit at $\Retau\simeq180$ and evaluated on a channel sixteen times larger in wall-parallel area, a few generations raise the drag reduction from $19.0\%$ to $25.7\%$, above the $22.5\%$ of opposition control. Because the policies before and after refinement share an architecture and a training history, the difference between the controlled flows follows from the refinement alone. It shows in the friction decomposition, in the Reynolds stresses and in the near-wall spectra, and the actuation moves from a weak coupling to the wall-normal velocity towards a strong coupling to the streamwise fluctuation.
Giorgio Maria Cavallazzi, Miguel Pérez Cuadrado, Alfredo Pinelli· 0 citations
PEARL employs an actor-adjoint algorithm that leverages automatic differentiation to compute policy gradients over short horizons and adjoint-based sensitivities of future returns approximated via neural networks, significantly reducing the number of environment interactions, while mitigating long-term gradient instabilities.
Matteo Tomasetto, Nicolò Botteghi, Gabriele Bruni et al.· 0 citations
Deep reinforcement learning for autonomous unmanned aerial vehicle control has largely been demonstrated with multirotor platforms and high-level machine-learning frameworks. This study presents a Dueling Double Deep Q-Network (D3QN) training pipeline implemented in Rust without an external machine-learning library and integrated with Godot 4 through GDExtension for fixed wing flight control. The controller addresses fixed-wing requirements, including airspeed maintenance, lift management, throttle regulation, stall avoidance, and coordinated turning. The network combines a duelling architecture, double Q-learning, prioritised experience replay, and three-step returns in a 512→256 hidden-layer configuration containing 142,088 parameters for the 16-dimensional input case. The agent selects among seven discrete actions and supports both 12-dimensional and 16-dimensional observation spaces through a cross-dimensional weight-transfer procedure. Training was conducted for 200 episodes using four random seeds. Across seeds, the mean best episodic reward was 4825±40, while the coefficient of variation for best reward was 0.8%. In the final 30 episodes, no crashes were recorded, although completion rates varied substantially between seeds.
Airspeed remained within ±8 m/s of the 50 m/s target. Batch-64 gradient updates required less than 1 ms, representing an approximately 35-fold reduction in latency relative to the preceding GDScript implementation, and the reported runtime memory footprint remained below 50 MB. These findings support the feasibility of native Rust-based D3QN training for real-time fixed-wing simulation, while the observed inter-seed variability indicates that reward shaping and convergence robustness require further evaluation.
Saugat Chaudhary Tharu, Shrutika Ojha, Rija Bhomi et al.· Journal of Advances in Mathe...· 0 citations