Closed-loop wall controllers learnt by multi-agent reinforcement learning are usually trained on periodic boxes far smaller than the flows they are meant to drive, and a large part of their drag reduction is lost when they are carried across. Retraining on the target domain is not an affordable remedy: the centralised critic that assigns credit to each wall patch degrades as patches are added, the zero-net-mass constraint couples the patches it is asked to separate, and the episodes must be collected in sequence at a cost that grows with the domain. We propose instead a short gradient-free refinement stage, applied to the transferred policy on the domain it will drive. An evolution strategy scores whole flow episodes against a regularised objective, so no credit has to be assigned to individual patches, and the candidates in a generation are independent and run in parallel. Applied to a recurrent multi-agent policy trained on a minimal flow unit at $\Retau\simeq180$ and evaluated on a channel sixteen times larger in wall-parallel area, a few generations raise the drag reduction from $19.0\%$ to $25.7\%$, above the $22.5\%$ of opposition control. Because the policies before and after refinement share an architecture and a training history, the difference between the controlled flows follows from the refinement alone. It shows in the friction decomposition, in the Reynolds stresses and in the near-wall spectra, and the actuation moves from a weak coupling to the wall-normal velocity towards a strong coupling to the streamwise fluctuation.
This work investigates Deep Reinforcement Learning (DRL) as a tool for model-free closed-loop active separation control in a fully turbulent wind tunnel flow over a one-sided diffuser. The agent controls an array of magnetic valves (on/off) that eject compressed air into the boundary layer, while the environmental state is reduced to the signal from a single wall-shear-stress sensor placed near the natural transitory detachment point. The control law is learned in real time using Proximal Policy Optimization. Compared to the standard learning design based on the weighted sum of all rewards following an action, we demonstrate that a horizon aligned with the convective time of the flow leads to faster convergence and a more robust control strategy. The resulting control law corresponds to a low-duty-cycle actuation pattern that yields a forward-flow fraction of approximately $53\%$. This compares favorably with conventional and optimized periodic open-loop control ($\sim 40\%$ and $\sim 51\%$, respectively). The findings of this article indicate that, when embedded into an online experiment, DRL represents an efficient tool to identify robust and interpretable active separation control strategies.
Sofia Avdiiv, Andre Weiner, B. Steinfurth· 0 citations
This is the first study in which a fully three-dimensional computational fluid dynamics simulation has been employed to train a DRL model for active flow control in wings, suggesting promising directions for extending DRL-based strategies to higher Reynolds numbers and more complex wing configurations where prior physical knowledge may be limited.
R. Montalà, B. Font, Pol Suárez et al.· Journal of Fluid Mechanics· 0 citations
HydroGym is introduced, a solver-independent reinforcement learning platform providing more than 60 validated, openly available flow control environments spanning from canonical laminar flows to complex turbulent flows, with systematic progression in the Reynolds number up to Re = 4 × 105, and Mach number variations in two and three dimensions.
Christian Lagemann, Sajeda Mokbel, Miro Gondrum et al.· Nature· 1 citation
A framework based on the twin-delayed deep deterministic policy gradient algorithm is coupled with a validated numerical simulation of a two-bladed VAWT, aimed at maximising power generation.
Jarno Platenburg, Brice Martin, Thierry Jardin et al.· Journal of Fluid Mechanics· 0 citations
Deep reinforcement learning for autonomous unmanned aerial vehicle control has largely been demonstrated with multirotor platforms and high-level machine-learning frameworks. This study presents a Dueling Double Deep Q-Network (D3QN) training pipeline implemented in Rust without an external machine-learning library and integrated with Godot 4 through GDExtension for fixed wing flight control. The controller addresses fixed-wing requirements, including airspeed maintenance, lift management, throttle regulation, stall avoidance, and coordinated turning. The network combines a duelling architecture, double Q-learning, prioritised experience replay, and three-step returns in a 512→256 hidden-layer configuration containing 142,088 parameters for the 16-dimensional input case. The agent selects among seven discrete actions and supports both 12-dimensional and 16-dimensional observation spaces through a cross-dimensional weight-transfer procedure. Training was conducted for 200 episodes using four random seeds. Across seeds, the mean best episodic reward was 4825±40, while the coefficient of variation for best reward was 0.8%. In the final 30 episodes, no crashes were recorded, although completion rates varied substantially between seeds.
Airspeed remained within ±8 m/s of the 50 m/s target. Batch-64 gradient updates required less than 1 ms, representing an approximately 35-fold reduction in latency relative to the preceding GDScript implementation, and the reported runtime memory footprint remained below 50 MB. These findings support the feasibility of native Rust-based D3QN training for real-time fixed-wing simulation, while the observed inter-seed variability indicates that reward shaping and convergence robustness require further evaluation.
Saugat Chaudhary Tharu, Shrutika Ojha, Rija Bhomi et al.· Journal of Advances in Mathe...· 0 citations