1 Introduction↩︎

Skin-friction drag in wall-bounded turbulence is generated by quasi-streamwise vortices whose induced wall-normal velocities sustain the Reynolds shear stress above its laminar value [2]. The vortices are part of a self-sustaining near-wall cycle of low-speed streaks and quasi-streamwise vortices [3]–[6] that persists autonomously within the buffer layer even in the absence of outer-flow forcing [7], making that layer the natural target for drag-reduction control.

In the case of established active flow control methods, like opposition control [8]–[10], spanwise forcing and streamwise-travelling waves of spanwise velocity [11]–[13], the drag-reduction mechanism rests on a specific, physically motivated interaction with the near-wall structures, and the actuation is designed around that interaction from the outset. Controllers discovered by learnt data-driven algorithms carry no such prior; their actuation strategy emerges from the optimisation, and what physical interaction it exploits is not known in advance.

Reinforcement learning (RL) has been the most productive research area for those controllers. Applied to the classical plane channel flow at low Reynolds numbers, it has produced closed-loop wall policies that report drag reductions competitive with the physics-based benchmarks [14]–[20], as surveyed in several reviews [21]–[23]. A common way to keep the formulation tractable is a shared, translationally invariant multi-agent policy [14], [24], trained with a centralised critic on a small periodic domain, executed independently at each wall patch, then applied to a larger channel. This template underlies numerous subsequent studies [15], [20], [25]–[28]. The drag reduction headline figures reported on minimal domains must, however, be read with care: a policy evaluated in a minimal flow unit can drive the flow towards relaminarisation, where the pressure gradient falls to its laminar value and the reported drag reduction saturates at a value that reflects the box rather than the controller [1]. A representative comparison therefore requires a domain large enough to keep the flow fully turbulent throughout, the setting adopted here.

Despite the need for a large evaluation domain, training is in practice confined to a small one by the structure of the multi-agent update rather than by simulation cost alone. A gradient step adjusts the shared policy parameters from a per-patch return whose credit is assigned by a central critic, and that critic must resolve each patch within a joint action space that widens as patches are added, so the estimate deteriorates on larger domains. Two inherent features of the wall-control problem tie this restriction down, one corrupting the credit assignment and the other raising its cost. First, because the actuation must inject no net mass, the joint action is projected onto its zero-mean subspace before reaching the wall. Under this projection, the action applied at any single patch becomes dependent on all others, breaking the separability of per-patch credit [1]. Second, every training episode requires advancing a costly direct numerical simulation, so the wall-clock cost of collecting experience grows with the domain. Consequently, training is conventionally restricted to a minimal flow unit, and the policy is transferred only afterwards.

An evolution strategy removes both constraints. It treats the expected return as a function of the policy parameters. At each generation, a population of perturbed parameter vectors is evaluated over complete flow episodes. Each episode returns a scalar reward. The parameter update is a weighted average of the perturbations, with no backpropagation through the flow and no central critic [29]–[32]. Because whole episodes are scored, the per-agent credit assignment that the zero-net-mass constraint corrupts does not arise. The population evaluations are independent and parallelise across rollouts. The search therefore runs directly on the large evaluation domain, with no small-box surrogate. To our knowledge, this is the first application of an evolution strategy to the control of a turbulent flow.

The evolution-strategy controller provides a setting to address a question that has not been settled on a large domain: whether a learnt policy that reaches opposition-level drag reduction does so through the opposition mechanism or through a different reorganisation of the near-wall cycle. Its actuation is smooth and free of the bang-bang saturation that characterises degenerate controllers [1]. The controlled flow is therefore interpretable in physical terms. We characterise it with a wall-normal decomposition of the skin friction [33], Reynolds-stress profiles scaled by each controller’s own friction velocity, the anisotropy invariants of the stress tensor [34], and the premultiplied near-wall spectra [35], [36].

2 Problem set-up and methodology↩︎

2.1 Flow configuration and numerics↩︎

We perform direct numerical simulations of an incompressible flow in a plane channel at a constant flow rate by using CaNS [37], [38]. The solver integrates the unsteady, incompressible Navier-Stokes equations using a low-storage third-order Runge–Kutta advancement within a pressure-projection algorithm. The friction Reynolds number is \(\mathit{Re}_{\tau}=H^+\simeq180\). The domain measures \((L_x^+,L_y^+,L_z^+)\simeq(1922,576,360)\) in viscous units, equivalently \((L_x/H,L_y/H,L_z/H)=(10.68,3.2,2)\), with \(H\) the channel half-height. It is discretised on a \(256\times256\times100\) grid with wall-normal points clustered towards the walls. Throughout, \(u\), \(v\) and \(w\) denote the streamwise, spanwise and wall-normal velocity components, and \(p\) the pressure. Control is applied as wall-normal blowing and suction \(w_{w}(x,y,t)\) at the wall, \(\Gamma_w\), with zero net mass flux enforced at every instant, \(\int_{\Gamma_w} w_{w}\, \mathrm{d}x \,\mathrm{d}y=0\). The flow is sensed at a detection plane at \(z_{d}^+\simeq14\). The actuation interval is \(\Delta t^+\simeq5\), longer than the fastest near-wall fluctuations but short compared to the typical streak lifetime.

2.2 Closed-loop controller and the evolution strategy↩︎

The controller is a recurrent neural network that maps the velocity field at the detection plane to a wall-normal velocity field at the wall. At each actuation step the hidden state \(h_t\) is updated and a new actuation field \(a_t\) is produced. The zero-net-mass constraint is enforced as the final layer: the projection \(P: a\mapsto a-\bar{a}\) subtracts the spatial mean. The resulting \(w_w(\cdot,t)=Pa_t\) therefore satisfies \(\int_{\Gamma_w}w_w \,\mathrm{d}x \,\mathrm{d}y=0\) at every instant for any parameter vector \(\theta\in\mathbb{R}^d\). The same architecture is used by the GRU-MARL controller of [1], trained by multi-agent gradient descent [39] on a minimal flow unit.

Here, the shared parameters \(\theta\) are optimised directly on the large evaluation domain by an evolution strategy [29], [31], [32]. The optimisation target is the net energy saving over a complete episode. Let \(P_p=|\langle\partial_x p\rangle|U_b\) be the mean pumping power and \(W_w=(L_xL_y)^{-1}\int_{\Gamma_w} w_w \,p \,\mathrm{d}x \,\mathrm{d}y\) the net power delivered by the controller at the wall. The total dissipation is \(\varepsilon=P_p+W_w/H\), and the fitness is \(F(\theta) = \left(\varepsilon_0-\varepsilon(\theta)\right)/\varepsilon_0,\) where \(\varepsilon_0\) is the uncontrolled value. Two metrics commonly used to score wall controllers omit the wall power \(W_w\) entirely: the bare drag reduction \(1-|\langle\partial_x p\rangle|/|\langle\partial_x p\rangle_0|\), which tracks only the mean pressure gradient, and the cubic surrogate \(\sim|w_w|^3\), which estimates actuation cost from the actuation amplitude rather than from the power exchanged with the wall. Because neither metric can register the energy a controller adds through the wall, a controller that injects energy can score well on either while lowering \(F(\theta)\) [1], [40], [41].

Unlike gradient-based training, the evolution strategy requires no derivative of \(F\) with respect to \(\theta\). Obtaining one would mean differentiating through both the DNS solver and the recurrent network; instead, each candidate parameter vector is scored by running one complete flow episode and reading off the resulting \(F\), with the simulation and the network treated as a black box. At each generation, \(n\) pairs of parameter vectors are drawn by adding mirrored Gaussian perturbations to the current estimate \(m\in\mathbb{R}^d\), \(\{m\pm\sigma\boldsymbol{\xi}_k\}_{k=1}^{n}\) with \(\boldsymbol{\xi}_k\sim\mathcal{N}(0,\mathbf{I})\), giving \(2n\) members evaluated in parallel, each as one DNS episode initialised from the same flow state so that fitness differences reflect \(\theta\) rather than the initial condition [32]. The mean is then advanced along \[\hat{g} = \frac{1}{2n\sigma}\sum_{k=1}^{n}\bigl(\upsilon_k^{+}-\upsilon_k^{-}\bigr)\boldsymbol{\xi}_k, \label{eq:es95update}\tag{1}\] where \(\upsilon_k^{\pm}\) is the centred rank of \(F(m\pm\sigma\boldsymbol{\xi}_k)\) within the generation: each member’s position among the \(2n\) fitness values, shifted so the ranks average to zero. This makes the update depend only on the relative ordering of the population and not on the numerical scale of \(F\) [32]. The parameter mean is then updated from \(\hat{g}\) using Adam [42], a standard adaptive-step-size rule. The \(2n\) DNS episodes within a generation are mutually independent and can be run concurrently, so given enough parallel resources the wall-clock cost of a generation reduces to that of a single episode.

2.3 Control cases and diagnostics↩︎

Four flows are compared: the uncontrolled channel, opposition control, and two learnt controllers that share the architecture of §2.2 and differ only in how their parameters were obtained. Opposition sets the wall velocity to the negative of the wall-normal velocity sensed at the detection plane, \(w_{w}=-w'(z_{d})\) [8]. The first learnt controller is trained by gradient-based multi-agent learning on a minimal flow unit, within a centralised-training, decentralised-execution framework [39], [43] constrained to keep the actuation physically coherent, and applied to the evaluation domain without further training [1]; it is labelled GRU-MARL. The second is the same policy whose parameters are then optimised directly on the evaluation domain by the evolution strategy of §2.2, and is labelled ES. We omit the saturated, memoryless policies whose behaviour motivates a separate study of reward specification [1], since the concern here is the mechanism of controllers that reduce drag at low actuation power.

Each controlled flow is characterised by its own friction velocity \(u_{\tau}=\sqrt{\tau_{w}/\rho}\) (being \(\tau_{w}\) the wall shear stress and \(\rho\) the density), evaluated after the control has acted, and all wall-scaled quantities below are normalised by this flow-specific value.

The contribution of the Reynolds shear stress to the skin friction is quantified with the identity of [33], which decomposes the friction coefficient \(C_f=\tau_w/(\tfrac12\rho U_b^2)\), with \(U_b=1\) constant flow rate, into a laminar contribution \(C_f^{\mathrm{lam}}\) at the same bulk Reynolds number and a wall-normal weighted integral of the Reynolds shear stress, \[C_f = C_f^{\mathrm{lam}} + c\int_{0}^{1}(1-z/H)\,\bigl(-\left\langle u'w' \right\rangle\bigr)\,\mathrm{d}(z/H) , \label{eq:fik}\tag{2}\] where the integral runs from the wall (\(z/H=0\)) to the centreline (\(z/H=1\)) and the numerical prefactor of the plane-channel identity is \(c=12\) [33]. Scale information is taken from the premultiplied spectra of the velocity fluctuations at fixed wall-normal distance [36].

3 Results↩︎

The comparison begins with the drag reduction itself. Optimised directly on the evaluation domain, the evolution-strategy controller reduces the skin friction by \(26.1\%\), above the \(17.4\%\) obtained by the same architecture trained by gradient-based multi-agent learning on the minimal box and transferred without further training, and marginally above the \(22.5\%\) of opposition control. The gap between the two learnt controllers is the drag reduction lost when the gradient-based policy is transferred from its minimal training box to the larger channel, and recovered when the evolution strategy optimises on that channel directly. The friction velocities differ between the cases, falling from \(u_{\tau}\approx 0.064\) in the uncontrolled flow to \(0.056\) for opposition, \(0.058\) for GRU-MARL and \(0.055\) for the evolution strategy. Reaching comparable drag reduction is the starting point of the comparison; the rest of the paper asks whether the underlying flow states coincide.

Figure 1 decomposes the skin friction across the channel with the identity 2 , plotted against the wall-normal coordinate \(z/H\) and scaled for every case by the single friction velocity of the uncontrolled flow, so that the area between the uncontrolled curve and a controlled one is that controller’s drag reduction. The controlled flows carry a smaller weighted Reynolds stress than the uncontrolled flow across the buffer layer and the region above it, and the cumulative curves show the reduction accruing over the same range of \(z/H\) for all three. The relative split between layers is similar across the controllers, the buffer layer carrying about a quarter of the weighted stress in each case, while the turbulent fraction of the mean-energy budget falls from \(0.41\) in the uncontrolled flow to about \(0.34\) to \(0.35\) under control. The decomposition locates the reduction but does not on its own distinguish the controllers; the difference between them lies in how the near-wall stress is organised, which the remaining diagnostics address.

3.1 Reynolds-stress reorganisation↩︎

Figure 2 shows the streamwise normal stress \(\left\langle u'u' \right\rangle^+\) and the Reynolds shear stress \(\left\langle u'w' \right\rangle^+\), each normalised by its own friction velocity. The shear stress, which carries the drag, follows opposition for both learnt controllers, yet their streamwise normal stress differs from that of every other case, opposition included. The learnt controllers therefore reach an opposition-like shear stress through a streamwise fluctuation field that no single benchmark reproduces.

Figure 3 maps the same information onto the anisotropy-invariant triangle. The trajectories of the controlled flows through the buffer layer occupy different regions of the triangle: a coordinate-free confirmation that the controlled states are distinct and that depart differently from the uncontrolled trajectory. This is consistent with the stress profiles of figure 2.

3.2 Near-wall structure, actuation and scale selectivity↩︎

Figure 4 shows instantaneous fields of the near-wall velocity and of the wall actuation for each controller. Opposition sets the wall velocity to the negative of the sensed wall-normal velocity fluctuations, so its actuation follows the footprint of the sweeps and ejections at the detection plane by construction. The learnt actuation instead aligns with the streamwise low- and high-speed streaks. Figure 5 makes the relationship explicit: the mean actuation conditioned on the local streamwise and wall-normal fluctuations varies principally with \(u'\) and less strongly with \(w'\). Across the learnt controllers the actuation correlates with \(u'\) at about \(0.5\) to \(0.6\), two to four times stronger in magnitude than its correlation with \(w'\). The learnt actuation thus tracks the streamwise streaks, and opposition the wall-normal velocity. We describe this spatial relationship only, and do not address the order in which the actuation and the structures influence one another.

The joint probability density of \((u',w')\) at the detection plane, in figure 6, shows how the two learnt controllers redistribute the momentum-carrying events. In the uncontrolled flow the ejections and sweeps contribute almost equally. Under every controller the sweep contribution exceeds the ejection contribution, so the ejections are weakened preferentially, and the effect is strongest for the evolution-strategy controller, whose ejection contribution to the inner-scaled Reynolds shear stress \(\left\langle u'w' \right\rangle^+\) falls to about \(-0.10\), from \(-0.32\) in the uncontrolled flow. These are contributions to the shear stress, not the quadrant occupation probabilities annotated in figure 6.

Figure 7 shows the premultiplied spectra of the streamwise and wall-normal velocity fluctuations near the wall. The energy of the controlled flows is modified at the spanwise wavelengths of the near-wall streaks, around \(\lambda_y^+\simeq100\) [36], and the learnt controllers concentrate their effect there. The streamwise streak energy, peaked near \(\lambda_x^+\simeq700\) in the uncontrolled and GRU-MARL flows, is shifted to shorter wavelengths under opposition and most strongly redistributed towards small scales by the evolution strategy, whose long streaks are broken into shorter fragments (figure 4).

3.3 Smoothness and statistical steadiness of the learnt control↩︎

The comparison above is meaningful only if the learnt actuation controls the turbulence rather than settling into a fixed pattern set by the box. Figure [fig:smooth-actions] shows the distribution of wall-actuation values for each controller: it is smooth and stays away from the amplitude bound, with a root-mean-square amplitude of about \(0.016\) and a negligible saturated fraction. The streamwise-averaged actuation also remains statistically stationary and does not lock into a standing wave in the spanwise direction. The actuation is, then, an unsaturated response that tracks the evolving flow, not the saturated, frozen forcing that an objective scored on pumping power alone would favour.

4 Discussion↩︎

An evolution strategy optimises a closed-loop wall controller directly on the large, fully turbulent channel at \(\mathit{Re}_{\tau}\simeq180\), and recovers the drag reduction that the same architecture loses when a gradient-based multi-agent policy, trained on a minimal box, is transferred to that channel. The gain has a simple reading: the search scores each complete episode by a single energy-aware number and never assigns a separate reward to each wall patch, so the zero-net-mass constraint that links the patches becomes part of how the actuation is formed rather than a source of interference in the optimisation, and the controller is tuned on the same flow it will drive. It reaches a drag reduction of about \(26\%\), above the \(17\%\) retained by the transferred minimal-box policy and marginally above the \(22\%\) of opposition control. The added cost is a population of rollouts at each generation, which can run in parallel because they are independent.

The four flows differ substantially in structure, and the three controlled states remain distinct even where their drag reductions are similar. Opposition and the evolution strategy remove comparable amounts of skin friction, yet the diagnostics place their flows in different parts of the buffer layer, with different Reynolds-stress profiles, each scaled by its own friction velocity, and different anisotropy. The evolution-strategy actuation aligns predominantly with the streamwise streaks, whereas opposition acts against the wall-normal velocity [8]–[10]; in redistributing the momentum-carrying events at the detection plane it weakens the ejections preferentially and concentrates its effect near the spanwise streak wavelength \(\lambda_y^+\simeq100\). Spanwise forcing produces a comparable lateral reorganisation of the streaks [13], though through an open-loop rather than a sensed actuation.

The clearest separation is in the streaks themselves. The instantaneous fields (figure 4) and the streamwise spectra (figure 7) show opposition mainly lowering the streak amplitude, whereas the evolution strategy breaks the long streaks into shorter fragments and moves the streamwise energy to smaller scales. That two controllers reaching similar drag reduction act so differently on the coherence of the near-wall cycle suggests different mechanisms, distinguished by the flow topology rather than by the drag figure alone. Because the controller is smooth and was optimised on the same large domain on which it is assessed, this separation reflects the controlled turbulence and not the size of a training box.

References↩︎