June 08, 2026
Reinforcement learning from human feedback (RLHF) is bottlenecked by reward hacking, where the policy exploits errors in a proxy reward model (RM) and produces high RM scores without genuine quality gains. A natural mitigation is pessimism: penalizing rewards in regions where the RM is uncertain. However, standard scalar RMs provide no principled notion of uncertainty. We argue that the right object is a distributional reward model \(p(r\mid x,y)\). Under either a Bayesian inference or a KL-distributionally robust optimization (KL-DRO) lens, the KL-regularized RLHF objective admits a closed-form effective reward \(\tilde{r}(x,y) = \pm\beta\log\mathbb{E}_p[e^{\pm r/\beta}]\). The pessimistic branch unifies the prior heuristics for RM ensemble aggregation: mean aggregation, worst-case optimization (WCO), and uncertainty-weighted optimization (UWO) all emerge as limits or truncations of this single expression. This also clarifies the implicit assumptions of each existing rule.
Reinforcement learning from human feedback (RLHF) is the dominant paradigm for aligning large language models with user intent [1]–[4]. Pairwise preferences are first used to fit a reward model (RM) and a policy is then optimized against the RM with a KL penalty to a reference policy, via policy-gradient methods such as PPO [5] or GRPO [6]. Because the RM is an imperfect proxy for human preferences, policy optimization can overoptimize into out-of-distribution regions, producing high RM scores without improving true response quality. This is an instance of Goodhart’s law [7]–[10] and has been documented across forms including sycophancy and [11], [12] length bias [13].
A natural mitigation is pessimism: when the RM is uncertain, a lower effective reward should be assigned. Prior work in RLHF instantiates this idea by training ensembles of RMs and aggregating their outputs in different ways. [14] introduce three rules: mean aggregation (averaging members), worst-case optimization (WCO, taking the minimum across members), and uncertainty-weighted optimization (UWO, mean minus a tunable multiple of the empirical variance). [15] make a similar case for ensembles, and follow-up work has explored adversarial [16] and robustness-driven [17] variants. These methods empirically help, but are typically presented as heuristics with little principled guidance on which to prefer and how their hyperparameters relate.
We argue here that to properly handle uncertainty, one should replace the scalar RM with a distributional reward model \(p(r \mid x, y)\). Once the RM is distributional, a single principle—either marginalization over reward uncertainty through a Bayesian inference lens [18], [19] or KL-distributionally robust optimization (KL-DRO) [20], [21]—yields a closed-form effective reward to substitute into the standard KL-RL objective. The aggregation rules used in practice emerge as truncations or limits of this expression. This perspective also connects RLHF pessimism to distributional reinforcement learning [22]–[25] and to offline-RL pessimism [26]–[28].
A unifying lens. From either a Bayesian inference or KL-DRO perspective, the RLHF objective with reward uncertainty admits a closed-form effective reward \(\tilde{r}(x,y) = \pm\beta\log\mathbb{E}_p[e^{\pm r/\beta}]\) (Sec. 3).
Unification of prior heuristics. Mean, WCO, and UWO emerge as the \(\beta\to\infty\) limit, \(\beta\to 0\) limit, and Gaussian truncation, respectively, of the pessimistic branch (Sec. 3.5).
Implementation guidance. We discuss what each heuristic implicitly assumes about the reward distribution \(p(r\mid x,y)\), and what is needed (calibration, posterior families) to instantiate the framework with a real RM (Sec. 4).
[7] established that the proxy RM score and a stronger “gold” RM score (which is used to model the true objective we seek to optimize, i.e., how a human would score the response) diverge as PPO progresses, the canonical signature of reward hacking. [8] formalize reward hacking, and [9] situate it among broader AI safety concerns. Empirically, hacked policies exhibit sycophancy [11], length inflation [13], and other surface artifacts that the proxy RM cannot distinguish from true quality [29].
[14] introduce Mean, WCO, and UWO aggregation over RM ensembles and find that pessimistic variants reduce reward hacking. [15] reach similar conclusions for best-of-\(n\) and PPO, while noting that ensembles can share failure modes. Lightweight ensembles share a frozen backbone and diversify only a head [30].
An alternative to ensembles is to place a posterior directly over a single head. Bayesian last-layer methods [31], [32] yield a closed-form Gaussian posterior over the reward \(\mathcal{N}(\mu(x,y), \sigma^2(x,y))\) at each prompt-response pair. [33] use RM uncertainty to route hard cases to a stronger LLM judge.
The Bayesian view of KL-regularized RL [18], [19] interprets policies as posteriors under an “optimality” observation. Robust optimization [20], [21] and pessimism in offline RL [26]–[28] provide the complementary, adversarial framing. Our derivation shows that, in the RLHF setting, these two seemingly disparate views produce the same functional form up to a sign.
Standard KL-regularized RLHF [2], [4] maximizes \[\label{eq:rlhf} \max_{\theta}\;\mathbb{E}_{x \sim \mathcal{D}}\!\Big[\mathbb{E}_{y \sim \pi_\theta(\cdot \mid x)} [r(x,y)] - \beta\, D_{\mathrm{KL}}\!\left(\pi_\theta\|\pi_{\mathrm{ref}}\right)\!\Big],\tag{1}\] with the well-known optimum \(\pi^*(y\mid x) \propto \pi_{\mathrm{ref}}(y\mid x)\exp(r(x,y)/\beta)\).
In the standard pipeline, \(r(x,y) = r_\phi(x,y)\) is a deterministic scalar from a learned RM. We instead treat the reward as a random variable \(r \sim p(r \mid x, y)\), arising, for example, from a deep ensemble or a Bayesian head, and ask: to best mitigate reward hacking, what scalar effective reward \(\tilde{r}(x,y)\) should replace \(r_\phi(x,y)\) in Eq. 1 ?
Following the inference-as-control view [18], [19], we introduce a binary optimality variable \(O \in \{0,1\}\) with \[\label{eq:inference95as95control} p(O{=}1\mid x,y,r) = e^{r/\beta},\tag{2}\] where \(r\) is shifted so that \(\sup r = 0\). Marginalizing over reward uncertainty gives \[p(O{=}1\mid x,y) = \int e^{r/\beta} p(r\mid x,y)\,dr = M_x(1/\beta),\] where \(M_x(t) := \mathbb{E}_p[e^{tr}]\) is the moment generating function of \(p(r\mid x,y)\). Bayes’ rule with prior \(\pi_{\mathrm{ref}}\) then yields \(\pi^*_{\mathrm{opt}}(y\mid x) \propto \pi_{\mathrm{ref}}(y\mid x)\, M_x(1/\beta)\). Matching to the KL-RL form \(\pi^* \propto \pi_{\mathrm{ref}}\exp(\tilde{r}/\beta)\) identifies \[\label{eq:opt} \boxed{\;\tilde{r}_{\mathrm{opt}}(x,y) = \beta \log \mathbb{E}_{p(r\mid x,y)}\!\left[e^{r/\beta}\right].\;}\tag{3}\] By Jensen’s inequality, since \(\exp\) is convex, \[\tilde{r}_{\mathrm{opt}}(x,y) \;\geq\; \mu(x,y),\] so uncertainty inflates the effective reward. This is the Bayesian-optimal use of an uncertain RM under the inference-as-control likelihood in Eq. 2 in the absence of any robustness concern.
To mitigate reward hacking we instead want a worst-case effective reward. We let an adversary choose the reward distribution \(Q\), paying a KL cost to deviate from the believed posterior \(p\) [20], [21]: \[\label{eq:dro} \tilde{r}_{\mathrm{rob}}(x,y) = \inf_Q \!\Big\{\mathbb{E}_Q[r] + \beta\, D_{\mathrm{KL}}(Q\,\|\,p(r\mid x,y))\Big\}.\tag{4}\] The parameter \(\beta\) controls the adversary’s reach: large \(\beta\) pins \(Q\) close to \(p\), small \(\beta\) permits arbitrary tilting. Solving the variational problem (see Appendix 6) yields the unique optimizer \[Q^*(r) \;\propto\; p(r\mid x,y)\, e^{-r/\beta},\] which is \(p\) exponentially tilted toward low rewards. Plugging back in, \[\label{eq:rob} \boxed{\;\tilde{r}_{\mathrm{rob}}(x,y) = -\beta \log \mathbb{E}_{p(r\mid x,y)}\!\left[e^{-r/\beta}\right].\;}\tag{5}\] This is the entropic risk measure of \(p(r\mid x,y)\). By Jensen’s inequality, \(\tilde{r}_{\mathrm{rob}}(x,y) \leq \mu(x,y)\): uncertainty deflates the effective reward.
The KL coefficient \(\beta\) in Eq. 1 controls the deviation of the policy from the reference, while \(\beta\) in Eq. 4 controls the robustness to reward uncertainty. These play distinct roles and need not coincide. In this work we tie the two for simplicity, yielding the simple coefficient \(\lambda=1 /(2 \beta)\). More generally one may consider \(c \beta\), with \(c\) reflecting the desired degree of pessimism relative to policy regularization.
Let \(K_x(t) := \log M_x(t) = \log\mathbb{E}_p[e^{tr}]\) be the cumulant generating function (CGF) of \(p(r\mid x,y)\), with cumulants \(\kappa_n\) defined by \(K_x(t) = \sum_{n\geq 1} \kappa_n t^n / n!\). Then \(\kappa_1 = \mu\), \(\kappa_2 = \sigma^2\), \(\kappa_3\) is the third central moment, \(\kappa_4 = \mathbb{E}[(r-\mu)^4] - 3\sigma^4\) (excess kurtosis \(\times\sigma^4\)), and so on.
Substituting into Eqs. 3 and 5 : \[\label{eq:cumulants} \tilde{r}_{\mathrm{opt/rob}} = \mu \pm \frac{\sigma^2}{2\beta} + \frac{\kappa_3}{6\beta^2} \pm \frac{\kappa_4}{24\beta^3} + \cdots\tag{6}\] Even cumulants (\(\kappa_2,\kappa_4,\ldots\)) flip sign between the two while odd cumulants (\(\kappa_3,\kappa_5,\ldots\)) do not. For symmetric distributions (\(\kappa_{2n+1} = 0\) for \(n\geq 1\)), \(\tilde{r}_{\mathrm{opt}}\) and \(\tilde{r}_{\mathrm{rob}}\) are exact mirror images about \(\mu\).
If \(p(r\mid x,y) = \mathcal{N}(\mu, \sigma^2)\), all cumulants beyond \(\kappa_2\) vanish, \(K_x(t) = \mu t + \sigma^2 t^2 / 2\), and the series in Eq. 6 terminate exactly: \[\label{eq:gauss} \tilde{r}_{\mathrm{opt}} = \mu + \frac{\sigma^2}{2\beta},\qquad \tilde{r}_{\mathrm{rob}} = \mu - \frac{\sigma^2}{2\beta}.\tag{7}\] Under any Gaussian distributional RM, the principled pessimistic reward is \(\mu - \sigma^2/2\beta\), with the variance coefficient set by the KL coefficient \(\beta\) rather than tuned as a free hyperparameter.
The aggregation rules of [14] emerge as limits and truncations of \(\tilde{r}_{\mathrm{rob}}\).
As \(\beta \to \infty\), the variance term in Eq. 6 vanishes (\(\sigma^2/2\beta \to 0\)), and similarly for all higher orders. Hence \(\tilde{r}_{\mathrm{rob}} \to \mu\). Equivalently, an infinite KL penalty forces the adversary’s distribution \(Q^*\) back onto \(p\), so the worst case coincides with the mean. Mean aggregation is the risk-neutral limit of the pessimistic effective reward.
As \(\beta \to 0\), the tilted distribution \(Q^*(r) \propto p(r) e^{-r/\beta}\) concentrates all of its mass on \(\inf\,\mathrm{supp}(p)\). For an empirical ensemble \(\{R_i\}_{i=1}^K\) where \(p\) is a uniform mixture of point masses, the infimum is \(\min_i R_i\). Hence \[\lim_{\beta \to 0} \tilde{r}_{\mathrm{rob}}(x,y) \;=\; \min_i R_i(x,y).\] Worst-case optimization is the unbounded-adversary limit of the pessimistic effective reward.
The UWO rule of [14] is \[R_{\mathrm{UWO}}(x,y) = \hat{\mu}(x,y) - \lambda\,\hat{\sigma}^2(x,y),\] with \(\lambda\) treated as a free hyperparameter. Intuitively, UWO works by penalizing the policy for generating responses for which there is high disagreement among reward models within the ensemble, which helps prevent the exploitation of a single reward model which might be erroneously assigning high rewards to incorrect or low-quality responses. Comparing with the Gaussian case of Eq. 7 , UWO is exactly the pessimistic effective reward under a Gaussian posterior, with \(\lambda = 1/(2\beta)\). UWO’s implicit assumptions are: (i) \(p(r\mid x,y)\) is approximately Gaussian; (ii) the variance coefficient can be decoupled from the KL coefficient \(\beta\).
\begin{table}[t]
| Method | Formula | Recovered as |
|---|---|---|
| Mean | \(\hat{\mu}\) | \(\beta\to\infty\) limit |
| WCO | \(\min_i R_i\) | \(\beta\to 0\) limit |
| UWO | \(\hat{\mu}-\lambda\hat{\sigma}^2\) | Gaussian, \(\lambda\) free |
| Ours | \(\hat{\mu}-\hat{\sigma}^2/2\beta\) | Gaussian, \(\lambda=1/2\beta\) |
-0.15in \end{table}
Table ¿tbl:tab:unification? summarizes the unification. The take-away is that the choice of aggregation rule is a choice about the assumed shape of \(p(r\mid x,y)\) and the adversary’s KL budget. Mean assumes no uncertainty matters; WCO assumes the worst possible distribution; UWO is the middle ground in a Gaussian approximation. Our framework makes these assumptions explicit and supplies a principled default (\(\lambda = 1/2\beta\)) when working with a Gaussian distributional RM.
In practice \(p(r\mid x,y)\) is approximated by a finite sample \(\{r_i(x,y)\}_{i=1}^K\)—either ensemble members or posterior samples. Let \[\hat{\mu} = \frac{1}{K}\sum_{i=1}^K r_i,\quad \hat{\sigma}^2 = \frac{1}{K-1}\sum_{i=1}^K (r_i - \hat{\mu})^2.\] Two natural estimators of \(\tilde{r}_{\mathrm{rob}}\) arise.
\[\hat{\tilde{r}}_{\mathrm{rob}}^{\,(\text{logMGF})} = -\beta\log\!\left(\frac{1}{K}\sum_{i=1}^K e^{-r_i/\beta}\right).\] This estimator is consistent but biased at finite \(K\): since \(\log\) is concave, Jensen gives \(\mathbb{E}[\hat{\tilde{r}}_{\mathrm{rob}}^{\,(\text{logMGF})}] \geq \tilde{r}_{\mathrm{rob}}\), with bias growing in \(\sigma/\beta\). It is sensitive to outliers when \(\beta\) is small (the \(\exp(-r_i/\beta)\) term blows up for any single low \(r_i\)).
\[\label{eq:est-gauss} \hat{\tilde{r}}_{\mathrm{rob}}^{\,(\text{Gauss})} = \hat{\mu} - \frac{\hat{\sigma}^2}{2\beta}.\tag{8}\] This is exact when \(p\) is Gaussian and drops higher cumulants otherwise. We note that for small \(K\) (as is typical for RM ensembles, \(K \in \{3,\ldots,10\}\) in [14]), higher cumulants are poorly estimated. The relative error on \(\hat{\sigma}^2\) alone is \(\sqrt{2/(K-1)} \approx 32\%\) for \(K=20\), so we recommend Eq. 8 as a default.
The pessimistic effective reward \(\tilde{r}_{\mathrm{rob}}\) is only as informative as the posterior \(p(r\mid x,y)\) used to compute it. Two concrete prerequisites:
(i) A genuine distributional RM. Scalar RMs trained with the Bradley-Terry likelihood [34] provide no uncertainty. Two practical options are: (a) a deep ensemble [14], [30], [35], whose empirical variance approximates \(\sigma^2(x,y)\); and (b) a Bayesian last-layer construction [31], [32], which yields a closed-form Gaussian posterior at the cost of a single forward pass and an \(O(d^2)\) matrix-vector product. The Bayesian construction is especially interesting as its posterior is Gaussian and Eq. 7 applies without truncation.
(ii) Calibration. The variance term \(\sigma^2(x,y)\) only modulates \(\tilde{r}_{\mathrm{rob}}\) usefully if it tracks true epistemic uncertainty. The Bradley–Terry loss collapses soft preferences (\(0.51\) and \(0.99\)) onto the same hard label and degrades calibration. Strictly proper scoring rules [36], in particular the Brier score [37], preserve annotator confidence and are known to improve calibration. Using AI feedback [3], [38], [39] provides soft labels naturally by prompting for a confidence score during AI feedback.
When Mean should suffice. The cumulant expansion (Eq. 6 ) shows that pessimism corrections decay as \(1/\beta^{n-1}\). For large \(\beta\) (heavy KL regularization), Mean is essentially optimal as all rules collapse to it.
When WCO is appropriate. WCO is the \(\beta \to 0\) limit, suitable when the adversary is essentially unconstrained. In RLHF, this corresponds to a setting where one expects the RM to be arbitrarily wrong in some direction. [15] report that WCO can underperform Mean when the ensemble is too small or correlated, which our framework explains: WCO discards all distributional information beyond the minimum.
Why UWO works. UWO is the right shape under a Gaussian posterior. Its empirical success suggests that practical RM ensembles are reasonably symmetric and unimodal. The free \(\lambda\) may be absorbing miscalibration; under the theory-prescribed value \(\lambda = 1/2\beta\), calibration becomes essential.
The cumulant expansion in Eq. 6 suggests three directions when \(p(r\mid x,y)\) is not well-approximated by a Gaussian. (i) For asymmetric but light-tailed distributions, third- and fourth-cumulant corrections to Eq. 8 can be estimated from larger ensembles. (ii) For genuinely heavy-tailed distributions where the MGF estimator is unstable, quantile-based pessimism (e.g., CVaR) is a robust alternative, connecting to quantile distributional RL [23]–[25]. (iii) Flexible posterior families, such as mixtures of Gaussians or normalizing flows fit directly to held-out preference data could replace the ensemble or Bayesian linear head.
The KL-DRO derivation connects RLHF reward hacking to pessimism in offline RL [26]–[28], where similar log-MGF / entropic-risk formulations arise from analogous principles. Our contribution is to localize this connection: in RLHF, the relevant random variable is the reward (not the transition dynamics, which are deterministic for an autoregressive policy), and the relevant penalty parameter is the standard KL coefficient \(\beta\) already present in Eq. 1 .
We have given a single derivation that captures the essence of reward uncertainty in RLHF. From either a Bayesian or a KL-DRO starting point, the closed-form effective reward is \(\tilde{r}(x,y) = \pm\beta\log\mathbb{E}_p[e^{\pm r/\beta}]\). The pessimistic branch unifies mean aggregation, WCO, and UWO as limits and truncations, and yields a principled default \(\tilde{r} = \mu - \sigma^2/2\beta\) under a Gaussian posterior in which the variance coefficient is set by the KL coefficient rather than tuned. The framework clarifies what each existing heuristic implicitly assumes and gives a recipe for going beyond Gaussian via higher cumulants or quantile-based pessimism.
We thank Kiante Brantley for initial guidance and helpful discussions during the early stages of this project. OpenAI GPT-5.5 and Refine.ink were used to assist with language editing and improve the clarity and readability of the manuscript
Let \(\pi_0 = \pi_{\mathrm{ref}}\). For each \((x,y)\), \(r \sim p(r\mid x,y)\). The KL-regularized objective with effective reward \(\tilde{r}\) is \[J(\pi) = \mathbb{E}_\pi[\tilde{r}(x,y)] - \beta\, D_{\mathrm{KL}}(\pi \| \pi_0), \qquad \beta > 0.\]
Introduce \(O \in \{0,1\}\) with \(p(O=1\mid x,y,r) = e^{r/\beta}\) where \(r\) is shifted so \(\sup r = 0\).
\[p(O{=}1\mid x,y) = \int e^{r/\beta} p(r\mid x,y)\,dr = M_x(1/\beta),\] where \(M_x(t) := \mathbb{E}_p[e^{tr}]\) is the MGF.
With prior \(\pi_0\), \[\pi^*_{\mathrm{opt}}(y\mid x) \propto \pi_0(y\mid x) M_x(1/\beta).\]
Matching to \(\pi^* \propto \pi_0 \exp(\tilde{r}/\beta)\) gives \[\tilde{r}_{\mathrm{opt}}(x,y) = \beta \log \mathbb{E}_p[e^{r/\beta}].\]
Since \(\exp\) is convex, \(\mathbb{E}_p[e^{r/\beta}] \geq e^{\mu/\beta}\), so \(\tilde{r}_{\mathrm{opt}} \geq \mu\).
\[\tilde{r}_{\mathrm{rob}}(x,y) = \inf_Q \{\mathbb{E}_Q[r] + \beta D_{\mathrm{KL}}(Q\|p)\}.\]
With multiplier \(\lambda\) for \(\int Q = 1\), \[\mathcal{L}(Q,\lambda) = \int Q(r) r\,dr + \beta\!\int Q\log Q\,dr - \beta\!\int Q\log p\,dr - \lambda\!\left(\int Q\,dr - 1\right).\]
Using \(\frac{d}{dQ}[Q\log Q] = \log Q + 1\): \[\frac{\delta \mathcal{L}}{\delta Q(r)} = r + \beta(\log Q + 1) - \beta\log p - \lambda = 0,\] giving \(Q^*(r) \propto p(r) e^{-r/\beta}\). The normalization is \(C(x,y) := \mathbb{E}_p[e^{-r/\beta}]\).
\(\log Q^*(r) - \log p(r) = -r/\beta - \log C\), so \[D_{\mathrm{KL}}(Q^*\|p) = -\tfrac{1}{\beta}\mathbb{E}_{Q^*}[r] - \log C.\] Therefore \(\mathbb{E}_{Q^*}[r] + \beta D_{\mathrm{KL}}(Q^*\|p) = -\beta\log C\) (the reward terms cancel exactly), giving \[\tilde{r}_{\mathrm{rob}}(x,y) = -\beta\log\mathbb{E}_p[e^{-r/\beta}].\]
\(\mathbb{E}_p[e^{-r/\beta}] \geq e^{-\mu/\beta}\); taking \(-\beta\log\) reverses the inequality, so \(\tilde{r}_{\mathrm{rob}} \leq \mu\).\(\square\)
Let \(K_x(t) = \log\mathbb{E}_p[e^{tr}] = \sum_{n\geq 1} \kappa_n t^n / n!\). Then \[\tilde{r}_{\mathrm{opt}} = \beta K_x(1/\beta) = \sum_{n\geq 1} \frac{\kappa_n}{n!\,\beta^{n-1}} = \mu + \frac{\sigma^2}{2\beta} + \frac{\kappa_3}{6\beta^2} + \frac{\kappa_4}{24\beta^3} + \cdots,\] \[\tilde{r}_{\mathrm{rob}} = -\beta K_x(-1/\beta) = \sum_{n\geq 1} \frac{(-1)^{n+1}\kappa_n}{n!\,\beta^{n-1}} = \mu - \frac{\sigma^2}{2\beta} + \frac{\kappa_3}{6\beta^2} - \frac{\kappa_4}{24\beta^3} - \cdots.\] For Gaussian \(p\), \(\kappa_{n\geq 3} = 0\) and the series terminate at \(\tilde{r} = \mu \pm \sigma^2/2\beta\).
Each term \(\kappa_n/(n!\beta^{n-1})\) for \(n\geq 2\) vanishes, leaving \(\tilde{r}_{\mathrm{rob}} \to \kappa_1 = \mu\). Equivalently, \(Q^*(r) \propto p(r) e^{-r/\beta} \to p(r)\) as \(\beta\to\infty\).
For an empirical ensemble \(\{R_i\}_{i=1}^K\) with \(p(r\mid x,y) = \frac{1}{K}\sum_i \delta(r - R_i)\), \[\tilde{r}_{\mathrm{rob}} = -\beta\log\!\left(\frac{1}{K}\sum_{i=1}^K e^{-R_i/\beta}\right).\] Let \(R_{\min} = \min_i R_i\) and factor out \(e^{-R_{\min}/\beta}\): \[\tilde{r}_{\mathrm{rob}} = R_{\min} - \beta\log\!\left(\frac{1}{K}\sum_{i=1}^K e^{-(R_i - R_{\min})/\beta}\right).\]
As \(\beta \rightarrow 0\), all terms in the sum with \(R_i>R_{\min }\) are exponentially suppressed; the sum approaches the count of minimizing members \(m\) divided by \(K\). The logarithm of this sum approaches the constant \(\log (m / K)\), so the overall correction term scales as \(O(\beta) \rightarrow 0\). Hence \(\tilde{r}_{\mathrm{rob}} \to R_{\min} = \min_i R_i\), recovering WCO.
For \(p = \mathcal{N}(\mu, \sigma^2)\), \(K_x(t) = \mu t + \sigma^2 t^2/2\) exactly, so \(\tilde{r}_{\mathrm{rob}} = \mu - \sigma^2/2\beta\). This matches \(R_{\mathrm{UWO}} = \hat{\mu} - \lambda\hat{\sigma}^2\) with \(\lambda = 1/(2\beta)\).