1 Introduction↩︎

Semantic segmentation models can achieve strong task performance while still producing unreliable confidence estimates. In dense prediction settings, calibration is particularly important because voxel- or pixel-wise confidence scores are used for downstream decisions, quality control, uncertainty-aware post-processing, and failure detection [1]–[3]. However, high segmentation accuracy does not guarantee reliable predictive probabilities: segmentation models may be miscalibrated, exhibiting over- or underconfidence, especially near object boundaries, ambiguous structures, or distribution shifts [4], [5]. Such errors can undermine the practical utility of confidence estimates even when the predicted segmentation masks are accurate.

Post-hoc calibration offers a practical way to correct confidence estimates after training, without retraining or modifying the underlying segmentation model. Train-time calibration methods, such as label smoothing [6], differentiable calibration losses [7], and focal-loss-based calibration [8], are complementary. We focus on post-hoc calibration because it can be applied to already deployed segmentation models without retraining, can be updated under changing deployment conditions, including distribution shifts, and allows confidence correction to be studied separately from segmentation-model training. Standard post-hoc methods typically learn a global transformation in logit or probability space on a held-out calibration set, often by minimizing cross-entropy [9]–[11]. These methods are largely inherited from image-level classification, where each sample is associated with a single prediction. In semantic segmentation, however, calibration is applied densely across many spatial locations, and the interaction between logit-space transformations, spatially structured predictions, and task-specific segmentation metrics has not been fully characterized.

A fundamental but often overlooked point is that logits are non-identifiable: adding a constant to all class logits at a given spatial location leaves the softmax probabilities unchanged. Thus, a predictive distribution corresponds not to a unique logit vector, but to an equivalence class of logits that differ by additive shifts, sometimes interpreted as a “gauge” degree of freedom. Consequently, two logit vectors that induce the same predictive distribution can yield different calibrated outputs if the calibration function operates directly in logit space and is sensitive to such shifts. This dependence on an arbitrary logit representative introduces representation sensitivity, potentially leading to inconsistent or unstable calibration behavior.

While logit non-identifiability also holds in image-level classification, its practical impact may be amplified in semantic segmentation. Calibration is applied across many voxels or pixels, and additive offsets may vary across spatial locations, effectively introducing a field of local gauge degrees of freedom. This spatial structure increases the dimensionality of the ambiguity and may make its practical consequences more pronounced for dense prediction tasks. Fig. 1 (a, b) illustrates this issue through shifted-logit examples and empirical free-energy maps, showing that such offsets occur in practice and vary spatially.

A second structural issue arises from the mismatch between training and calibration objectives. Segmentation models are typically trained with task-specific losses such as the Dice loss or its variants [12], [13], whereas post-hoc calibration usually minimizes cross-entropy [9], [10]. Because optimizing cross-entropy can alter class orderings, calibration may change the predicted segmentation mask and potentially reduce the DSC. In our experiments, such calibration-induced decision changes degrade segmentation performance (Fig. 1 (c)), motivating calibrators that correct confidence estimates without altering the original decision map.

Despite the widespread use of post-hoc calibration in segmentation, the interplay between logit non-identifiability, spatially dense calibration, and calibration-induced decision changes has not been systematically analyzed. We address this gap through two complementary structural constraints on post-hoc calibration:

• We formalize translation invariance as a condition for representation-consistent calibration, characterize which common calibrators satisfy it, and construct matched translation-invariant (TI) counterparts of shift-sensitive calibrators to isolate the effect of removing representation dependence.

• We study decision-preserving calibration under argmax- and order-preservation constraints. Since enforcing these constraints collapses affine softmax calibrators to temperature scaling, we introduce class-conditional affine calibrators that can be made argmax- or order-preserving while retaining greater expressivity, allowing us to quantify the calibration–segmentation trade-off induced by decision preservation.

• Across natural-image and medical segmentation benchmarks, and under corruption-based covariate shift, paired comparisons show that TI variants improve calibration metrics in most matched comparisons. Decision-preserving variants prevent calibration-induced DSC drops by construction, while retaining strong calibration performance.

Overall, our goal is not to identify a universally optimal calibration pipeline, since performance also depends on calibrator expressivity and dataset-specific characteristics. Instead, we clarify which structural properties make post-hoc calibration well-defined for dense prediction. Translation invariance ensures that calibration does not depend on an arbitrary logit representative, while argmax or order preservation prevents calibration from altering the deployed segmentation map.

2 Related Work↩︎

2.0.0.1 Post-hoc calibration of neural networks.

Post-hoc calibration aims to align predicted confidence with empirical accuracy without modifying the trained model. Pioneering work focused on binary or small-scale classification, including Platt scaling [14], histogram binning [15], isotonic regression [16], and Bayesian binning into quantiles [17]. For modern multiclass neural networks, commonly used methods include temperature scaling (TS), vector scaling (VS), matrix scaling (MS), and Dirichlet calibration (DC) [9], [10], as well as order-preserving and Gaussian process-based extensions [18]–[20]. A fundamental but often overlooked issue in calibration is that logits are not uniquely defined: adding a constant to all logits leaves softmax probabilities unchanged. The implications of this invariance for post-hoc calibration have received little attention. In practice, calibration is often performed directly in logit space using affine transformations such as vector or matrix scaling, which may depend on the particular logit representative chosen. As a result, calibrated predictions may vary across equivalent logit embeddings. Although one could argue that logit shifts may encode useful information in certain contexts, for example in energy-based formulations for out-of-distribution detection [21], [22], post-hoc calibration of predictive probabilities should not depend on arbitrary representational degrees of freedom unless explicitly modeled. In contrast to prior work, we formalize translation invariance as a necessary condition for representation-consistent calibration and characterize which calibration pipelines satisfy it, or how this property can be enforced.

2.0.0.2 Post-hoc calibration in semantic segmentation.

In semantic segmentation, reliable voxel- or pixel-wise confidence estimates are important for downstream decisions, quality control, failure detection, and uncertainty-aware post-processing [1]–[3]. Calibration methods developed for classification are commonly transferred to dense prediction by treating pixels or voxels as independent calibration samples and applying a global mapping [23]. This global approach assumes uniform calibration across the image, although calibration errors often vary spatially, especially near object boundaries and ambiguous structures [5], [24]. To address spatial heterogeneity, adaptive methods such as Local Temperature Scaling (LTS) [25] learn spatially varying temperature maps, where the temperature at each location is predicted from the network’s outputs, such as logits. Selective scaling mechanisms have also been proposed to enable more flexible calibration in semantic segmentation by focusing adjustment on misclassified pixels [26]. A second challenge is the mismatch between calibration and task-specific segmentation objectives. Segmentation models are often trained or evaluated with overlap-based criteria such as Dice [12], [13], whereas post-hoc calibration is typically fitted by minimizing cross-entropy. Consequently, a calibrator can improve the calibration loss while changing class orderings, thereby modifying the predicted segmentation map and causing measurable DSC drops [23]. This motivates decision-preserving calibration, in which calibrated probabilities preserve either the original argmax or the complete class ordering. In segmentation, methods with this property commonly rely on temperature-scaling variants [25], [26], which are naturally order-preserving. Decision and order preservation have also been considered in classification. Mix-n-Match [11] promotes accuracy-preserving calibration and introduces ETS, a convex mixture of temperature scaling, the identity map, and the uniform distribution, to increase expressivity while preserving accuracy. [18] go further by learning intra order-preserving calibration maps with a constrained neural architecture that preserves top-\(k\) predictions. In dense prediction, this issue has received comparatively less attention, although the loss mismatch discussed above makes it particularly important. We propose to construct matched decision-preserving counterparts of expressive unconstrained calibrators, in order to isolate the effect of this constraint and quantify the calibration–segmentation trade-off. The main difficulty is to impose argmax or order preservation without reducing the calibrator to temperature scaling. To this end, we introduce class-conditional affine calibrators that can be made to preserve either the original argmax or the complete class ordering while retaining greater expressivity.

3 Methods↩︎

3.1 Translation-Invariant Post-hoc Calibration↩︎

For a \(C\)-class segmentation task, each location \(v\) is associated with a ground-truth label \(y^v \in \{1,\dots,C\}\) and a logit vector \(\mathbf{z}^v \in\mathbb{R}^C\) produced by a segmentation model. Class probabilities are obtained through the softmax transformation \(\mathbf{p}^v=S(\mathbf{z}^v)\). Post-hoc calibration learns a mapping \(g\) such that the calibrated probabilities \(\mathbf{q}^v = g(\mathbf{z}^v)\) better reflect empirical accuracies. In the case of top-label calibration, this can be expressed as \[\label{eq:calib} \mathbb{P}\!\left(y^v=\hat{y} \mid \mathbf{q}^v_{\hat{y}}\right) = \mathbf{q}^v_{\hat{y}},\tag{1}\] where \(\hat{y} = \arg\max_c \mathbf{q}^v_c\). In practice, \(g\) is fitted by minimizing a proper scoring rule, typically the negative log-likelihood (NLL), on a calibration set.

Since the softmax is invariant to additive shifts, two logit vectors \(\mathbf{z}^{v_1}\) and \(\mathbf{z}^{v_2}\) differing by a common additive constant yield identical probability distributions (\(S(\mathbf{z}^{v_1}) = S(\mathbf{z}^{v_2})\)), but applying a calibrator in logit space can break this equivalence, producing different calibrated predictions: \(g(\mathbf{z}^{v_1}) \neq g(\mathbf{z}^{v_2})\). To preserve this equivalence, we define a calibrator as translation-invariant (TI) if it satisfies \[\label{eq:translation95inv} g(\mathbf{z}) = g(\mathbf{z} + c\mathbf{1}), \quad \forall \mathbf{z} \in \mathbb{R}^C,\; \forall c \in \mathbb{R}.\tag{2}\]

Under this property, any two logit vectors yielding the same softmax probabilities necessarily produce identical calibrated predictions. Equivalently, the output of a TI calibrator depends only on the predictive distribution induced by the softmax and not on the particular logit representation. Translation invariance is a natural consistency requirement for post-hoc calibration of predictive probabilities. Otherwise, the calibrator would depend on a degree of freedom that carries no predictive information.

3.1.0.1 Standard post-hoc calibrators.

We consider standard multiclass post-hoc calibrators applied voxel-wise, including temperature scaling (TS) and its affine extensions, vector scaling (VS) and matrix scaling (MS) [9]. We further study two TS-based variants: ensemble temperature scaling (ETS) [11], which mixes the temperature-scaled distribution with the original softmax output and a uniform prior, and local temperature scaling (LTS) [25], where a temperature function \(\tau_\theta\) (e.g., a CNN) predicts a spatially varying temperature field from the full logit map \(\mathbf{Z}\) and optional features \(\xi\):

\[\begin{align} g^{\mathrm{TS}}_{T}(\mathbf{z}) &= S(\mathbf{z}/T), && T>0, \tag{3} \\ g^{\mathrm{VS}}_{\mathbf{a},\mathbf{b}}(\mathbf{z}) &= S(\operatorname{diag}(\mathbf{a})\,\mathbf{z} + \mathbf{b}), && \mathbf{a},\mathbf{b}\in\mathbb{R}^C, \tag{4} \\ g^{\mathrm{MS}}_{\mathbf{W},\mathbf{b}}(\mathbf{z}) &= S(\mathbf{W}\mathbf{z} + \mathbf{b}), && \mathbf{W}\in\mathbb{R}^{C\times C},\; \mathbf{b}\in\mathbb{R}^C, \tag{5} \\ g^{\mathrm{ETS}}_{\lambda,T}(\mathbf{z}) &= \lambda_1 S(\mathbf{z}/T) + \lambda_2 S(\mathbf{z}) + \lambda_3 \mathbf{u}, && \lambda_k\ge 0,\; \sum_{k=1}^3\lambda_k=1,\; \mathbf{u}=\frac{1}{C}\mathbf{1}, \tag{6} \\ g_{\theta}^{\mathrm{LTS}}(\mathbf{z}^v) &= S\!\left( \mathbf{z}^v/\tau_\theta(v;\mathbf{Z},\xi) \right), && \tau_\theta(v;\mathbf{Z},\xi)>0. \tag{7} \end{align}\]

The following properties follow directly from the definitions. TS and ETS are translation-invariant (TI). VS is TI if and only if all components of \(\mathbf{a}\) are equal, in which case it reduces to temperature scaling with a class-wise additive bias. MS is TI if and only if \(\mathbf{W} \mathbf{1} = \alpha \mathbf{1}\) for some \(\alpha \in \mathbb{R}\), which is equivalent to all rows of \(\mathbf{W}\) summing to the same value. Assuming non-constant logits, LTS is TI if and only if the temperature predictor \(\tau_\theta\) is invariant to additive logit shifts (proof in Appendix 6.1). However, this property is not guaranteed in [25] since \(\tau_\theta\) takes the raw logit field \(\mathbf{Z}\) as input.

Violating translation invariance can lead to undesirable behavior. As an example, consider MS: \(g(\mathbf{z}+c\mathbf{1})=S(\mathbf{W} \mathbf{z}+ \mathbf{b}+ c \mathbf{W}\mathbf{1})\). If \(\mathbf{W}\mathbf{1}\notin\mathrm{span}\{\mathbf{1}\}\), then \(g\) is not TI; in the extreme case \(c\to+\infty\), the output distribution converges to a one-hot distribution concentrated on \(k^\star = \arg\max_k (\mathbf{W}\mathbf{1})_k\), independently of \(S(\mathbf{z})\). Thus, an arbitrary shift of the logits, which leaves the predictive distribution unchanged, can fully determine the calibrated prediction.

3.1.0.2 Making calibrators translation-invariant (TI).

We consider two mechanisms to enforce translation invariance. The first mechanism constrains the calibrator such that additive logit shifts only induce a constant shift in the pre-softmax outputs. For matrix scaling (MS), this can be achieved by enforcing that all rows of \(\mathbf{W}\) have identical sums (this variant is denoted MS\(_c\)). This constraint can be implemented via an unconstrained parameterization. Specifically, we learn a free matrix \(\mathbf{W}_{\mathrm{free}} \in \mathbb{R}^{C \times (C-1)}\) and a scalar \(s \in \mathbb{R}\). The full matrix \(\mathbf{W} \in \mathbb{R}^{C \times C}\) is obtained by appending a final column to \(\mathbf{W}_{\mathrm{free}}\), defined such that each row sums to \(s\).

The second mechanism consists of feeding the calibrator with canonical logits instead of their raw representatives. In particular, subtracting the free energy \(F(\mathbf{z})\) (with a sign convention opposite to some energy-based formulations) from \(\mathbf{z}\) yields a canonical representative of the equivalence class, removing the arbitrary global offset while preserving the predictive distribution: \[\label{eq:canon95logits} \log S(\mathbf{z}) = \mathbf{z} - F(\mathbf{z})\,\mathbf{1},\tag{8}\] \[\label{eq:free95energy} F(\mathbf{z}) = \log \sum_{c=1}^C e^{z_c}.\tag{9}\] Equation 8 shows that feeding a calibrator with \(\log S(\mathbf{z})\) is equivalent to subtracting the free energy \(F(\mathbf{z})\) from \(\mathbf{z}\), both ensuring translation invariance.

This viewpoint recovers Dirichlet calibration (DC) [10] as MS applied to canonical logits: \[g^{\mathrm{DC}}_{W,b}(\mathbf{z}) = S\bigl(\mathbf{W}\log S(\mathbf{z})+\mathbf{b}\bigr),\] and therefore DC is TI by construction. The same idea applies to LTS. The temperature predictor \(\tau_\theta\) can be made invariant to additive logit shifts by using log-probabilities as input instead of raw logits: \[g_{\theta}^{\mathrm{LTS}}(\mathbf{z}^v) = S\!\left( \mathbf{z}^v / \tau_\theta\bigl(v; \log S(\mathbf{Z}), \xi\bigr) \right).\]

3.2 Argmax/Order Preservation in Calibration↩︎

Since calibration and segmentation objectives are not fully aligned, improvements in calibration may come at the expense of segmentation accuracy. It remains an open question whether enforcing argmax preservation during calibration limits the achievable calibration gains. To investigate this question, we construct argmax- and order-preserving counterparts of standard calibrators to quantify the resulting calibration–segmentation trade-off. Among the standard multiclass post-hoc calibrators presented in Eqs. 3 –7 , only TS, ETS, and LTS are inherently argmax-preserving and order-preserving. When argmax- or order-preserving constraints are imposed, VS and MS collapse to TS (Appendix 6.2). Consequently, comparisons between VS/MS and TS do not isolate the effect of enforcing argmax preservation, as they simultaneously involve a severe reduction in model expressiveness.

To obtain meaningful argmax- and order-preserving counterparts of standard calibrators, one must ensure that the imposed constraints do not collapse the model to overly restrictive forms. This requires introducing additional flexibility beyond global transformations. We introduce a conditional calibration framework in which each class is equipped with a dedicated calibrator, selected at inference time according to the argmax of the input logits. Class-conditioning alone does not guarantee argmax or order preservation, since a selected calibrator may still alter the ranking of the logits. We therefore enforce argmax or order preservation within each class-specific calibrator, which ensures that the resulting global model inherits the same property.

3.2.1 Enforcing Argmax and Order Preservation for Conditional Class-wise Calibration↩︎

3.2.1.1 General case.

Let \(\Psi_1\) and \(\Psi_2\) denote the argmax and order cones, respectively: \[\begin{align} \Psi_1 &= \{ \mathbf{z} \in \mathbb{R}^C \mid z_1 \ge z_k,\;\forall k \neq 1 \}, \\ \Psi_2 &= \{ \mathbf{z} \in \mathbb{R}^C \mid z_1 \ge z_2 \ge \cdots \ge z_C \}. \end{align}\]

For any permutation \(\pi\), we denote by \(\mathbf{v}[\pi]\) the vector obtained by reordering \(\mathbf{v}\) according to \(\pi\): \[\mathbf{v}[\pi] = \bigl(v_{\pi(1)},\ldots,v_{\pi(C)}\bigr).\] We introduce a permutation \(\pi_{\mathbf{z}}\) associated with each input \(\mathbf{z}\), which induces the canonical reordering \(\mathbf{z}^{\mathrm{can}}=\mathbf{z}[\pi_{\mathbf{z}}],\) where \(\pi_{\mathbf{z}}\) is chosen such that \(\mathbf{z}^{\mathrm{can}}\in\Psi_1\) or \(\mathbf{z}^{\mathrm{can}}\in\Psi_2\), depending on whether argmax or order preservation is enforced. In the argmax-preserving case (\(\Psi_1\)), \(\pi_{\mathbf{z}}\) exchanges the first index with the index of \(\arg\max_k z_k\), while leaving all other indices unchanged. In the order-preserving case (\(\Psi_2\)), \(\pi_{\mathbf{z}}\) is defined such that the components of \(\mathbf{z}^{\mathrm{can}}\) are sorted in non-increasing order.

Let \(i(\mathbf{z})=\pi_{\mathbf{z}}(1)\) denote the class occupying the first position in the canonical ordering. The corresponding class-wise calibrator \(g_{i(\mathbf{z})}\) is then applied to \(\mathbf{z}^{\mathrm{can}}\), producing calibrated probabilities in canonical order. The final calibrated probabilities are obtained by applying the inverse permutation: \[g(\mathbf{z}) = \left( g_{i(\mathbf{z})} \left( \mathbf{z}[\pi_{\mathbf{z}}] \right) \right) [\pi_{\mathbf{z}}^{-1}], \qquad i(\mathbf{z})=\pi_{\mathbf{z}}(1).\]

Let \(h_i : \mathbb{R}^C \to \mathbb{R}^C\) denote the pre-softmax transformation associated with the \(i\)-th class-wise calibrator, so that \(g_i = S\circ h_i\). The conditional class-wise calibration is argmax-preserving (resp. order-preserving) if \(\Psi_1\) (resp. \(\Psi_2\)) is invariant under \(h_i\), i.e., \(h_i(\mathbf{z}) \in \Psi_1 \quad \forall \mathbf{z} \in \Psi_1\) (resp. \(\Psi_2\)).

The key advantage of the conditional formulation is that preservation constraints are enforced in a fixed canonical space, independent of the original permutation of logits. This is less restrictive than enforcing argmax or order preservation directly in the original logit space, where the location of the maximum and the ordering pattern may vary across samples. Note that this leads to a difference in the information available to the class-wise calibrators. In the argmax-preserving setting, \(g_i\) operates on inputs expressed in a fixed canonical order, where the mapping between positions and class identities remains consistent across samples. In contrast, in the order-preserving setting, only the first position is consistently associated with the selected class \(i\), whereas the remaining positions are associated with ranks rather than fixed class identities. In principle, one could consider extending the order-preserving formulation by learning separate calibrators for different ordering patterns, thereby explicitly modeling distinct rank configurations. In practice, however, this would lead to a combinatorial explosion in the number of possible ordering patterns, making it intractable.

3.2.1.2 Argmax preservation for conditional class-wise calibration under MS.

We derive the constraint for a single class-wise affine map and omit the class index \(i\) for readability. We first enforce set invariance of \(\Psi_1\) under MS in the absence of bias: \(h(\mathbf{z}) = \mathbf{W}\mathbf{z}\). We introduce the transformed coordinates \(\mathbf{x} = \mathbf{G}\mathbf{z}\), where \[\label{eq:G} \mathbf{G} = \begin{pmatrix} 1 & 0 & 0 & \cdots & 0\\ 1 & -1 & 0 & \cdots & 0\\ 1 & 0 & -1 & \cdots & 0\\ \vdots & & \ddots & \ddots & \vdots\\ 1 & 0 & \cdots & 0 & -1 \end{pmatrix} \in \mathbb{R}^{C \times C}.\tag{10}\]

In this coordinate system, the argmax cone \(\Psi_1\) is mapped to \[\Psi_3 = \{ \mathbf{x} \in \mathbb{R}^C \mid x_i \ge 0, i \ge 2 \}.\]

Invariance of \(\Psi_1\) under \(\mathbf{W}\) is equivalent to invariance of \(\Psi_3\) under \(\mathbf{W}' = \mathbf{G}\mathbf{W}\mathbf{G}^{-1}\), which holds if and only if \[\mathbf{W}' = \begin{pmatrix} a & \mathbf{r}^\top \\ \mathbf{0} & \mathbf{A} \end{pmatrix}, \quad a \in \mathbb{R},\; \mathbf{r} \in \mathbb{R}^{C-1},\; \mathbf{0} \in \mathbb{R}^{C-1},\; \mathbf{A} \in \mathbb{R}_+^{(C-1)\times(C-1)}.\]

We further extend the framework to the biased case (\(\mathbf{b} \neq \mathbf{0}\)). Since the bias term directly shifts the logits, preserving the argmax-cone structure requires the bias vector itself to lie in the same cone, i.e., \(\mathbf{b} \in \Psi_1\).

An important property of the resulting calibration method is its translation invariance, which can be verified by checking that \(\mathbf{W} \mathbf{1}_C = a \mathbf{1}_C\). Moreover, we derive from the previous results a fully differentiable and unconstrained parameterization. We parameterize a free matrix \(\tilde{\mathbf{A}} \in \mathbb{R}^{(C-1)\times(C-1)}\) and enforce non-negativity via a softplus transformation: \(\mathbf{A}_{ij} = \mathrm{softplus}(\tilde{\mathbf{A}}_{ij})\). We then construct \(\mathbf{W}' \in \mathbb{R}^{C\times C}\) by embedding \(\mathbf{A}\) into the lower-right block, setting the first column (except for \(\mathbf{W}'_{11}\)) to zero, and parameterizing the first row as an unconstrained vector in \(\mathbb{R}^C\). Then, \(\mathbf{W}\) is set to \(\mathbf{G}^{-1} \mathbf{W}'\mathbf{G}\). For the bias, we parameterize \(\tilde{\mathbf{b}} \in \mathbb{R}^C\) and enforce ordering via \(\mathbf{b}_1 = \tilde{\mathbf{b}}_1\) and \(\mathbf{b}_i = \mathbf{b}_1 - \mathrm{softplus}(\tilde{\mathbf{b}}_i)\) if \(i \ge 2\).

3.2.1.3 Order preservation for conditional class-wise calibration under MS.

Enforcing invariance of \(\Psi_2\) under MS follows exactly the same construction as for \(\Psi_1\) with: \[\mathbf{G} = \begin{pmatrix} 1 & 0 & 0 & \cdots & 0\\ 1 & -1 & 0 & \cdots & 0\\ 0 & 1 & -1 & \cdots & 0\\ \vdots & & \ddots & \ddots & \vdots\\ 0 & 0 & \cdots & 1 & -1 \end{pmatrix} \in \mathbb{R}^{C \times C}.\]

The bias vector must now satisfy \(\mathbf{b} \in \Psi_2\). To enforce this constraint, we parameterize a free vector \(\tilde{\mathbf{b}} \in \mathbb{R}^C\) and define \(\mathbf{b}_1 = \tilde{\mathbf{b}}_1\), \(\mathbf{b}_i = \mathbf{b}_{i-1} - \mathrm{softplus}(\tilde{\mathbf{b}}_i)\) if \(i \ge 2,\) which guarantees \(\mathbf{b}_1 \ge \mathbf{b}_2 \ge \cdots \ge \mathbf{b}_C\).

3.3 Instantiation of Segmentation/Calibration Pipelines↩︎

Logits provided to the calibrator are produced by a segmentation model. In this work, we consider deep ensembles [27], which are widely adopted in semantic segmentation to improve performance and predictive robustness. They typically yield higher Dice scores (DSC) and improved reliability compared to single networks [27]–[29]. However, improved segmentation performance does not necessarily translate into well-calibrated confidence estimates, and ensemble predictions are often still miscalibrated [4].

An ensemble consists of \(M\) independently trained segmentation models. At voxel \(v\), model \(m\) produces logits \(\mathbf{z}_m(v) \in \mathbb{R}^C\). Ensemble predictions are aggregated by a pooling operator acting either on logits \(\{\mathbf{z}_m(v)\}_{m=1}^M\) or on probabilities \(\{\mathbf{p}_m(v)\}_{m=1}^M\), yielding pooled logits \(\mathbf{z}(v)\) or pooled probabilities \(\mathbf{p}(v)\) that are passed to the calibrator. When pooling is performed in probability space, we provide \(\log \mathbf{p}(v)\) to the calibrator, ensuring consistency with the logit-based formulation.

We consider two standard pooling strategies: probability averaging, also known as mixture-of-experts fusion, and logit averaging, which induces a product-of-experts–type aggregation and penalizes inter-model disagreement more strongly. Pipelines based on probability averaging are denoted \(g(\bar{\mathbf{p}})\), whereas those based on logit averaging are denoted \(g(\bar{\mathbf{z}})\). Both pooling operators are invariant to member-wise additive logit shifts: adding \(\delta_m\mathbf{1}\) to all logits of model \(m\) leaves the resulting pooled probabilities unchanged. Moreover, \(g(\bar{\mathbf{p}})\) pipelines are TI by construction, since the calibrator operates on log-probabilities rather than logits, which guarantees translation invariance (Sec. 3.1).

We next introduce the pipelines considered in our experiments.

3.3.0.1 Uncalibrated baselines.

To assess the intrinsic impact of calibration, we consider three uncalibrated baselines: a single segmentation model, denoted \(p_0\), and a deep ensemble using either logit averaging (\(\bar{\mathbf{z}}\)) or probability averaging (\(\bar{\mathbf{p}}\)) for pooling.

3.3.0.2 Translation invariance.

To investigate the effect of translation invariance, we compare the non-TI pipeline MS(\(\bar{\mathbf{z}}\)) with two TI counterparts: MS\(_c\)(\(\bar{\mathbf{z}}\)) and DC(\(\bar{\mathbf{z}}\)). MS\(_c\) enforces translation invariance in logit space via a row-sum constraint on the weight matrix, while DC(\(\bar{\mathbf{z}}\)) can be interpreted as applying MS to canonical log-probabilities \(\log S(\bar{\mathbf{z}})\) (cf. Section 3.1). Note that MS becomes equivalent to DC when pooling is performed in probability space (\(\bar{\mathbf{p}}\)); therefore, it is not included in this comparison.

For LTS, translation invariance depends on the representation provided to the temperature network \(\tau_\theta\), as discussed in Section 3.1. We consider two variants: the original logit-based formulation of [25], which is not TI, and a probability-based variant that enforces it. The first is denoted LTS(\(\bar{\mathbf{z}}\)), and the second LTS(\(\log S(\bar{\mathbf{z}})\)), highlighting that the temperature network operates on canonical log-probabilities of the pooled predictions.

3.3.0.3 Argmax/Order preservation.

We denote CDC(\(\bar{\mathbf{p}}\)) and CDC(\(\bar{\mathbf{z}}\)) the conditional class-wise calibration schemes introduced in Section 3.2.1, where each class is equipped with a DC calibrator. Consequently, both pipelines are translation-invariant (TI) but neither order- nor argmax-preserving. We compare CDC(\(\bar{\mathbf{p}}\)) and CDC(\(\bar{\mathbf{z}}\)) with their respective order- and argmax-preserving counterparts, constructed using the algorithms presented at the end of Section 3.2.1. For probability pooling, CDC(\(\bar{\mathbf{p}}\)) is compared with CMS\(_{\mathrm{op}}\)(\(\bar{\mathbf{p}}\)) and CMS\(_{\mathrm{ap}}\)(\(\bar{\mathbf{p}}\)), where CMS\(_{\mathrm{op}}\) enforces order preservation and CMS\(_{\mathrm{ap}}\) enforces argmax preservation. Similarly, for logit pooling, CDC(\(\bar{\mathbf{z}}\)) is compared with CMS\(_{\mathrm{op}}\)(\(\bar{\mathbf{z}}\)) and CMS\(_{\mathrm{ap}}\)(\(\bar{\mathbf{z}}\)).

Note that CMS\(_{\mathrm{op}}\) and CMS\(_{\mathrm{ap}}\) are TI, whereas CMS is not. Therefore, comparing CMS\(_{\mathrm{op}}\) and CMS\(_{\mathrm{ap}}\) with CMS would conflate translation invariance with order or argmax constraints. In contrast, CDC is TI by construction and is thus used as the non–order-/non–argmax-preserving reference.

3.3.0.4 Standard pipelines.

We also evaluate standard temperature-based calibrators (TS, ETS), which preserve class ordering and are TI. We further include Dirichlet calibration (DC), which is TI by construction. This allows us to assess both the impact of the pooling operator and the role of calibrator expressiveness on performance.

4 Experiments and Results↩︎

4.1 Experimental Setup↩︎

4.1.0.1 Datasets.

We evaluate the calibration pipelines from Section 3.3 on three segmentation benchmarks spanning medical and natural-image domains: BraTS GLI 2024 [30] (3D brain MRI glioma segmentation), Cityscapes [31] (2D urban scene parsing), and Massachusetts Roads (RoadSeg) [32] (2D aerial road extraction). These datasets vary in modality (MRI vs.RGB), dimensionality (3D vs.2D), and number of classes, thereby enabling evaluation across heterogeneous imaging and semantic settings. Dataset statistics, split sizes, and the number of classes are summarized in Table 1. As shown in the table, we use a calibration set of size \(n=50\) for all experiments; a sensitivity analysis (Appendix 11) indicates that calibration metrics largely stabilize for \(n \ge 25\). Representative input images and ground-truth overlays are shown in Figure 2.

7pt

4.1.0.2 Segmenter training.

For each dataset, we train a deep ensemble of \(M=5\) nnU-Net [13] v2.4.1 models initialized with different random seeds. The 2D or 3D configuration is selected according to the dataset dimensionality, both using the ResEncM preset and default preprocessing. Optimization follows the nnU-Net defaults (SGD with momentum, polyLR schedule, deep supervision, and automatic patch and batch-size configuration). All ensemble members share identical preprocessing, data augmentation, and optimization settings. The ensemble is trained once per dataset on the official training split and remains fixed for all subsequent calibration experiments.

4.1.0.3 Post-hoc calibration protocol.

Calibration is performed post hoc on the fixed ensemble segmenter. For each dataset, we perform \(R=3\) independent repeats, each using a new random partition of the held-out images (Table 1) into calibration, validation, and test subsets. In each repeat \(r\), calibrator parameters are fitted on \(\mathcal{D}^{(r)}_{\mathrm{cal}}\) by minimizing the voxel-wise negative log-likelihood (NLL) of the pooled ensemble predictions with an additional regularization term. When applicable, early stopping and hyperparameter selection use \(\mathcal{D}^{(r)}_{\mathrm{val}}\), and final results are reported on \(\mathcal{D}^{(r)}_{\mathrm{test}}\). Details on optimization, regularization, and hyperparameter search are provided in Appendix 7.

4.1.0.4 Model parameterization and expressiveness

A common way to compare the expressiveness of calibration methods is to count trainable parameters. However, this can be partially misleading since most used models are overparameterized due to the softmax’s invariance to additive constants. It is therefore more meaningful to consider the number of identifiable parameters. For example, in the MS case, subtracting from each column of \(\mathbf{W}\) its first entry and subtracting \(b_1\) from \(\mathbf{b}\) leaves the induced softmax distribution unchanged. Consequently, without loss of generality, one may fix the first row of \(\mathbf{W}\) to zero and set \(\mathbf{b}_1 = 0\).

8pt

Table ¿tbl:tab:num95params? reports both the total number of parameters and the number of identifiable parameters for each calibrator. In particular, the identifiable parameter counts exploit the fact that any MS-based calibrator admits an equivalent parameterization with \(\mathbf{b}_1 = 0\) and a zero first row in \(\mathbf{W}\). For CMS\(_{ap}\) and CMS\(_{op}\), the first row of the matrix \(\mathbf{A}\) from which \(\mathbf{W}\) is computed can also be fixed to zero without loss of generality.

In practice, we retain the overparameterized parameterizations during optimization and account for these redundancies only when comparing model expressiveness. Importantly, overparameterized models are necessary during optimization due to the regularization term used during training (cf. Appendix 7). We use a Dirichlet-style regularizer, described in [10], that penalizes off-diagonal weights and large biases via an \(\ell_2\) norm. Under such a regularization, the identity mapping is not penalized; however, in the reduced parameterization presented above, the identity is not representable, and its equivalent representation has a first column given by \((0, -1, -1, \dots)^\top\), which is strongly penalized. This leads to different regularization behavior depending on the chosen parameterization.

4.1.0.5 Evaluation.

Segmentation accuracy is measured with the multiclass Dice score (DSC), which allows us to quantify whether calibration affects segmentation performance. For methods that may change the predicted class, we further report the class-flip rate, defined as the percentage of voxels whose predicted class changes after calibration. Calibration is assessed with negative log-likelihood (NLL), expected calibration error (ECE) [9], boundary-aware ECE (BA-ECE) [5], and Average Calibration Error (ACE) [2]. NLL is a strictly proper scoring rule and corresponds to the loss used to fit the calibration models. ECE is computed with \(B=50\) fixed-width confidence bins; a sensitivity analysis over \(B\in\{20,30,40,50\}\) shows that ECE values change only marginally and that the relative ranking across methods is stable (Appendix 12). BA-ECE evaluates calibration as a function of distance to the ground-truth boundary; details on distance bands and weighting are provided in Appendix 8. Since ECE weights bin-wise calibration gaps by bin occupancy, it can be dominated by highly populated confidence ranges. We therefore also report ACE, which averages calibration gaps uniformly over non-empty bins; we use \(B=15\), with a binning sensitivity analysis also reported in Appendix 12. All metrics, except ACE, are computed for each image in the held-out test set and then averaged across repeats. We report 95% confidence intervals obtained via a two-level hierarchical bootstrap [33] with \(N_{\mathrm{boot}}=2000\) replicates, resampling repeats first and then cases within each selected repeat. For ACE, image-level estimates can be unstable because some confidence bins may contain very few voxels. We therefore compute ACE by pooling voxels within each repeat and report the resulting repeat-level values as mean [min, max].

4.2 Pooled Logit Offset Variability↩︎

As discussed in Sec. 3.1, segmenter logits can be shifted by an arbitrary constant without affecting the resulting softmax probabilities. To analyze whether such shifts occur in practice, we compute the free energy (cf. Equation 9 ). Since such offsets can influence non-TI calibrators, this provides a practical diagnostic for whether enforcing translation invariance can matter. It is not, by itself, a predictor of the empirical gain from enforcing invariance: a non-TI calibrator may still learn an approximately invariant solution if the calibration data expose the relevant offsets. For each held-out image, we compute the spatial standard deviation of the pooled-logit free energy, \(\mathrm{Std}_v(F_{\mathrm{pool}}(v))\). Table 2 reports the mean across images for ensembles with \(M{=}5\) models. Spatial variability is substantial for all datasets and is markedly larger for BraTS and Cityscapes than for RoadSeg. The same trend holds for smaller ensembles (see Appendix 10).

12pt

Figure 3 shows representative spatial maps of \(F_{\mathrm{pool}}(v)\). These maps illustrate the spatial variability of the pooled-logit free energy across datasets.

4.3 Comparison of Post-Hoc Calibration Pipelines↩︎

We evaluate the calibration pipelines (Sec. 3.3) across the three segmentation benchmarks. Table ¿tbl:tab:results? summarizes the results in terms of calibration and segmentation metrics. For conciseness, only point estimates are reported; corresponding intervals are provided in Appendix 14. Note that the trends observed in-distribution persist under corruption-based covariate shift on Cityscapes, and are often amplified under stronger perturbations (Appendix 13):

2pt

4.3.0.1 1. Ensembles improve segmentation but do not reliably fix calibration. (Table ¿tbl:tab:results? (a))

Ensemble pooling consistently improves segmentation accuracy over a single model, although the magnitude of the gain depends on the dataset. On Cityscapes, DSC increases from \(71.7\) (\(p_0\)) to \(78.1\) with probability pooling (\(\bar p\)) and \(79.4\) with logit pooling (\(\bar z\)). On BraTS, the improvement is more moderate (\(75.2\) to \(76.0\)–\(76.9\)), while on RoadSeg it is marginal (\(76.3\) to \(76.7\)). Logit pooling yields slightly higher DSC than probability pooling on BraTS and Cityscapes, with negligible differences on RoadSeg.

For calibration, ensemble pooling improves or leaves metrics essentially unchanged, but the magnitude of the gains depends on the dataset and pooling strategy. Improvements are marginal on RoadSeg, more pronounced on Cityscapes with probability pooling, and mixed across metrics on BraTS. Overall, compared with a single model, ensembling improves or preserves segmentation and does not degrade calibration metrics in our experiments. However, the calibration gains remain modest compared with those obtained by dedicated post-hoc calibration methods, motivating the calibration pipelines evaluated next.

4.3.0.2 2. TI pipelines provide representation-consistent calibration.

We next evaluate whether enforcing translation invariance (TI) improves calibration in matched pipeline comparisons. Calibration quality also depends on other factors, in particular calibrator expressivity. It is therefore not surprising that, outside matched pairs, a non-TI method may outperform a TI method.

We first compare matrix scaling MS(\(\bar z\)) with its two TI counterparts MS\(_c\)(\(\bar z\)) and DC(\(\bar z\)) (Table ¿tbl:tab:results? (b)). Cityscapes gives the clearest effect: relative to MS(\(\bar z\)), both TI variants reduce NLL (from \(0.404\) to \(0.324\)–\(0.328\)), BA-ECE (from \(0.156\) to \(0.096\)–\(0.098\)), and ACE (from \(0.072\) to \(0.040\)–\(0.046\)). On BraTS, where ECE is dominated by the background class, class- and boundary-aware metrics show the same trend: MS\(_c\) improves BA-ECE from \(0.043\) to \(0.033\) and ACE from \(0.069\) to \(0.041\), while DC gives smaller gains. These gains are unlikely to be explained solely by differences in model complexity, as MS and DC have the same number of trainable and identifiable parameters, while MS\(_c\) has slightly fewer of both (Table ¿tbl:tab:num95params?). Rather, they further support the benefit of removing representation sensitivity. On RoadSeg, all affine variants are close, consistent with the lower pooled free-energy variability reported in Section 4.2. The binary setting also makes MS\(_c\) particularly restrictive: for \(C=2\), MS\(_c\) has only two identifiable parameters, compared with three for MS and DC (Table ¿tbl:tab:num95params?), which may explain why DC can be slightly preferable despite both variants being TI.

Taken together, these results support TI as a useful design principle for affine calibration: it makes calibration representation-consistent and improves or matches shift-sensitive counterparts. However, these affine calibrators are not constrained to preserve the argmax, so improving calibration can come at the cost of changing the predicted segmentation. Compared with the uncalibrated logit-pooled ensemble \(\bar z\), the three calibration pipelines have negligible DSC changes on RoadSeg (\(76.7\) to \(76.5\)–\(76.7\)), but large drops on BraTS (\(76.9\) to \(67.6\)–\(70.2\)) and Cityscapes (\(79.4\) to \(70.4\)–\(73.4\)). Thus, the calibration gains obtained by these affine methods are accompanied by substantial losses in segmentation accuracy on two of the three datasets. A flip audit for MS(\(\bar z\)) shows that harmful flips generally outnumber beneficial ones: 50.4% vs.% on Cityscapes and 74.8% vs.% on BraTS, with the remaining flips leaving voxel-level accuracy unchanged. Visual examples of the spatial distribution of these flips are provided in Appendix 9. This motivates the decision-preserving variants evaluated next in Point 3.

For LTS, we compare the original logit-based formulation LTS(\(\bar z\)) from [25], which does not guarantee translation invariance, with our TI variant LTS(\(\log S(\bar z)\)) (Table ¿tbl:tab:results? (c)). This is a matched comparison: both variants use the same local temperature network, have the same number of parameters (\(357\) on RoadSeg, \(882\) on BraTS, and \(3{,}507\) on Cityscapes; Table ¿tbl:tab:num95params?), and preserve the order by construction, so DSC is unchanged from the input segmentation. Across the three datasets, the TI variant improves almost all calibration metrics, with BA-ECE on Cityscapes remaining unchanged. The gains are particularly clear on RoadSeg, where ECE decreases from \(0.004\) to \(0.002\) and ACE from \(0.024\) to \(0.007\), and on BraTS, where BA-ECE decreases from \(0.134\) to \(0.090\). LTS(\(\log S(\bar z)\)) also improves ACE on BraTS (\(0.237\) to \(0.216\)), although ACE remains worse than for the uncalibrated baselines. We attribute this to the severe foreground–background imbalance: the voxel-wise NLL objective and ECE are dominated by easy background voxels, whereas BA-ECE and ACE emphasize sparse uncertain regions and tumor boundaries. The flexible local temperature map can therefore fit the dominant confidence regime without reliably correcting calibration on rare foreground structures. Overall, using canonical inputs makes LTS representation-consistent and improves calibration without increasing model capacity, highlighting the practical benefit of translation invariance.

4.3.0.3 3. Decision preservation exposes dataset-dependent calibration–segmentation effects (Table ¿tbl:tab:results? (d)).

We next isolate the effect of decision preservation within expressive affine calibration by comparing the unconstrained CDC pipeline with its argmax- and order-preserving counterparts, CMS\(_{\mathrm{ap}}\) and CMS\(_{\mathrm{op}}\). Unlike CDC, the CMS variants preserve the original segmentation map by construction and therefore retain the pooled DSC. Across all datasets, unconstrained CDC induces decision changes. These changes are small on RoadSeg and BraTS, where CDC flips at most \(0.6\%\) and \(0.1\%\) of pixels, respectively, but become substantial on Cityscapes, with about \(5\%\) of pixels changing label. Consequently, CDC consistently reduces DSC relative to the decision-preserving CMS variants, with the largest drops on Cityscapes (\(69.3\) vs.\(78.1\) for \(\bar p\), and \(74.2\) vs.\(79.4\) for \(\bar z\)), followed by BraTS (\(72.0\) vs.\(76.0\) and \(74.0\) vs.\(76.9\)). On RoadSeg, the effect is more modest, with DSC decreasing from \(76.7\) to \(75.7\).

The effect on calibration is both dataset- and metric-dependent. On Cityscapes, decision preservation does not reveal a clear calibration cost. While ECE is slightly increased or remains unchanged by the preservation constraints, both NLL and BA-ECE improve relative to CDC. In particular, CMS\({\mathrm{ap}}\) achieves the lowest NLL with both pooling strategies, whereas CMS\({\mathrm{op}}\) yields the best BA-ECE. ACE also improves with order preservation, although the behavior of argmax preservation is less consistent across pooling strategies. Overall, on Cityscapes, preserving the decision map restores the pooled DSC while also improving most calibration metrics. On BraTS, decision-preserving variants achieve very close NLL and ECE values compared with CDC, while improving BA-ECE. ACE is more mixed and often slightly worse, especially for CMS\(_{\mathrm{op}}\). Thus, preserving the decision map does not induce a clear degradation of calibration performance. RoadSeg is the only dataset where a clear calibration–segmentation trade-off is observed. CDC achieves the best calibration across all reported metrics. This may be partly explained by the binary nature of the problem. When C=2, the identifiable degrees of freedom decrease from 6 for CDC to only 4 for the decision-preserving CMS variants (Table ¿tbl:tab:num95params?), making the preservation constraints comparatively more restrictive.

Overall, enforcing argmax or order preservation prevents unintended segmentation changes by construction, but its calibration cost is not universal. In our experiments, a clear trade-off is observed only in the low-class RoadSeg setting, whereas on BraTS and Cityscapes the decision-preserving variants achieve comparable or even better calibration performance. This suggests that, when the conditional affine calibrator remains sufficiently expressive, decision-preserving constraints do not inherently compromise calibration and may even act as a beneficial regularizer. Conversely, in binary or other low-class settings, the same constraints can reduce calibration flexibility and expose a more explicit calibration–segmentation trade-off.

4.3.0.4 4. Practical implications for choosing a calibration pipeline.

When preservation of the deployed segmentation map is required by the downstream task, decision-preserving constraints are mandatory. When this requirement is relaxed, these constraints may still act as a beneficial regularizer for calibration, and constrained and unconstrained calibrators should therefore be systematically compared. In contrast, translation invariance is both a structural requirement for representation-consistent post-hoc calibration and, in our matched comparisons, an empirically beneficial constraint, which we therefore recommend by default.

Among TI pipelines, methods can be categorized along two dimensions: calibration flexibility and decision preservation. On both Cityscapes and BraTS, no single method consistently outperforms all others across all calibration metrics. Nevertheless, simple temperature-based methods (TS and ETS), which are also order-preserving, achieve excellent performance on Cityscapes. On RoadSeg, LTS(\(\log S(\bar z)\)), which provides a more flexible TI and order-preserving alternative to TS and ETS, yields the strongest ECE, BA-ECE, and ACE. On BraTS, CDC and the CMS variants perform particularly well. Overall, no single calibrator is uniformly optimal across datasets and evaluation metrics. Reliability diagrams in Appendix 15 provide additional qualitative support for these dataset-specific observations.

5 Conclusion↩︎

We showed that post-hoc calibration for semantic segmentation can be gauge-dependent: softmax logits are only identifiable up to an additive shift, and common calibrators can produce different outputs for equivalent representations of the same predictive distribution. Motivated by this, we introduced representation-consistent calibration and derived a simple condition—translation invariance—that distinguishes well-defined calibrators (e.g., TS, DC) from representation-dependent ones (e.g., MS/VS and LTS). We also presented practical ways to enforce translation invariance, including canonicalizing inputs via log-softmax and introducing new TI variants, such as constrained matrix scaling (MS\(_c\)). We further investigated the mismatch between training and calibration objectives and found that likelihood-driven calibration can change argmax decisions and degrade segmentation quality. To analyze this effect, we introduced class-conditional affine calibrators and their argmax- and order-preserving counterparts, CMS\(_{\mathrm{ap}}\) and CMS\(_{\mathrm{op}}\), enabling matched comparisons that quantify the calibration–segmentation trade-off induced by decision preservation. Across natural-image and medical segmentation benchmarks, including under covariate shift, translation invariance emerges as both a structural requirement and an empirically beneficial constraint in our matched comparisons. Moreover, enforcing argmax or order preservation may induce an implicit regularization effect that can benefit calibration. Together, these results establish translation invariance and decision preservation as two fundamental design axes for post-hoc calibration in segmentation.

Broader Impact Statement↩︎

By identifying translation invariance and decision preservation as structural properties of post-hoc calibration, this work provides practical design principles for future calibration methods in dense prediction. Translation invariance, in particular, is a lightweight, architecture-agnostic constraint that can be readily incorporated into a wide range of pipelines by using log probabilities instead of logits as input to the calibrator. Beyond its theoretical motivation, our results suggest that translation invariance is a beneficial design choice in practice and should therefore be considered by default when designing post-hoc calibration methods.

More generally, calibration plays a crucial role in safety-critical applications such as medical image segmentation by making predictive confidence better reflect observed accuracy, thereby improving downstream decision-making in both human and automated settings. It should therefore be viewed as a tool for decision support rather than a substitute for expert judgment.

Acknowledgments↩︎

This work of the ITI HealthTech, as part of the ITI 2021-2028 program of the University of Strasbourg, CNRS and Inserm, was partially supported by IdEx Unistra (ANR-10-IDEX-0002) and SFRI (STRAT’US project, ANR-20-SFRI-0012) under the framework of the French Investments for the Future Program. The authors would like to acknowledge the High Performance Computing Center of the University of Strasbourg for supporting this work by providing scientific support and access to computing resources. Part of the computing resources were funded by the Equipex Equip@Meso project (Programme Investissements d’Avenir) and the CPER Alsacalcul/Big Data.

Appendix↩︎

This appendix contains the following items:

• A. Additional Proofs

• B. Implementation and Training Details

• C. Spatial Calibration Assessment

• D. Spatial Patterns of Label Flips

• E. Sensitivity to Ensemble Size

• F. Sensitivity to Calibration Set Size

• G. Sensitivity to Binning

• H. Calibration under Corruption-Based Covariate Shift

• I. Complete Quantitative Results with Uncertainty Intervals

• J. Reliability Diagrams

We will publicly release the calibration code, hyperparameter-search configurations, evaluation scripts, and figure-generation commands.

6 Additional Proofs↩︎

6.1 Translation Invariance of Local Temperature Scaling↩︎

We analyze when Local Temperature Scaling (LTS) is translation-invariant. Let \[g_\theta^{\mathrm{LTS}}(\mathbf{z}(v)) = S\!\left( \frac{\mathbf{z}(v)}{\tau_\theta(v;\mathbf{Z},\xi)} \right),\] where \(\tau_\theta(v;\mathbf{Z},\xi)>0\) is a temperature predicted from the full logit field \(\mathbf{Z}\) and optional features \(\xi\). Consider a spatially varying additive shift \(c(v)\) from which we derive a full logit field \(\mathbf{Z}'\) with \(\mathbf{z}'(v)=\mathbf{z}(v)+c(v)\mathbf{1}\). If the temperature predictor is itself invariant to such shifts, we have: \(\tau_\theta(v;\mathbf{Z}',\xi)=\tau_\theta(v;\mathbf{Z},\xi)\).

For simplicity, and since this case is of limited practical relevance, we restrict ourselves to non-degenerate logits, i.e., \(\mathbf{z}(v) \notin \mathrm{span}\{\mathbf{1}\}\). In the degenerate case, where all class logits are equal, the softmax output is uniform and independent of the temperature, so any violation of shift invariance of \(\tau_\theta\) has no effect on the output of LTS.

Since softmax is invariant to additive constants, LTS is translation-invariant if the transformed logits differ only by an additive constant, i.e., \[\frac{\mathbf{z}'(v)}{\tau_\theta(v;\mathbf{Z}',\xi)} - \frac{\mathbf{z}(v)}{\tau_\theta(v;\mathbf{Z},\xi)} \in \mathrm{span}\{\mathbf{1}\}.\]

If the temperature predictor is itself invariant to such shifts, we get: \[\frac{\mathbf{z}'(v)}{\tau_\theta(v;\mathbf{Z}',\xi)} - \frac{\mathbf{z}(v)}{\tau_\theta(v;\mathbf{Z},\xi)} = \frac{c(v)}{\tau_\theta(v;\mathbf{Z},\xi)}\mathbf{1} \in \mathrm{span}\{\mathbf{1}\}.\] Thus, LTS is translation-invariant.

Conversely, suppose that for some location \(v\), \(\tau_\theta(v;\mathbf{Z}',\xi)\neq \tau_\theta(v;\mathbf{Z},\xi)\). Let \[\lambda'=\frac{1}{\tau_\theta(v;\mathbf{Z}',\xi)}, \qquad \lambda=\frac{1}{\tau_\theta(v;\mathbf{Z},\xi)}.\] Then \[\label{eq:s} \frac{\mathbf{z}'(v)}{\tau_\theta(v;\mathbf{Z}',\xi)} - \frac{\mathbf{z}(v)}{\tau_\theta(v;\mathbf{Z},\xi)} = (\lambda'-\lambda)\mathbf{z}(v)+\lambda'c(v)\mathbf{1}.\tag{11}\] Since \(\mathbf{z}(v)\notin \mathrm{span}\{\mathbf{1}\}\) and since \(\lambda'\neq\lambda\), the vector of Eq. 11 is not in \(\mathrm{span}\{\mathbf{1}\}\). Therefore, under the hypothesis that \(\mathbf{z}(v)\notin \mathrm{span}\{\mathbf{1}\} \forall v\), LTS is translation-invariant if and only if the temperature predictor is invariant to additive logit shifts.

Note that the original LTS formulation, whose temperature network takes raw logits as input, does not guarantee translation invariance. In contrast, feeding the temperature network with canonical log-probabilities \(\log S(\mathbf{Z})\) makes \(\tau_\theta\) invariant to additive shifts and yields a translation-invariant LTS pipeline.

6.2 Order- and Argmax-Preserving Affine Calibrators Collapse to Temperature Scaling↩︎

We first consider the bias-free case and write the calibrated probabilities as \(g(\mathbf{z})=S(\mathbf{W}\mathbf{z})\), where \(S\) is the softmax function. For each class \(i \in \{1,\dots,C\}\), we define the cone \(\Psi^{(i)}_1=\{\mathbf{z} \in \mathbb{R}^C \mid \mathbf{z}_i \ge \mathbf{z}_k,\;\forall k \neq i\}\), i.e., the set of logits whose maximum is attained at class \(i\). A matrix \(\mathbf{W}\) preserves the argmax if and only if each cone \(\Psi^{(i)}_1\) is invariant under \(\mathbf{W}\), i.e., \(\mathbf{z} \in \Psi^{(i)}_1 \Rightarrow \mathbf{W}\mathbf{z} \in \Psi^{(i)}_1\).

As shown in Section 3.2.1, the invariance of \(\Psi^{(1)}_1\) (denoted \(\Psi_1\) therein) under \(\mathbf{W}\) is equivalent to the existence of a matrix \(\mathbf{W}'\) such that: \(\mathbf{W} = \mathbf{G}^{-1}\mathbf{W}'\mathbf{G}\), with \(\mathbf{W}'=\begin{pmatrix} a & \mathbf{r}^\top \\ \mathbf{0} & \mathbf{A} \end{pmatrix}\), where \(a \in \mathbb{R}\), \(\mathbf{r} \in \mathbb{R}^{C-1}\), and \(\mathbf{A} \in \mathbb{R}_+^{(C-1)\times(C-1)}\), and \(\mathbf{G}\) defined in Eq. 10 . In the following, we denote by \(\mathbf{W}^{(i)}\) a matrix leaving \(\Psi^{(i)}_1\) invariant. For simplicity, we illustrate the structure of \(\mathbf{W}^{(1)}\) for \(C=4\) and proceed with the proof without loss of generality in this setting. A direct computation yields: \[\mathbf{W}^{(1)}= \begin{pmatrix} s & -r_1 & -r_2 & -r_3\\ s-\rho_2 & a_{11}-r_1 & a_{12}-r_2 & a_{13}-r_3\\ s-\rho_3 & a_{21}-r_1 & a_{22}-r_2 & a_{23}-r_3\\ s-\rho_4 & a_{31}-r_1 & a_{32}-r_2 & a_{33}-r_3 \end{pmatrix}, \quad s=a+\sum_{i=1}^3 r_i,\quad \rho_{i+1}=\sum_{j=1}^3 a_{ij}.\]

The structure of \(\mathbf{W}^{(1)}\) is fully determined by the following two constraints: (i) all rows have identical row sums; (ii) in each column \(j>1\), the first entry is a minimal element. These constraints in turn imply that the first entry of the first column is a maximal element.

The characterization of \(\mathbf{W}^{(i)}\) can be obtained from that of \(\mathbf{W}^{(1)}\) by simultaneously permuting the \(i\)-th row and column with the first row and column. For instance, exchanging rows and columns \(1\) and \(2\) yields \[\mathbf{W}^{(2)} = \begin{pmatrix} a_{11}-r_1 & s-\rho_2 & a_{12}-r_2 & a_{13}-r_3\\ -r_1 & s & -r_2 & -r_3\\ a_{21}-r_1 & s-\rho_3 & a_{22}-r_2 & a_{23}-r_3\\ a_{31}-r_1 & s-\rho_4 & a_{32}-r_2 & a_{33}-r_3 \end{pmatrix}.\]

By construction, \(\Psi^{(2)}_1\) is invariant under \(\mathbf{W}^{(2)}\). We now require consistency with the invariance of \(\Psi^{(1)}_1\), i.e., that the ordering constraints derived for \(\mathbf{W}^{(1)}\) remain valid under this permutation, while the row-sum condition is already preserved. We obtain in particular \(a_{12}=a_{13}=0\), together with \(\rho_2 \ge \rho_3\) and \(\rho_2 \ge \rho_4\). The corresponding constraints on the first column, namely \(a_{11} \ge a_{21}\) and \(a_{11} \ge a_{31}\), become redundant once the constraints from \(\Psi^{(3)}_1\) and \(\Psi^{(4)}_1\) are incorporated, as these imply \(a_{21}=a_{31}=0\).

Indeed, repeating the argument for \(\Psi^{(3)}_1\) and \(\Psi^{(4)}_1\) yields \(a_{21}=a_{23}=0\), \(\rho_3 \ge \rho_2\), \(\rho_3 \ge \rho_4\), and \(a_{31}=a_{32}=0\), \(\rho_4 \ge \rho_2\), \(\rho_4 \ge \rho_3\). Combining all these constraints, we conclude that \(a_{ij}=0\) for all \(i \neq j\), and \(\rho_2 = \rho_3 = \rho_4 = \lambda > 0\). Finally, argmax-preserving matrices take the form \[\label{eq:jenaimarre} \mathbf{W}= \begin{pmatrix} s & -r_1 & -r_2 & -r_3\\ s-\lambda & \lambda-r_1 & -r_2 & -r_3\\ s-\lambda & -r_1 & \lambda-r_2 & -r_3\\ s-\lambda & -r_1 & -r_2 & \lambda-r_3 \end{pmatrix}.\tag{12}\]

This can be expressed as \(\mathbf{W} = \lambda \mathbf{I} + \mathbf{1} \mathbf{c}^\top\), where \(\lambda > 0\) and \(\mathbf{I}\) denotes the identity matrix. The vector \(\mathbf{c} \in \mathbb{R}^C\) is defined by \(c_1 = s-\lambda\) and \(c_{i+1} = -r_i\) for \(i \in \{1,2,3\}\). In particular, \(\mathbf{c}\) is arbitrary since both \(s \in \mathbb{R}\) (corresponding to the free parameter \(a\) in \(\mathbf{W}'\)) and \(\mathbf{r} \in \mathbb{R}^{C-1}\) are unrestricted. In the general case (\(C \geq 2\)), the same reasoning applies within each of the \(C\) cones.

We now consider the affine case \(g(\mathbf{z})=S(\mathbf{W} \mathbf{z}+\mathbf{b})\). Since the argmax cones must remain invariant under translation, the bias vector must belong to all cones simultaneously, which implies \(\mathbf{b}=k\mathbf{1}\) (\(k\in\mathbb{R}\)).

Finally, using the translation invariance of the softmax, \(S(\mathbf{x} + k \mathbf{1})=S(\mathbf{x})\), we obtain \[S(\mathbf{W} \mathbf{z} + \mathbf{b}) = S(\lambda \mathbf{z} + \mathbf{1} \mathbf{c}^\top \mathbf{z} + k \mathbf{1} ) = S(\lambda \mathbf{z} ) = S\!\left(\frac{\mathbf{z}}{T}\right), \qquad T=\frac{1}{\lambda}.\]

Therefore, MS calibrators that preserve the argmax reduce exactly to TS. Since TS is also order preserving, the same conclusion holds under order preservation.

7 Implementation and Training Details↩︎

7.0.0.1 Additional implementation details.

All models are implemented in PyTorch [34]. The training loss is the voxel-wise cross-entropy (negative log-likelihood) for all methods, with the following Dirichlet-style regularizer described in [10] for affine calibrators (MS, MS\(_c\), DC) : \[\mathcal{R}(W,b) = \lambda \frac{1}{C(C-1)} \sum_{i\neq j} W_{ij}^2 + \mu \frac{1}{C} \sum_{j=1}^{C} b_j^2,\] which penalizes off-diagonal weights and large biases. For CMS, the regularization is applied for each one of the \(C\) models. For LTS, \(\ell_2\) regularization is applied, following [25].

Hyperparameters are optimized with Optuna [35] using a TPE sampler and a budget of 20 trials per learned calibrator, selecting configurations by validation NLL on the validation split of each calibration repeat. The search space is defined as follows:

• Learning rate (LR): log-uniform in \([10^{-4},\,3\times10^{-2}]\),

• Early stopping patience (P): in \(\{10,20,50\}\),

• Regularization strengths \(\lambda,\mu\): mixture of exact zero and log-uniform in \([10^{-6},\,10^{-1}]\),

• Weight decay \(\alpha\) for \(\ell_2\) regularization (LTS only): mixture of exact zero and log-uniform in \([10^{-6},\,10^{-2}]\).

Optimization uses Adam, early stopping on validation NLL, and ReduceLROnPlateau scheduling. The \(C\) class-wise experts of each class-conditional affine calibrator share the same hyperparameters. All calibrators are parameterized to represent identity (absence of calibration) at initialization.

7.0.0.2 Local Temperature Scaling (LTS).

Following [25], the temperature predictor \(\tau_\theta\) follows a tree-style gated convolutional design, combining multiple candidate temperature maps with learned gates. Positivity is enforced with a softplus transform, and convolutions are applied in-plane with kernel size \(5\) and dilation \(2\).

8 Spatial Calibration Assessment↩︎

We compute the boundary-aware expected calibration error (BA-ECE) following [5]. The main idea is to assess calibration in regions defined by their distance to the ground-truth boundary, so as to place more emphasis on locations where segmentation errors are typically more critical.

More precisely, let \(d(v)\) denote the unsigned Euclidean distance (in voxels) from voxel \(v\) to the ground-truth boundary, with \(d(v)=0\) on the boundary itself. We partition the image into distance bands \[D_k = \{ v \mid d(v)\in \Delta_k \},\] using the default intervals: \(\Delta_1=[0,3),\quad \Delta_2=[3,7),\quad \Delta_3=[7,15),\quad \Delta_4=[15,+\infty).\)

BA-ECE is then computed over these regions as in [5], using inverse-distance weighting so that regions closer to the boundary contribute more strongly. Empty bands are ignored.

9 Spatial Patterns of Label Flips↩︎

Calibration may change the predicted class if it is not argmax-preserving. MS(\(\bar{z}\)) is one such pipeline that may flip labels and alter the segmentation. Figure 4 provides illustrative examples on BraTS and Cityscapes datasets, showing that label flips are not uniformly distributed: they concentrate primarily along tumor and semantic boundaries. This occurs because likelihood-based calibration can locally alter the class ordering when margins are small, leading to spatially structured label changes.

10 Sensitivity to Ensemble Size↩︎

Table 3 reports \(\mathbb{E}[\mathrm{Std}_v(F_{\mathrm{pool}}(v))]\) for pooled ensembles of size \(M\in\{1,2,3,4,5\}\). The statistic varies primarily by dataset (RoadSeg lowest, Cityscapes intermediate, BraTS highest) and varies only slightly with \(M\). Importantly, substantial spatial variation is present for every tested ensemble size, including small ensembles (\(M=1,2\)), indicating that the offset variability motivating TI calibration is not an artifact of using \(M=5\).

6pt

11 Sensitivity to Calibration Set Size↩︎

We vary the number of held-out samples used to fit the post-hoc calibrator: \(n \in \{5, 25, 50, 75, 99\}\) for Cityscapes and \(n \in \{3, 14, 27, 41, 50\}\) for BraTS. The test split is kept fixed throughout; increasing \(n\) only changes the calibration/validation partition of the non-test pool. Performance is reported as the difference (\(\Delta\)) relative to the reference configuration used in our main experiments, corresponding to \(n=50\) samples for both datasets.

For computational efficiency, we consider a representative subset of calibrators covering different model capacities and constraints: TS(\(\bar{z}\)), MS(\(\bar{z}\)), MS\(_c\)(\(\bar{z}\)), LTS(\(\bar{z}\)), and CDC(\(\bar{z}\)).

Fig. 5 shows that performance is generally stable around the reference budget of 50 calibration samples. The largest variability occurs in the low-data regime, where the more expressive calibrators can exhibit increased NLL or calibration error and wider confidence intervals. Differences between moderate and larger calibration sets are small relative to the uncertainty bands, supporting our use of \(n=50\) as a pragmatic trade-off between stable calibration and preserving held-out data.

12 Sensitivity to Binning↩︎

Expected Calibration Error (ECE) [9] and ACE [2] both depend on the discretization of the confidence axis. To ensure our conclusions are not driven by a particular binning choice, we recompute ECE for \(B\in\{20,30,40,50\}\) bins and ACE for \(B\in\{15,20,30,40\}\) bins across datasets and methods (Fig. 6). Following Appendix 11, we restrict this sensitivity analysis to a representative subset of calibrators. Across datasets, changing the number of bins produces only small changes in ECE for most methods. ACE is more sensitive in absolute value because it averages calibration gaps uniformly over non-empty bins; increasing the number of bins can therefore amplify sparsely populated confidence regions. The range includes the two settings used in the main experiments: \(B=15\) for ACE, chosen as a compromise between resolution, stability, and readable reliability diagrams, and \(B=50\) for ECE. The qualitative comparisons used in the paper do not change with binning.

13 Calibration under Corruption-Based Covariate Shift↩︎

To assess robustness under covariate shift, we evaluate the Cityscapes calibrators from the main paper on corrupted test images. All calibrators are fitted on the clean in-distribution Cityscapes calibration set, exactly as in the main experiments, and are then evaluated without refitting on the corrupted test images. We focus on Cityscapes because standardized corruption protocols are available; RoadSeg shows limited sensitivity to calibration choices, and realistic covariate shifts for BraTS are beyond the scope of this work.

We generate two corrupted Cityscapes test sets with mild and strong severities, following the common-corruptions paradigm of ImageNet-C [36] and its adaptation to Cityscapes [37]. For each severity level, each image is transformed exactly once by sampling a perturbation type from a fixed perturbation bank using a deterministic hash of the filename and a fixed random seed. This yields a one-to-one correspondence between corrupted images and ground-truth labels and ensures reproducibility. We restrict perturbations to photometric transformations in order to induce controlled covariate shift while preserving pixel-level label alignment. The perturbation bank consists of Gaussian noise, Gaussian blur, brightness scaling, JPEG compression, and fog/haze. No geometric transformations are applied (Table 4).

Figure 7 shows representative corrupted examples under strong severity, and Table ¿tbl:tab:results95ood? reports the resulting calibration performance under corruption-based covariate shift. We discuss the main trends below.

14pt

2pt

13.0.0.1 1. Ensembles improve segmentation but do not reliably fix calibration. (Table ¿tbl:tab:results95ood? (a))

The same trends observed in-distribution persist under covariate shift. Probability pooling improves DSC over the single model (mild: 77.2 vs.; strong: 68.2 vs.), while logit pooling again achieves the highest segmentation accuracy (mild: 78.4; strong: 70.2). Pooling reduces some calibration errors under shift, especially for \(\bar p\), but the effect remains pooling- and metric-dependent; ensembling alone therefore does not reliably correct miscalibration under covariate shift.

13.0.0.2 2. TI pipelines appear more robust under shift. (Table ¿tbl:tab:results95ood? (b))

The in-distribution trends largely persist under covariate shift, but the gap between offset-sensitive and TI pipelines becomes more pronounced, especially for NLL and ECE. In the matched MS comparison, the unconstrained MS(\(\bar z\)) consistently underperforms its two TI counterparts MS\(_c\)(\(\bar z\)) and DC(\(\bar z\)). A similar pattern is observed for LTS: using the TI input representation LTS(\(\log S(\bar z)\)) improves NLL, ECE, and BA-ECE at both corruption severities. ACE is slightly worse under mild corruption (0.078 vs.), but better under strong corruption (0.090 vs.). Overall, these results suggest that TI pipelines are more robust under shift, with improvements in most comparisons, although the gains are not uniform across all metrics and severities.

13.0.0.3 3. For Cityscapes, decision preservation improves segmentation without compromising calibration. (Table ¿tbl:tab:results95ood? (d))

The pattern observed in-distribution for Cityscapes becomes more pronounced under strong shift: non-argmax-preserving pipelines can suffer large DSC drops. Under strong corruption, CDC(\(\bar p\)) drops to 57.7 DSC with an 11.1% flip rate, whereas CMS\(_{\mathrm{ap}}\)(\(\bar p\)) preserves the pooled segmentation and maintains 68.2 DSC, with improvements in most calibration metrics. A similar pattern holds for logit pooling. Overall, as in the in-distribution setting, enforcing decision preservation improves segmentation performance and also leads to improvements in most calibration metrics.

14 Complete Quantitative Results with Uncertainty Intervals↩︎

Table ¿tbl:tab:full95results? reports the quantitative results for all datasets, pipelines, and metrics, together with uncertainty intervals. The point estimates are identical to those reported in the main paper. Except for ACE, intervals are 95% percentile confidence intervals (CI) computed using a hierarchical bootstrap over the \(R=3\) repeats considered in the paper, resampling both repeats and cases within each repeat, following standard bootstrap methodology [33]. For ACE, the table reports the min–max range across repeat-level ACE values under the convention described in the evaluation protocol. The procedure was implemented using SciPy [38].

2pt max width=,max totalheight=.95,center

15 Reliability Diagrams↩︎

Figure 8 reports test-split reliability curves under the same convention as Table ¿tbl:tab:results?, with one panel per dataset comparing the single-model baseline, the uncalibrated pooling distribution, the best-ACE calibrated method selected without imposing order preservation, and the best-ACE order-preserving calibrated method.

References↩︎