Silent Failures in Federated Personalization of Foundation Models
January 01, 1970
Abstract
Foundation models are increasingly personalized on decentralized private data through federated learning and are now deployed at scale under growing regulatory requirements for post-market monitoring. We argue that this convergence creates a distinct and under-recognized class of trustworthiness failures, which we term “Silent Failures.” These include amplified bias, fairness collapse, and alignment erosion that may remain difficult to detect because federated learning’s privacy constraints limit visibility into model behavior. A landscape analysis of existing benchmarks reveals a structural divide. Federated benchmarks evaluate system performance but provide limited insight into model behavior, whereas centralized trustworthiness benchmarks assess behavior but require model access incompatible with federated privacy. We introduce a taxonomy of six silent failure modes arising from the interaction of foundation model personalization, dataset shift, and core federated constraints. Our analysis shows that privacy-preserving training alone is insufficient for trustworthy deployment. We conclude with a research agenda for privacy-preserving behavioral evaluation and propose that silent failures become a standard diagnostic category for trustworthy federated artificial intelligence.
<ccs2012> <concept> <concept_id>10010147.10010257</concept_id> <concept_desc>Computing methodologies Machine learning</concept_desc> <concept_significance>500</concept_significance> </concept> <concept> <concept_id>10010147.10010178</concept_id> <concept_desc>Computing methodologies Artificial intelligence</concept_desc> <concept_significance>500</concept_significance> </concept> <concept> <concept_id>10010147.10010178.10010219</concept_id> <concept_desc>Computing methodologies Distributed artificial intelligence</concept_desc> <concept_significance>500</concept_significance> </concept> </ccs2012>
1 Introduction↩︎
Foundation models (FMs) are increasingly personalized on decentralized private data through federated learning (FL) [1], [2], a paradigm we refer to as Federated Foundation Model Personalization (FedFMP). Parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) [3] enable on-device adaptation of large models, while FL preserves privacy by keeping training data local [4]. Recent methods support heterogeneous devices [5], federated instruction tuning [6], [7], and cross-domain fine-tuning [8].
Federated foundation models remain hard to audit behaviorally. The privacy constraints that make FL valuable – secure aggregation, limited visibility, and heterogeneous clients – limit the behavioral monitoring required for trustworthy deployment [9], [10]. Our contribution is to identify silent failures as a distinct failure category in FedFMP rather than as a single new failure mode. We refer to the failures that arise under these conditions as “Silent Failures” as summarized in Figure 1 and Table 1. These failures manifest as degradations in model behavior that remain difficult to detect as FL restricts access to training data and intermediate model signals. We question treating privacy-preserving training as sufficient.
| Failure Mode | Description | FM Perspective | Dataset Shift Factor | |
|---|---|---|---|---|
| Amplified Bias | Personalization on skewed local data can amplify biases; privacy constraints hinder centralized auditing of local disparities. | Local objective differences or biased prompts can exacerbate output disparities during adaptation. | Within-client subpopulation shifts and skewed label distributions. | |
| Confidence Miscalibration | Model confidence can decouple from accuracy on shifted data; global aggregation may mask localized overconfident errors. | Local fine-tuning may reinforce hallucination patterns, producing confidently incorrect generations. | Covariate or concept shifts that alter local data factuality. | |
| Fairness Collapse | Aggregating adapters tuned to distinct distributions can widen disparities. Affected subgroups may be too small to alter global metrics. | Personalization can overfit to prevalent local prompts, exacerbating the FM’s inherent prompt sensitivity. | Between-client divergence in prompt-style distributions across heterogeneous user populations. | |
| Adaptation Misalignment | Overfitting to narrow local objectives may degrade general capabilities and safety. The server may not distinguish adaptation from forgetting. | In PEFT-based personalization, small adapter updates can alter safety behavior encoded in the frozen backbone. | Deviations in local task objectives or rapid concept drift. | |
| Out-of-Domain Degradation | Performance can degrade outside the local specialization domain. Privacy limits cross-client OOD evaluation. | Over-specialization on narrow local data weakens the FM’s zero-shot generalization capabilities. | Sustained covariate shift narrowing the effective training distribution to a specialized local domain. | |
| 2-5 | Alignment Erosion | Local degradation of behaviors (safety, persona) may be masked by aggregation. The server lacks local-output access for audits. | Shifting local content may weaken safety guardrails and increase hallucination propensity. | Concept drift in local content that shifts the behavioral ground-truth. |
Three trends motivate this focus. (i) Deployment scale. Federated personalization is moving toward large client populations, where rare failures can still affect many users. (ii) Empirical risk. Recent studies show that fine-tuning on as few as ten examples can compromise safety alignment [11]. Benchmark evaluations may also be corrupted by data contamination even in centralized settings [12], [13]. These risks increase when training occurs across heterogeneous and partially observable client environments. (iii) Accountability. As artificial intelligence (AI) systems are deployed in high-stakes domains such as healthcare and finance, emerging governance frameworks increasingly require continuous post-deployment monitoring of model behavior [14]. Current federated evaluation tools provide limited support for such monitoring.
This paper argues that silent failures form a distinct problem category. Their defining property is that the privacy architecture of FL reduces the observability needed for detection, not only for mitigation. We show this gap through an analysis of twelve representative benchmarks spanning federated and centralized evaluation settings. Addressing these failures requires evaluation tools that operate under realistic privacy constraints and heterogeneous client conditions, which current frameworks only partly support.
2 Problem Formulation↩︎
Consider a federated network of \(K\) clients coordinated by a central server. Each client \(k\) personalizes a pre-trained FM \(\mathcal{F}\) with frozen parameters \(\theta_{\text{pre}}\) by training a small adapter \(\Delta\theta_k\) via PEFT, producing outputs \(\mathcal{F}(x;\theta_{\text{pre}},\Delta\theta_k)\). Then, client \(k\) holds a private dataset \(D_k^{(t)}\) drawn from a local distribution \(P_k^{(t)}(X,Y)\) that may shift over time. In each round, the client minimizes a local objective \[\begin{align} \Delta\theta_k^{(t+1)} \leftarrow \arg\min_{\Delta\theta} \mathbb{E}_{(x,y)\sim D_k^{(t)}} [\mathcal{L}(\mathcal{F}(x;\theta_{\text{pre}},\Delta\theta),y)] . \end{align}\]
The server aggregates updates via FedAvg \[\begin{align} \Delta\theta_{\text{agg}}^{(t+1)} = \frac{1}{K}\sum_{k=1}^{K}\Delta\theta_k^{(t+1)} . \end{align}\] We adopt this simple aggregation rule as a baseline for analyzing failure modes rather than as an optimal strategy.
The central challenge is dataset shift [15], [16]. Distributions may vary across clients (\(P_k \neq P_j\)) and drift within a client over time (\(P_k^{(t)} \neq P_k^{(t-1)}\)). Because FL limits the server’s local-data view, these shifts can be hard to observe. This information asymmetry, combined with the frozen backbone’s limited adaptability and simple aggregation, creates conditions for silent failures, where trustworthiness can degrade without triggering standard performance alarms.
3 Silent Failures from FedFMP↩︎
We identify six failure modes organized by causal layer in Figure 1 and Table 1. These failures span three causal layers, each with a distinct origin:
Data-level failures (amplified bias, confidence miscalibration) originate from skewed or noisy local inputs that can distort adapter updates.
Model-level failures (fairness collapse, adaptation misalignment) arise when aggregation combines adapters shaped by conflicting local objectives.
System-level failures (Out-of-Domain degradation, alignment erosion) appear as behavioral drift after deployment and accumulate across training rounds.
In all cases, FL privacy constraints reduce the observability needed for detection. Table 1 characterizes each failure from the perspectives of foundation models and dataset shift. Table 2 shows how three core FL constraints, aggregation, limited data observability, and statistical heterogeneity, amplify or conceal each mode.
| Failure Mode | Aggregation | Limited Data Observability | Statistical Heterogeneity |
|---|---|---|---|
| Amplified Bias | Biased local updates can compound when divergent demographic skews reinforce rather than cancel. | Local demographic and label distributions are hard to audit centrally. | Skewed local demographics or labels can lead adapters to encode different bias directions. |
| Confidence Miscalibration | Averaging locally overconfident adapters can smooth elevated ECE signals, concealing domain-specific miscalibration in the global model. | Local output factuality is hard to verify; global validation may lack domain specificity. | Varying data factuality and domain coverage across clients can produce heterogeneous calibration errors. |
| Fairness Collapse | Averaging adapters overfitted to divergent prompt styles can produce an uneven performance profile across clients. | Local prompt distributions and per-client subgroup performance are not visible to the central server. | Between-client divergence can specialize adapters for different populations, producing uneven behavior. |
| Adaptation Misalignment | Aggregating adapters that alter different safety-relevant subspaces can produce global incoherence within a round. | Local alignment decay is hidden from the server, allowing safety-relevant forgetting with limited detection. | Diverse local tasks can make adapters diverge, amplifying incoherence upon aggregation. |
| Out-of-Domain Degradation | Averaging over-specialized adapters can weaken cross-client generalization. | The server may not diagnose over-specialization, so OOD fragility can accumulate until deployment. | High heterogeneity can make each client’s data effectively OOD for others; FM specialization amplifies this fragility. |
| Alignment Erosion | Per-round accumulation of small behavioral degradations can compound before detection. | Without local outputs, per-client compliance scores are uncomputable at the server, so cumulative erosion can be missed. | Clients may drift toward different behavioral standards as their local content evolves, making global alignment a moving target. |
3.1 Amplified Bias↩︎
In FedFMP, bias can be amplified when clients personalize on skewed local data with conflicting objectives [17]. Let \(M(\mathcal{F}, s)\) denote a model-performance metric evaluated on a data slice \(s\), where \(s\) may be a demographic group or a client distribution; instances include accuracy, calibration, or generation-bias measures (e.g., toxicity, regard, gender polarity; [18]). Following standard group-fairness formulations [19], we measure bias as the disparity of \(M\) between two demographic groups \(g_a\) and \(g_b\): \[\begin{align} \text{Bias}(\mathcal{F}) = |M(\mathcal{F}, g_a) - M(\mathcal{F}, g_b)|. \end{align}\]
Amplification occurs when the aggregated model exhibits greater bias than the pre-trained baseline, i.e., \(\text{Bias}(\mathcal{F}_{\text{agg}}) > \text{Bias}(\mathcal{F}_{\text{pre}})\). Although averaging may seem to reduce bias, clients with biases in different directions produce conflicting updates whose aggregation can yield interference and global bias beyond any individual client’s [20], [21]. FL privacy limits auditability, so amplification may be hard to detect. As shown in Figure 1, amplified bias creates conditions that increase the risk of confidence miscalibration (Section 3.2) and fairness collapse (Section 3.3).
3.2 Confidence Miscalibration↩︎
Foundation models can produce fluent but incorrect content [22]. Under distribution shift, calibration studies show high-confidence errors and elevated ECE [23]. In FedFMP, local factual noise may reinforce such errors during personalization. A common measure of miscalibration is the Expected Calibration Error (ECE), following calibration literature [24], [25] and recent confidence-aware calibration work [26]: \[\begin{align} \text{ECE} = \sum_{m=1}^{M} \frac{|B_m|}{n} |\text{acc}(B_m) - \text{conf}(B_m)|. \end{align}\]
If a local distribution shift in \(P_k^{(t)}\) strengthens hallucinated patterns, the client adapter \(\Delta\theta_k\) can increase the local ECE. Aggregation of such adapters may then produce a global model \(\mathcal{F}_{\text{agg}}\) with high ECE. This failure is harder to detect in federated settings than in centralized ones. In centralized fine-tuning, domain-specific holdout sets can reveal elevated ECE before deployment. In FedFMP, privacy constraints limit inspection of client outputs, and the server often lacks domain-matched validation data to detect localized miscalibration [9], [10]. The ECE signal exists at the client level but may be obscured at the server, where aggregation smooths it out.
Miscalibrated confidence can feed back into adaptation misalignment, which is examined in Section 3.4. When local models report unreliable confidence estimates, these signals bias the feedback used for personalization. The resulting updates distort the optimization trajectory and gradually push the system away from the intended objective.
3.3 Fairness Collapse↩︎
Unlike bias amplification, which reflects demographic disparities within local data (Table 1), Fairness Collapse describes growing performance imbalance across clients caused by heterogeneous personalization. Prompt-based models are sensitive to prompt order and semantics [27], [28]. In FedFMP, local fine-tuning can adapt to prompt styles prevalent in majority user groups, while aggregation combines adapters specialized for different local distributions [4], [29], [30]. As FL restricts access to client data and outputs, the server may not observe local prompt or subgroup patterns, obscuring the resulting imbalance [31].
We model this collapse through the variance of \(M(\mathcal{F}, P_k^{(t)})\) across the \(K\) clients, where \(M\) is the performance metric defined in Section 3.1 with the client distribution \(P_k^{(t)}\) as the data slice. The disparity across clients is measured by the standard deviation \[\begin{align} \text{Disp}(\mathcal{F}) = \text{std}\bigl( M(\mathcal{F}, P_1^{(t)}), \dots, M(\mathcal{F}, P_K^{(t)}) \bigr). \end{align}\]
We identify fairness collapse when \(\text{Disp}(\mathcal{F}_{\text{agg}}) \gg \text{Disp}(\mathcal{F}_{\text{pre}})\) beyond legitimate cross-client distributional differences, indicating that aggregation has amplified rather than balanced performance disparities. This divergence arises because each adapter is optimized for a local distribution with distinct characteristics, and their aggregation amplifies cross-client disparities [4], [21].
This failure differs from the within-dataset demographic bias in Section 3.1. When clients represent distinct user populations, a global model may underserve some clients relative to others. This pattern creates uneven service quality across the federation. The fairness concern arises from structural heterogeneity between clients, not demographic attributes within a single dataset. Such disparity can contribute to OOD degradation in Section 3.5 when inter-client differences compound distributional mismatch.
3.4 Adaptation Misalignment↩︎
Adapting foundation models through federated PEFT introduces a risk that personalization either erodes general capabilities or overfits to narrow local objectives [32], [33]. In FedFMP, strong local adaptation signals can cause personalized models to diverge from their base alignment. When such adapters are aggregated across heterogeneous clients, the resulting global model may combine incompatible adaptations. Because FL privacy constraints obscure the internal state of local models, the server may struggle to distinguish beneficial adaptation from safety-relevant forgetting.
This failure can be formalized as a trade-off between a local personalization loss \(\mathcal{L}_{\text{local}}\) and a base alignment loss \(\mathcal{L}_{\text{base}}\) [33]. Misalignment arises when optimizing the adapter update \(\Delta\theta_k\) for the local objective degrades the base alignment: \[\begin{align} &\text{if } \Delta\theta_k = \arg\min_{\Delta\theta}\, \mathcal{L}_{\text{local}}\!\left(\mathcal{F}(x;\theta_{\text{pre}},\Delta\theta)\right), \\ &\text{then } \mathcal{L}_{\text{base}}(\mathcal{F}_k) \gg \mathcal{L}_{\text{base}}(\mathcal{F}_{\text{pre}}), \end{align}\] where \(\mathcal{F}_k\) denotes the personalized model. Although adapter parameters are compact (\(|\Delta\theta_k| \ll |\theta_{\text{pre}}|\)), PEFT updates concentrate on high-impact parameter subspaces (Table 1). Small adapters can alter safety-critical behavior encoded in the frozen backbone. This vulnerability appears even in centralized fine-tuning settings [11] and is relevant to alignment objectives studied in RLHF [34]. Aggregating adapters that forget different aspects of the base alignment may therefore produce a globally inconsistent model [4], [21].
Misaligned adapters can exacerbate OOD degradation in Section 3.5 and feed back into alignment erosion in Section 3.6 over successive rounds.
3.5 Out-of-Domain Degradation↩︎
In FedFMP, intensive personalization can cause a model to overfit to a client’s narrow local data distribution. The resulting model performs well locally but degrades on data from other clients [35], [36]. Because FL environments are highly heterogeneous, clients may effectively act as out-of-domain environments for each other. At the same time, privacy constraints limit server diagnosis of the over-specialization that leads to this failure.
The failure appears when a personalized model \(\mathcal{F}_k\), trained on client \(k\)’s distribution \(P_k^{(t)}\), is evaluated on the distribution of a different client \(j\): \[\begin{align} M(\mathcal{F}_k, P_j^{(t)}) \ll M(\mathcal{F}_k, P_k^{(t)}). \end{align}\] This behavior reflects a well-known challenge in heterogeneous FL [29]. When such over-specialized adapters are aggregated, the resulting global model \(\mathcal{F}_{\text{agg}}\) can exhibit poor generalization. This limitation motivates research on shift-robust personalization [37] and federated OOD learning frameworks [38].
In FedFMP, it can also erode the zero-shot generalization capability of the foundation model, creating a trade-off that does not arise in task-specific models. OOD degradation marks a convergence point in the taxonomy. It can arise from fairness collapse and adaptation misalignment and contribute to alignment erosion.
3.6 Alignment Erosion↩︎
Alignment erosion refers to the gradual degradation of a model’s safety alignment, persona, or factual reliability even when task accuracy remains high [34], [39]. In FedFMP, this occurs when local personalization weakens safety guardrails due to shifts in local content or evaluation criteria [11], [40]. Aggregation can mask these small degradations, while FL privacy constraints limit inspection of local outputs that might reveal behavioral violations.
We model this phenomenon as concept drift [41]–[43] in adherence to a set of behavioral constraints \(\mathcal{C}=\{c_1,c_2,\dots\}\). Let \(S(\mathcal{F},c)\in[0,1]\) denote a score that measures compliance with constraint \(c\), computed using a reward model trained via RLHF [34]. Motivated by hallucination and safety-degradation evidence [11], [22], erosion occurs when optimizing a local objective \(\mathcal{L}_k^{(t)}\) on a shifted distribution \(P_k^{(t)}\) produces a personalized model \(\mathcal{F}_k\) with reduced compliance, \(S(\mathcal{F}_k,c) < S(\mathcal{F}_{\text{pre}},c)\). Because the server may not compute \(S(\mathcal{F}_k,c)\) for each client, aggregation of these slightly degraded adapters may proceed with limited detection. Because each round’s aggregated model \(\mathcal{F}_{\text{agg}}^{(t)}\) initializes the next local training round, degradations may compound: over \(T\) rounds, even small decrements \(\delta S\) can accumulate so that \(S(\mathcal{F}_{\text{agg}}^{(T)},c) \ll S(\mathcal{F}_{\text{pre}},c)\) [9], enabled by the aggregation and observability constraints in Table 2.
Unlike adaptation misalignment (Section 3.4), which emerges from a single round of local optimization, alignment erosion can accumulate across many aggregation rounds, fed by upstream OOD degradation and misaligned adapters.
4 The Monitoring Gap in Federated Systems↩︎
| Benchmark Name | Primary Focus | Amplified Bias | Confidence Miscalibration | Fairness Collapse | Adaptation Misalignment | Out-of-Domain Degradation | Alignment Erosion | |
|---|---|---|---|---|---|---|---|---|
| LEAF [44] | FL Heterogeneity | \(\triangle\) | – | – | \(\triangle\) | – | ||
| FedScale [45] | FL Systems | – | – | \(\triangle\) | – | \(\triangle\) | – | |
| FLamby [46] | FL Healthcare | \(\triangle\) | – | \(\triangle\) | \(\triangle\) | \(\triangle\) | – | |
| FedLLM-Bench [7] | FL LLM Tuning | \(\triangle\) | – | \(\triangle\) | \(\triangle\) | \(\triangle\) | ||
| FLHetBench [47] | FL Systems | – | – | – | – | \(\triangle\) | – | |
| PFLlib [48] | FL Personalization | – | – | – | – | – | – | |
| HELM [49] | Holistic Evaluation | – | – | \(\triangle\) | \(\triangle\) | |||
| BIG-bench [50] | Reasoning | \(\triangle\) | – | – | \(\triangle\) | – | ||
| DecodingTrust [51] | LLM Trust | – | \(\triangle\) | – | ||||
| JailbreakBench [52] | Safety | – | – | – | – | – | ||
| TrustLLM [53] | Trust Evaluation | – | – | – | ||||
| HarmBench [54] | Red Teaming | – | – | – | – | – |
FedFMP still lacks mature behavioral monitoring. This section explains why. We examine the conflict between FL privacy and behavioral evaluation, the mechanisms that mask failures, and the evidence from existing benchmarks.
4.0.0.1 The Foundational Conflict from FL Constraints.
The monitoring gap arises from a structural difference between centralized and federated systems. In centralized settings, monitoring relies on direct access to data, model internals, and ground-truth labels [41]–[43]. Even with this access, reliable evaluation remains difficult. Benchmark data can leak into model training [12]. Reported performance may reflect contamination rather than generalization [13]. Detecting contamination without access to training data remains difficult [55]. In contrast, federated learning enforces privacy-by-design constraints that restrict such inspection [9]. As a result, many established monitoring methods become difficult to apply [10]. This difference in observability has produced two distinct evaluation paradigms that are difficult to reconcile.
4.0.0.2 Systemic Mechanisms of Detection Failure.
The interaction between FM behavioral failures (Table 1) and FL constraints (Table 2) can conceal silent failures. Two effects are central. First, privacy introduces a trade-off with monitoring resolution. Secure aggregation protects client data by hiding individual updates [10], [56], but this aggregation masks local signals that often reveal emerging failures [9], [57]. Second, reduced visibility creates statistical masking. Failures that affect a small subset of clients may not appear in global metrics. For example, fairness collapse in a minority population can remain hidden if its impact on the global average is small [20], [31]. Similar masking can occur for miscalibration or catastrophic forgetting [23], [32].
4.0.0.3 Benchmark Landscape Analysis.
This structural limitation appears in the benchmark ecosystem. To examine the issue empirically, we analyze the current evaluation landscape, summarized in Table 3. Prior surveys [58]–[60] mainly catalogue benchmark coverage. Our analysis instead evaluates how benchmarks capture specific failure modes. The results suggest a consistent divide. Federated benchmarks measure system-level performance but offer limited visibility into model behavior. Centralized trustworthiness benchmarks measure behavioral properties but assume direct model access, which conflicts with FL privacy constraints. We find no single framework that spans both paradigms. The monitoring gap therefore reflects a structural limitation rather than a simple oversight.
Each benchmark in Table 3 is rated on a three-point scale. A check mark () indicates direct measurement, meaning the failure mode metric is reported by the benchmark’s official evaluation scripts. A triangle (\(\triangle\)) indicates proxy or partial coverage, where the metric can be derived from standard outputs with limited post-processing or applies only to a subset of tasks. A dash (–) indicates that the benchmark’s design does not directly address the failure mode. Note that these ratings involve interpretive judgment and may reflect subjective decisions.
4.0.0.4 Distinction from Adversarial Robustness.
Silent failures differ from recent research on red teaming [61], [62] and jailbreaking [63]–[65]. Adversarial studies examine failures caused by external attacks. Silent failures arise from internal system dynamics during federated personalization. Adversarial evaluation assumes direct access to the model for probing. Silent failures occur when such access is unavailable because FL privacy constraints restrict observability. This lack of visibility defines the monitoring gap.
5 Research Vision for Trustworthy FedFMP↩︎
The monitoring gap identified in Section 4 reflects a structural property of FL more than a simple implementation limitation. Privacy constraints restrict the visibility needed to evaluate model behavior. This paper therefore focuses on defining the problem space rather than proposing a single technical fix.
5.0.0.1 Robust Aggregation and Personalization.
Standard aggregation methods optimize for predictive performance but can produce global models whose behavioral properties match no client’s local distribution. Robust averaging, architecture-aware grouping [56], [66], [67], and personalization with explicit failure-mode constraints [30], [68] offer partial remedies. The central tension is whether an aggregation rule can bound cross-client fairness disparity without sacrificing the local adaptation that makes federated personalization useful in the first place.
5.0.0.2 Reliable Monitoring and Auditing.
Detecting behavioral degradation in federated systems requires monitoring methods that operate without direct access to client data or model internals. This is both a tooling gap and a structural consequence of privacy-by-design: secure aggregation conceals the individual updates that most commonly signal emerging failures [57]. Progress requires federated auditing protocols that can assess fairness, calibration, and alignment from aggregate signals alone [10], [31].
5.0.0.3 Adaptive Personalization and Continual Learning.
Most personalization strategies assume stable data distributions. In practice, client data evolve, and static adapters can drift behaviorally over time. Continual learning mechanisms for mitigating catastrophic forgetting [32], [33] offer a starting point. How to integrate shift-aware personalization [37] with federated continual learning under non-stationary client data, while preserving privacy guarantees, remains an open problem [1], [69].
5.0.0.4 Rigorous Benchmarking and Explainability.
Existing frameworks such as OpenFedLLM [6] and FlowerTune [8] provide strong baselines for system-level evaluation. They do not yet treat fairness, calibration, or OOD robustness [35] under realistic heterogeneous shifts [36], [70] as first-class metrics. A parallel direction studies federated explainability. When direct inspection of client data is unavailable, interpretability applied to aggregated model behavior may partially recover the observability that privacy removes. This approach can turn a constraint into a diagnostic signal.
5.0.0.5 A Vision for Success.
If this agenda succeeds, federated AI evaluation will treat behavioral diagnostics as a core concern alongside system performance. Silent failures will become a recognized diagnostic category. Benchmark suites will measure behavioral properties under privacy constraints. FL frameworks will include privacy-preserving behavioral auditing as a standard capability rather than a bespoke post hoc tool. The result will be a federated AI ecosystem where privacy-preserving training is paired with evidence for trustworthy deployment.
6 Conclusion↩︎
Federated personalization of foundation models creates silent failures: behavioral degradations that current evaluation tools often miss. Our taxonomy and twelve-benchmark analysis suggest that this gap is structural. Federated benchmarks track system performance, while centralized trustworthiness benchmarks assume model and data access that FL restricts.
The path forward is not to abandon federated personalization, but to evaluate it differently. Privacy-preserving training should be paired with privacy-preserving checks for fairness collapse, miscalibration, OOD degradation, and alignment erosion. FedFMP systems should therefore be judged not only by whether they train efficiently across clients, but by whether their behavioral failures can be audited under the same privacy constraints that govern training.
Our analysis builds on benchmarks developed by many research groups in federated learning and trustworthiness evaluation. The monitoring gap we identify reflects the independent evolution of these two fields, not a limitation of any single benchmark. We hope this work encourages closer exchange between them.
Dr. YongKyung Oh was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (RS-2024-00407852).
7 Benchmark Paper Overviews↩︎
This appendix summarizes the benchmark papers used in Table 3.
7.1 Federated Benchmarks↩︎
7.1.0.1 LEAF
This modular benchmark suite covers federated settings through datasets, protocols, and reference implementations built around statistical and systems heterogeneity [44]. It captures client imbalance and variation. Behavioral failures such as alignment erosion or confidence miscalibration are outside its scope.
7.1.0.2 FedScale
This benchmark combines federated datasets with a scalable runtime for tasks including image classification, object detection, language modeling, and speech recognition [45]. Its evidence is primarily systems-level. Trustworthiness appears only through proxies such as heterogeneity and robustness.
7.1.0.3 FLamby
This benchmark targets cross-silo federated learning in healthcare using natural institutional splits, multiple modalities, and baseline implementations [46]. These splits expose fairness and out-of-domain risks across clinical sites. Foundation-model alignment and safety behavior lie outside its scope.
7.1.0.4 FedLLM-Bench
This benchmark evaluates federated learning for large language models using datasets for instruction tuning and preference alignment, together with multiple training methods and metrics [7]. It is the only federated benchmark in this set that explicitly covers LLM tuning and preference alignment. The metrics center on training outcomes rather than privacy-preserving behavioral monitoring.
7.1.0.5 FLHetBench
This benchmark studies device and state heterogeneity in federated learning through sampling methods and metrics for client availability and resource variation [47]. For silent failures, its role is system-level: it shows how availability and resource constraints can mask problems. It does not test demographic, safety, or alignment degradation.
7.1.0.6 PFLlib
This benchmark library covers many FL and pFL algorithms across heterogeneous scenarios and datasets [48]. It supports evaluation of personalization and cross-client generalization. The reported outcomes remain algorithmic rather than behavioral.
7.2 Centralized Trustworthiness Benchmarks↩︎
7.2.0.1 HELM
This benchmark evaluates language models across scenarios, metrics, and behaviors including accuracy, calibration, robustness, fairness, bias, and toxicity [49]. Its behavioral coverage is wider than that of the federated benchmarks, although the centralized setup assumes access to inputs and outputs that federated privacy constraints may restrict.
7.2.0.2 BIG-bench
This benchmark collects a large set of tasks for measuring and extrapolating language-model capabilities [50]. It probes reasoning and capability limits. Federated heterogeneity, personalization, and deployment-time trustworthiness failures are not its target.
7.2.0.3 DecodingTrust
This benchmark evaluates GPT-model trustworthiness across toxicity, stereotype bias, adversarial robustness, privacy, ethics, fairness, and out-of-domain robustness [51]. It covers many behavioral risks. It does not model the federated mechanism by which local failures can disappear during aggregation.
7.2.0.4 JailbreakBench
This benchmark standardizes evaluation of jailbreak attacks and defenses through open adversarial prompts, a fixed behavior set, scoring functions, and a public leaderboard [52]. It informs alignment-erosion analysis. Its unit of evaluation is adversarial safety rather than federated personalization or client-level monitoring.
7.2.0.5 TrustLLM
This trustworthiness benchmark covers large language models across truthfulness, safety, fairness, robustness, privacy, and ethics [53]. These categories overlap with silent failures. The evaluation, however, does not model federated privacy constraints.
7.2.0.6 HarmBench
This benchmark evaluates automated red teaming and robust refusal across attack methods, target models, and harmful behavior categories [54]. It is relevant to alignment and safety failures. It does not study how those failures arise or stay hidden during federated personalization.