1 Introduction↩︎

Federated Learning (FL) has emerged as a prominent paradigm for privacy-preserving collaborative machine learning, particularly in scenarios where centralized data collection is infeasible due to regulatory, organizational, or ethical constraints. By keeping raw data local and exchanging only model updates, FL enables learning across distributed data silos while reducing direct data exposure risks [1]. This privacy-by-design motivation and the broader framing of FL for explainable AI models are explicitly discussed in the foundational perspective on Federated Explainable Artificial Intelligence (FedXAI) [2].

Despite these advantages, FL alone does not address a fundamental limitation of modern machine learning: the lack of transparency and interpretability of learned models. In practice, many real-world federated deployments still rely on black-box models, making it difficult for stakeholders to validate decisions, audit model behavior, or understand failure modes under heterogeneous clients. This issue becomes especially evident in realistic settings with non-IID data, where both model behavior and explanation consistency can vary across clients, as empirically illustrated in distributed Intrusion Detection Systems (IDS) evaluation with SHAP-based analysis [3].

Explainable Artificial Intelligence (XAI) is commonly used to enhance transparency by providing human-understandable explanations of predictions. However, when considered in isolation, XAI often assumes centralized access to data, model internals, or repeated querying–assumptions that conflict with the decentralized and privacy-constrained nature of FL. Moreover, explanation artifacts themselves may introduce additional privacy and security risks in federated settings; for example, FL-IDS studies highlight that explanations can become a new leakage surface and motivate secure/controlled aggregation of explanation signals [4]. These limitations indicate that explainability cannot be treated as a purely post-hoc add-on, but must be redesigned to operate under federated constraints.

The need to jointly address privacy and explainability becomes most evident in high-stakes domains, where model outputs can trigger irreversible actions, regulatory consequences, or safety-critical interventions. Across the FedXAI literature, this co-design motivation repeatedly appears in healthcare, finance, IDS, and energy systems. In healthcare, FL combined with SHAP-based explanations has been used to provide clinically interpretable decision support while keeping patient data local [5]. In finance, federated fraud detection pipelines integrate SHAP to support transparency and auditability under inter-bank data silos [6]. In cybersecurity and IDS, secure aggregation of explanation signals is leveraged to provide trustworthy interpretations without exposing raw traffic data [4]. In energy systems, explainability is integrated into clustered FL to handle heterogeneity and deliver interpretable forecasting drivers (e.g., radiation and meteorological factors) [7]. Across these domains, FedXAI emerges as a natural solution in settings where data cannot be centralized and decision-making processes must remain human-validated.

Despite the growing body of work at the intersection of FL and XAI, existing surveys remain insufficient to capture the full scope of this emerging paradigm. Prior survey-style discussions either emphasize FL/XAI separately or provide high-level conceptual framing without systematically organizing design choices, evaluation dimensions, and deployment constraints–an issue acknowledged in early FedXAI survey perspectives [1]. Furthermore, the rapid expansion of FedXAI research has resulted in fragmented methodologies, inconsistent evaluation practices, and limited agreement on standardized benchmarks and explanation metrics. In addition, evaluation remains fragmented: metrics for explanation quality, consistency under non-IID data, privacy leakage risk from explanations, and the cost of generating/sharing explanations are not yet standardized, making cross-paper comparison difficult.

Several recent surveys have investigated FL, XAI, and their intersection from different perspectives. Early FedXAI discussions mainly provided conceptual motivations for integrating explainability into privacy-preserving federated systems [1]. Later studies explored this intersection in domain-specific contexts. For example, [8] discussed FL and XAI as promising future directions for IoT intrusion detection systems, while [9] focused on explainable and federated AI in healthcare and clinical decision support. Trust-oriented perspectives were further explored in [10], where explainability was considered alongside fairness, privacy, robustness, and accountability as components of trustworthy FL. Similarly, [11] reviewed the integration of FL and XAI for Internet of Medical Things (IoMT) applications, emphasizing privacy preservation and interpretability in smart healthcare systems. More recently, [12] reviewed FedXAI for next-generation IoT environments, focusing on scalability, transparency, communication efficiency, and trust in decentralized systems.

Although these surveys provide valuable insights, most existing studies either focus on specific application domains or discuss explainability as a secondary trust component rather than a core design dimension throughout the FL lifecycle. Furthermore, current FedXAI research still lacks unified taxonomies, standardized evaluation protocols, and consistent benchmarking strategies under heterogeneous and privacy-constrained settings. In contrast, this survey adopts a unified taxonomy-driven and evaluation-aware perspective that systematically analyzes explainability across federated architectures, training workflows, evaluation methodologies, and deployment scenarios.

The main contributions of this survey are summarized as follows:

• A multi-dimensional taxonomy is introduced to systematically categorize FedXAI methods across learning paradigms, explainability types, and integration levels within the federated pipeline.

• A lifecycle-oriented perspective is presented to analyze how explainability interacts with different stages of FL, including local training, aggregation, communication, and deployment.

• Key evaluation gaps and methodological limitations in current FedXAI studies are identified and analyzed, with particular emphasis on metrics, benchmarks, and experimental protocols under non-IID and privacy-constrained settings.

• An open challenges roadmap is synthesized, outlining critical research directions toward robust, privacy-preserving, and trustworthy FedXAI systems.

Figure 1 provides an overview of the logical structure of this survey. The proposed multi-axis taxonomy is positioned as the central element, guiding the organization of methodological approaches, evaluation criteria, datasets, and application domains. This structure enables a coherent and systematic analysis of FedXAI, from foundational concepts to practical deployment challenges and future research directions.

2 Background and Preliminaries↩︎

2.1 Federated Learning Fundamentals↩︎

FL is a distributed machine learning paradigm in which multiple clients–such as mobile devices, organizations, or edge nodes–collaboratively train a shared global model without exchanging their raw data. This concept, popularized by McMahan et al. [13], enables learning in privacy-sensitive and geographically distributed environments by transmitting only model updates, parameters, or gradients rather than local raw data [14]. By design, FL reduces direct risks of data exposure while still allowing models to benefit from decentralized and heterogeneous data sources.

The rapid adoption of FL is driven by both technological and regulatory factors that limit the feasibility of centralized data collection. Increasing concerns about data privacy, coupled with stringent legal frameworks such as the GDPR, restrict unrestricted data aggregation across organizations [5]. At the same time, the massive volume of data generated by personal devices, sensors, and enterprise systems makes centralized training inefficient or impractical. As a result, many real-world environments are characterized by isolated “data silos” that cannot be shared due to privacy, regulatory, or organizational constraints. FL addresses these challenges by enabling collaborative model training while ensuring that data remain local and under the control of their owners [15].

A standard mathematical formulation of FL is presented by Pei et al. [16]. Consider \(K\) clients \(\{C_1, \dots, C_K\}\), where each client \(C_k\) owns a private dataset \(D_k\) with \(n_k\) samples. Let \(n = \sum_{k=1}^K n_k\) be the total number of samples across all clients. The global learning objective is defined as: \[\min_{\omega \in \mathbb{R}^d} F(\omega) = \sum_{k=1}^{K} \frac{n_k}{n} F_k(\omega),\] where the local empirical loss at client \(k\) is given by \[F_k(\omega) = \frac{1}{n_k} \sum_{j=1}^{n_k} f_{k,j}(\omega; x_{k,j}, y_{k,j}),\] with \(f_{k,j}\) denoting the loss incurred on local sample \((x_{k,j}, y_{k,j})\). This formulation mirrors centralized empirical risk minimization, with the critical distinction that raw data remain distributed and never centrally stored or accessed.

From a performance standpoint, Blanco-Justicia et al. [17] emphasize that a practical FL algorithm should achieve predictive accuracy close to that of a centralized model, i.e., a model trained under a traditional learning paradigm in which all data are centrally collected and jointly optimized: \[\left| \text{Accuracy}_{\text{fed}} - \text{Accuracy}_{\text{centralized}} \right| < \delta,\] where \(\delta\) is a small tolerable margin. This criterion reflects the expectation that FL should preserve both privacy and competitive model performance.

Depending on the deployment context, FL systems are commonly categorized into cross-device, cross-silo, and hierarchical FL (HFL) settings. Cross-device FL typically involves a large number of resource-constrained and intermittently available clients (e.g., smartphones or IoT devices), whereas cross-silo FL involves a smaller number of stable, resource-rich clients such as organizations or institutions. Hierarchical FL extends these models by introducing multi-level aggregation structures, enabling scalable coordination across edge, regional, and central servers.

A defining challenge in FL is statistical heterogeneity across clients, commonly referred to as non–Independent and Identically Distributed (non-IID) data. This heterogeneity can arise from differences in label distributions, feature distributions, data quantity imbalance, or concept drift over time. non-IID data often leads to client drift, unstable convergence, and inconsistent global model behavior, making FL optimization substantially more challenging than centralized learning.

The core operational workflow of FL is based on the Federated Averaging (FedAvg) algorithm [13]. Training proceeds in iterative communication rounds between a central server and a subset of participating clients. At round \(t\), the server broadcasts the current global model \(\omega^{(t)}\) to a subset of selected clients. Each client performs several steps of stochastic gradient descent (SGD) on its local data to compute a local update \(g_k^{(t)}\), which is then sent back to the server. The server aggregates these updates using weighted averaging: \[\omega^{(t+1)} \leftarrow \omega^{(t)} - \eta \sum_{k=1}^{K} \frac{n_k}{n} g_k^{(t)},\] where \(\eta\) denotes the global learning rate [18]. While this mechanism avoids raw data sharing, it provides only partial privacy guarantees and remains vulnerable to inference attacks on model updates [19], [20].

To strengthen privacy protection, FL systems often incorporate additional mechanisms such as secure aggregation, differential privacy, homomorphic encryption, and secure multi-party computation. These techniques aim to prevent leakage of sensitive information from model updates while maintaining collaborative learning efficiency.

Overall, these principles and mechanisms have given rise to a diverse ecosystem of FL algorithms, each designed to address specific challenges such as statistical heterogeneity, communication efficiency, noisy updates, and system-level constraints. Table 1 provides a consolidated overview of representative FL methods, summarizing their categories, core ideas, addressed challenges, and inherent limitations.

2.2 Explainable Artificial Intelligence↩︎

XAI encompasses a broad class of methods and frameworks aimed at mitigating the opacity of modern machine learning models and enabling systematic interpretation of their decision-making processes. While concerns about explainability have accompanied AI systems for decades, the widespread adoption of highly non-linear, high-capacity models–particularly deep neural networks–has significantly intensified the need for formal explanatory mechanisms. A notable milestone in this direction was the launch of DARPA’s XAI program, which explicitly emphasized the development of AI systems capable of articulating the reasoning behind their outputs [31]. More recent surveys further highlight that increasing model complexity and deployment in high-stakes domains have elevated transparency, interpretability, and accountability from desirable properties to practical requirements [32].

Beyond technical motivations, XAI is strongly shaped by ethical and regulatory considerations. Regulatory frameworks, such as the European Union’s Artificial Intelligence Act (AI Act) [33], introduce explicit requirements for algorithmic accountability, transparency, and meaningful explanations of automated decisions, reinforcing the necessity of XAI systems [34], [35]. Consequently, the overarching goal of XAI is not merely to visualize model behavior, but to transform complex decision mechanisms into representations that can be understood, scrutinized, and acted upon by human stakeholders. This subsection introduces the core conceptual distinctions in XAI that serve as the foundation for the subsequent FedXAI taxonomy and evaluation framework.

A fundamental distinction in XAI lies between interpretable-by-design models and post-hoc explanation methods [32]. Interpretable-by-design approaches rely on inherently transparent model structures–such as linear models, decision trees, and fuzzy rule-based systems–where the decision logic is directly accessible. These models are particularly attractive in domains requiring immediate and unambiguous interpretability, including healthcare and fraud detection, but often face limitations in expressive power and scalability.

In contrast, post-hoc XAI methods aim to explain the predictions of already-trained black-box models without modifying their internal structure. This category includes attribution-based, surrogate-based, and visualization-based techniques that operate after model training. While post-hoc methods provide flexibility and model-agnostic applicability, they may only approximate the true internal logic of the model, potentially introducing abstraction or oversimplification in explanations [36].

Another important classification distinguishes model-specific from model-agnostic explanation techniques. Model-specific methods exploit internal architectural properties of particular model families to generate explanations. For instance, Grad-CAM [37] leverages gradient information in convolutional neural networks to produce spatial heatmaps highlighting influential input regions, and has been widely applied in domains such as medical imaging and autonomous perception [38]. While such methods often yield high-fidelity explanations, their applicability is restricted to specific model architectures [39].

Model-agnostic methods, in contrast, treat the underlying model as a black box and rely solely on input-output behavior. Prominent examples include LIME [40], which approximates local decision boundaries using interpretable surrogate models, and SHAP [41], which employs cooperative game theory to assign Shapley values quantifying feature contributions. Extensions such as Anchors [42] generate high-precision IF–THEN rules that locally anchor predictions. Although model-agnostic methods offer broad applicability, they often incur higher computational cost and may exhibit sensitivity to sampling strategies in high-dimensional settings.

XAI methods can further be categorized based on the scope of explanation. Model-level explanations aim to characterize the overall behavior of a model across the entire dataset, providing insights into learned feature relationships, biases, and decision trends. Techniques such as model-level feature importance and Partial Dependence Plots are commonly used for this purpose [43]. These methods are particularly useful for auditing and validation but are limited in explaining individual predictions.

In contrast, instance-level explanations focus on individual predictions and describe how specific input features influence a particular output. Methods such as saliency maps [44] and counterfactual explanations [45] enable instance-level reasoning and are especially valuable in personalized decision support and human-in-the-loop systems. In practice, model-level and instance-level explanations are often complementary rather than mutually exclusive.

In the literature, instance-level and model-level explanations are also referred to as local and global explanations, respectively. To avoid ambiguity with FL terminology, we use the terms instance-level and model-level explanations throughout the remainder of the paper.

Beyond categorization, recent XAI research emphasizes the importance of evaluating explanation quality. Fidelity measures how accurately an explanation reflects the true behavior of the underlying model. Stability assesses the sensitivity of explanations to small perturbations in input data or model parameters, while robustness concerns the resilience of explanations to noise, distribution shifts, or adversarial manipulation. These properties are increasingly recognized as critical, particularly when explanations are used for decision-making, auditing, or regulatory compliance [32].

While the above distinctions are well-established in centralized learning, their implications become substantially more complex in federated environments. Decentralized data access, statistical heterogeneity, and privacy constraints challenge both the generation and evaluation of explanations. These challenges motivate the need for FedXAI, where explainability mechanisms must be co-designed with FL architectures rather than treated as independent post-hoc tools. In particular, the diverse landscape of explanation paradigms used in the FedXAI papers reviewed in this survey and summarized in Table 2 highlights that different XAI methods impose fundamentally different requirements in terms of data accessibility, model transparency, computational cost, and privacy sensitivity. Such heterogeneity directly affects how explanations can be generated, aggregated, and validated in federated settings. The following subsection formalizes this integration and delineates the scope of FedXAI.

2.3 Federated Explainable Artificial Intelligence↩︎

The increasing interaction between FL and XAI has given rise to a distinct research paradigm commonly referred to as FedXAI. While early studies often applied explainability techniques to federated models in an ad-hoc or post-hoc manner, more recent research frames FedXAI as a principled design space in which federated optimization and explainability mechanisms are jointly considered [1]. This shift reflects growing recognition that explainability is not merely an auxiliary analysis tool, but a core component of trustworthy, privacy-preserving collaborative learning.

A critical distinction must be drawn between XAI-under-FL and FedXAI. XAI-under-FL refers to the straightforward application of existing explainability techniques to models trained via FL. In this setting, explainability is treated as an external, post-hoc analysis step that does not influence the federated optimization process itself. Typical examples include generating SHAP or LIME explanations on a trained federated model at either the client or server side, without modifying aggregation rules, communication protocols, or training dynamics. Several applied studies follow this paradigm, using attribution methods primarily to interpret the outcomes of federated models, for instance in intrusion detection or environmental prediction tasks [3], [63].

In contrast, FedXAI treats explainability as a first-class design objective that is explicitly integrated into the FL pipeline. Following foundational perspectives in the literature, FedXAI encompasses methods where explainability is co-designed with federated optimization, aggregation, personalization, or coordination mechanisms [1]. In this paradigm, explanations are not merely outputs for human interpretation, but active signals that may guide client selection, aggregation weighting, clustering, personalization, robustness analysis, or system-level decision-making. Consequently, FedXAI extends beyond explaining federated models to enabling explainability-aware and explanation-guided FL.

Recent approaches exemplify this trend by incorporating explanation signals directly into aggregation or clustering strategies under non-IID data, thereby improving both transparency and learning performance [7].

Formally, FedXAI can be characterized as a class of distributed learning systems that jointly satisfy the following properties: (i) model training adheres to FL principles, ensuring data locality and privacy preservation; (ii) explanation generation operates under the same decentralization and privacy constraints as model training; and (iii) explanations are systematically incorporated into the learning, aggregation, or evaluation process rather than being confined to post-hoc analysis.

This distinction is central to the taxonomy developed in this survey, as it differentiates methods that merely explain federated models from those that use explainability to shape FL behavior.

While FedXAI enhances transparency and trust, it also introduces new security and privacy risks that are absent or less pronounced in conventional FL. In particular, explanation artifacts themselves may constitute sensitive information and therefore expand the attack surface of federated systems.

Explanation leakage refers to the risk that explanations–such as feature attributions, saliency maps, or rule-based summaries–may inadvertently reveal sensitive information about local training data, data distributions, or client-specific characteristics [1], [4]. Even when raw data are never shared, explanation statistics can enable inference attacks that approximate feature distributions, identify dominant attributes, or expose correlations unique to individual clients. This risk has been explicitly acknowledged in FedXAI studies employing attribution-based explanations, where explanation aggregation must be carefully controlled to avoid privacy violations [4].

Explanation poisoning constitutes a complementary threat, wherein malicious or compromised clients deliberately manipulate explanation outputs to influence global interpretations or downstream decision-making [55], [58]. In explainability-aware aggregation or clustering schemes, poisoned explanations may distort aggregation weights, bias feature relevance assessments, or mislead human operators. Unlike traditional model poisoning attacks, explanation poisoning targets the interpretability layer itself, potentially undermining trust even when predictive performance appears unaffected.

Additional threat vectors arise from inconsistencies between instance-level and model-level under non-IID data, which adversaries may exploit to amplify confusion or conceal malicious behavior. These risks highlight that explanation integrity, confidentiality, and robustness must be explicitly considered in FedXAI system design.

The above distinctions and threat considerations underline that FedXAI cannot be realized by naively porting centralized XAI techniques into federated settings. Instead, explanation generation, aggregation, and evaluation must be aligned with federated constraints, adversarial assumptions, and communication budgets. Secure aggregation of explanation signals, robustness-aware explanation metrics, and privacy-preserving explanation mechanisms therefore emerge as essential components of FedXAI architectures.

These considerations motivate the multi-axis taxonomy introduced in the next section, which systematically categorizes FedXAI methods based on how explainability is integrated into the FL lifecycle, the type and scope of explanations produced, the role of explanation signals in training or aggregation, and the federated setting under which they operate.

3 Survey Methodology↩︎

This survey is conducted as a systematic literature review following a PRISMA-inspired methodology and established systematic review guidelines [77], [78] to ensure transparency, reproducibility, and methodological rigor. The overall workflow consists of three main stages: (i) keyword definition, (ii) source selection, and (iii) multi-stage screening and eligibility assessment. The study selection process is illustrated in Fig. 2.

3.1 Keyword Definition↩︎

A structured search strategy was developed to comprehensively cover the intersection of FL and XAI. The queries combined “federated learning” with major explainability-related concepts, including “federated explainable AI”, “explainable federated learning”, “interpretable federated learning”, “federated XAI”, and “interpretable machine learning”.

To address terminological variations, additional terms such as “trustworthy federated learning”, “privacy-aware explainability”, and “federated model interpretation” were incorporated. All keywords were combined using Boolean operators (AND, OR) and applied to titles, abstracts, and keywords across databases.

3.2 Source Selection↩︎

The literature search was conducted across Google Scholar and Scopus to ensure broad coverage and maximize recall. Multiple keyword combinations were applied, resulting in 2335 records prior to screening.

After removing duplicate, non-English, conference-abstract-only, and irrelevant records, 715 studies remained for the screening phase. All searches were performed up to March 2026. Most relevant papers were published after 2017, indicating that FedXAI is an emerging research area. The complete study selection process is illustrated in Fig. 2.

3.3 Multi-stage screening and eligibility assessment↩︎

A multi-stage screening process was conducted, including title/abstract filtering and full-text assessment. During the screening stage, 480 non-peer-reviewed studies were excluded, resulting in 235 reports sought for retrieval. Among them, 50 reports could not be retrieved due to inaccessible full texts.

Subsequently, 185 reports were assessed for eligibility. Studies were included if they explicitly integrated FL with explainability mechanisms, proposed explainability-aware federated frameworks, or investigated interpretable and transparent learning strategies within federated environments.

Studies focusing solely on FL or XAI, as well as tutorials, editorials, duplicate records, and non-peer-reviewed works, were excluded. After full-text assessment, 100 studies were excluded because they were not directly relevant to the research objectives. Following the screening and eligibility phases, 85 studies were selected for the final analysis. Backward snowballing was also performed to ensure completeness of the review.

A qualitative assessment was conducted to verify the methodological soundness, experimental validity, and reproducibility of the selected studies.

4 A Multi-axis Taxonomy for FedXAI↩︎

To systematically organize and analyze the rapidly expanding literature at the intersection of FL and XAI, we propose a multi-axis taxonomy for FedXAI. Unlike conventional classifications that focus solely on explanation methods, the proposed taxonomy captures multiple complementary dimensions of explainability in federated systems, including how explanations are generated, why they are employed, where they are integrated within the federated pipeline, and under which federated conditions they operate.

Concretely, each FedXAI approach is characterized along five orthogonal axes: (A) the role of explainability within the FL lifecycle, (B) the family of XAI techniques employed, (C) the scope and target of explanations, (D) the integration level and the nature of shared artifacts, and (E) the FL setting and heterogeneity assumptions.

Together, these axes define a structured design space for analyzing, comparing, and designing FedXAI systems across diverse explainability objectives and federated settings. The practical use of the taxonomy for system design is further discussed in Section 4.6.

An overview of the proposed taxonomy and its categorical dimensions is summarized in Figure 3.

4.1 Axis A: Role of Explainability in the FL Lifecycle↩︎

Axis A captures why explainability is introduced in a federated system. Unlike centralized XAI, where explanations are typically post-hoc and user-facing, FedXAI exhibits a spectrum of roles ranging from transparency to system-level coordination and security. From a design perspective, this axis helps practitioners identify the primary purpose of explainability in their system: post-hoc transparency, training diagnosis, aggregation and coordination, personalization, or security and trust management.

In transparency, explainability is applied after federated training to interpret model predictions without influencing optimization or aggregation. This is the most common and least invasive form of FedXAI.

For instance, Oki et al. [3] apply SHAP to analyze how federated training recovers intrusion detection performance under non-IID data, comparing feature relevance between federated and centralized settings. Similarly, Kumar et al. [5] employ SHAP to provide clinically interpretable explanations for federated liver disease prediction while preserving institutional data privacy. Here, explainability functions as an observability mechanism to analyze training dynamics, non-IID effects, and model evolution across rounds or episodes.

Carillo et al. [76] integrate attribution stability analysis and latent-space visualization into federated class-incremental learning, enabling diagnosis of catastrophic forgetting and bias correction behavior. At the system level, Patni and Lee [54] use XAI to explain resource utilization and communication bottlenecks in hierarchical FL. In aggregation-centric FedXAI, explanation signals actively influence coordination mechanisms such as client weighting or clustering.

Ali et al. [7] propose SHAP-driven clustered FL for solar forecasting, where feature attributions determine cluster formation and aggregation. In a different direction, SemFedXAI [67] introduces ontology-guided semantic aggregation, using knowledge-based explanations to align heterogeneous healthcare clients. Personalization-oriented FedXAI leverages explainability to adapt models or explanations to client-specific contexts.

Figure 4 illustrates the architecture of the SemFedXAI framework [67], which represents a canonical example of explanation-driven aggregation in FL. Unlike conventional FedAvg-based pipelines, SemFedXAI integrates domain ontologies and knowledge graphs into both the explanation and aggregation processes. Clients enrich feature representations using semantic concepts, generate ontology-aware explanations, and transmit semantically weighted updates to the server. The central aggregator then performs semantic aggregation, aligning heterogeneous client updates based on shared medical concepts rather than purely numerical similarity. This example illustrates how axis A can be operationalized: when the design goal is coordination across heterogeneous clients, explainability is not only used for interpretation but also becomes part of the aggregation logic.

Taheri et al. [55] integrate SHAP and LIME into federated intrusion detection for connected vehicles, supporting adaptive local models under highly dynamic data distributions. FedMedSecure [66] further exploits attention weights and prototype explanations to enable few-shot personalization in federated healthcare cybersecurity. In this role, explainability supports attack detection, trust assessment, and explanation integrity.

Yazdinejad et al. [64] combine SHAP with differential privacy and homomorphic encryption to explain threat detection decisions while mitigating information leakage. Kalakoti et al. [4] address explanation confidentiality by securely aggregating client-side SHAP values using secure multiparty computation. Thus, axis A provides an initial design guideline: practitioners should select the role of explainability according to the intended system objective, before choosing a specific XAI technique or integration strategy.

4.2 Axis B: XAI Family↩︎

Axis B categorizes FedXAI approaches based on the family of explainability techniques employed, independently of the FL topology or application domain. This axis captures how explanations are generated and reflects a fundamental design choice that directly impacts explanation fidelity, computational cost, communication overhead, and compatibility with federated constraints. From a system design perspective, this axis helps practitioners select explanation mechanisms according to model complexity, resource availability, privacy requirements, and interpretability needs.

In the surveyed literature, FedXAI methods predominantly fall into four explainability families: (i) model-agnostic post-hoc explainers, (ii) visualization-based techniques, (iii) attention- or architecture-based mechanisms, and (iv) interpretable-by-design models. Each family exhibits distinct strengths and limitations when deployed in decentralized and privacy-preserving environments. Model-agnostic explainability techniques constitute the most widely adopted XAI family in FedXAI. These methods treat the trained federated model as a black box and derive explanations by probing input–output relationships, making them readily applicable to heterogeneous models and clients.

In IoT-centric FedXAI systems, explainability is typically applied in a post-hoc manner and does not interfere with federated optimization. A representative example is the FL-XAI framework for malicious traffic detection in IoT networks proposed by Bilal et al. [57], where a global model is trained using standard FedAvg aggregation, and interpretability is achieved through instance-level SHAP attributions computed on the trained federated model.

Such post-hoc explainers are particularly suitable for resource-constrained or heterogeneous FL settings because they can be integrated without modifying the underlying federated optimization pipeline.

SHAP and LIME dominate this category, particularly in tabular and time-series domains such as intrusion detection, fraud detection, and healthcare analytics. For example, Oki et al. [3] employ Kernel SHAP to analyze how FL alters feature importance distributions in distributed intrusion detection systems, demonstrating convergence toward centralized feature relevance under FL. Similarly, Awosika et al. [6] integrate SHAP-based explanations into federated financial fraud detection to provide both model-level and transaction-level transparency while preserving inter-bank data privacy.

The popularity of model-agnostic explainers stems from their flexibility and minimal assumptions about model structure. However, in federated settings they often introduce substantial computational overhead and may raise privacy concerns when explanation statistics are shared or aggregated across clients. Visualization-based XAI techniques are predominantly used in FedXAI applications involving high-dimensional spatial or temporal data, most notably medical imaging and signal analysis. These methods generate saliency maps or activation heatmaps that highlight regions of the input contributing most strongly to a prediction.

Grad-CAM and related saliency techniques are extensively employed in federated medical imaging. Mastoi et al. [70] apply Grad-CAM and saliency maps to explain federated brain tumor classification from MRI images, enabling clinicians to verify that models focus on clinically relevant tumor regions despite decentralized training. Grad-CAM saliency maps highlight clinically relevant tumor regions in MRI images. In hierarchical FL for leukemia diagnosis, Pervez et al. [75] combine saliency maps, occlusion sensitivity, and RISE to evaluate explanation fidelity across different federation levels.

Visualization-based explainers offer intuitive and domain-aligned interpretations, particularly for expert users. Nevertheless, their explanations are often qualitative, model-specific, and difficult to aggregate or compare across clients, which limits their direct use in explainability-driven coordination or optimization. Accordingly, this family is particularly appropriate for high-stakes domains such as healthcare, where visual interpretability and clinician trust are prioritized over lightweight deployment.

Attention-based and architecture-integrated explainability methods derive interpretability directly from model components such as attention weights, feature-routing mechanisms, or prototype similarities. In contrast to post-hoc methods, these approaches embed explainability into the learning architecture itself.

In smart grid load forecasting, Sarker et al. [53] integrate attention mechanisms within a federated deep learning model, using attention weights to highlight influential temporal and contextual features driving energy demand predictions. Similarly, FedMedSecure [66] leverages cross-attention mechanisms and prototype similarity in a federated few-shot learning framework, providing interpretable insights into feature relevance and decision confidence under extreme data scarcity.

Architecture-based explainability typically offers lower runtime overhead at inference time and tighter coupling between learning and explanation. However, interpretability is often indirect and requires careful validation to ensure that attention weights or internal activations faithfully reflect causal importance, particularly in non-IID federated environments. These methods are therefore well suited for personalization-oriented or adaptive FedXAI systems, where explanation signals are tightly coupled with model behavior.

Interpretable-by-design approaches pursue transparency through inherently explainable model structures, such as decision trees, rule-based systems, and fuzzy models. In FedXAI, this family represents the strongest form of integration between learning and explainability, as explanations are intrinsic to the trained model rather than generated post hoc.

Corcuera Bárcena et al. [46] propose an FL framework for Takagi–Sugeno–Kang fuzzy regression models, where interpretable fuzzy rules are learned locally and aggregated centrally without exchanging raw data. Extending this line of work, federated fuzzy regression trees [49] enable privacy-preserving induction of transparent tree structures through the exchange of sufficient statistics, achieving performance comparable to centralized learning while retaining full interpretability.

Interpretable-by-design FedXAI methods provide high transparency, low explanation ambiguity, and reduced risk of explanation leakage. However, they often require customized aggregation schemes and may sacrifice predictive performance or scalability when compared to deep neural models. Consequently, they are most appropriate in applications requiring strong transparency guarantees and low explanation ambiguity, even at the cost of reduced model flexibility.

4.3 Axis C: Scope and Target of Explanations↩︎

Axis C characterizes FedXAI methods based on the granularity (scope) at which explanations are generated and the explanation target. This distinction is particularly important in federated settings, where explanations may serve different stakeholders (clients, servers, system operators) and may operate at different structural levels of the learning process. From an operational perspective, this axis helps determine whether explanations should support individual decision interpretation, global model auditing, client-group coordination, or system-level monitoring.

The scope defines the level at which explanations are generated and interpreted, ranging from individual predictions to aggregated or cluster-level behavior. Instance-level explanations focus on individual predictions or instances at a specific client. For example, Raza et al. [69] employ Grad-CAM adapted to 1D ECG signals to provide patient-specific explanations for federated arrhythmia classification. These explanations remain confined to each healthcare institution and are intended for clinician-level interpretation.

Model-level explanations summarize the overall behavior of the federated model. Oki et al. [3] generate global SHAP summaries to compare feature importance between federated and non-distributed intrusion detection systems, illustrating how federated aggregation reshapes model-level decision logic under non-IID data.

Cluster-level explanations occupy an intermediate granularity, explaining the behavior of groups of similar clients. Ali et al. [7] use SHAP values to form client clusters in solar energy forecasting, where explanations characterize cluster-specific feature relevance and directly guide cluster-wise aggregation.

Accordingly, practitioners may select instance-level explanations for personalized decision support, model-level explanations for auditing and transparency, or cluster-level explanations for coordination and adaptive aggregation.

The target specifies which component of the federated system is being explained. Many FedXAI works target model predictions, particularly in high-stakes applications such as fraud detection and intrusion detection. For instance, Awosika et al. [6] use SHAP to explain transaction-level fraud predictions produced by a federated deep learning model, supporting regulatory transparency.

Other approaches focus on model parameters or symbolic structures. Corcuera Bárcena et al. [49] explain federated fuzzy regression trees by exposing learned rules, splits, and linguistic terms, enabling direct inspection of the global model structure rather than post-hoc attribution.

Explainability may also target training dynamics. Carillo et al. [76] analyze attribution stability and latent representations across federated incremental learning episodes, using explanations to understand catastrophic forgetting and bias evolution over time.

Finally, some works target system-level resources and coordination decisions. Patni and Lee [54] apply SHAP and LIME to explain resource allocation, communication cost, and scheduling decisions in hierarchical FL, shifting the focus of explainability from prediction outcomes to system behavior. This paradigm is explicitly illustrated in their framework design, where an explainable AI-powered resource management unit is embedded into the core control workflow and directly influences scheduling, aggregation, and communication efficiency.

In conclusion, axis C provides guidance on selecting both the granularity and the target of explanations according to the intended stakeholder and operational objective of the federated system.

4.4 Axis D: Integration Level and Shared Artifacts↩︎

Axis D captures where explainability is integrated within the federated architecture and which artifacts are exchanged between participants. This axis is critical for understanding privacy implications, communication overhead, and trust boundaries in FedXAI systems. From a practical perspective, axis D helps determine how explanations should be distributed across clients and servers, and which explanatory artifacts can be safely exchanged under privacy and communication constraints.

In client-centric FedXAI, explanations are generated locally and are not shared with the server. This design maximizes privacy and aligns with human-in-the-loop use cases.

For example, Raza et al. [69] generate Grad-CAM explanations locally at healthcare institutions for ECG classification, ensuring that both data and explanations remain confined to the client. The server only aggregates model parameters, without access to explanations or raw signals.

Client-side integration is therefore particularly suitable for privacy-sensitive applications where explanations are intended primarily for local users or domain experts. In server-centric approaches, explanations are generated or aggregated at the central coordinator, providing a model-level view of model behavior.

Awosika et al. [6] compute model-level SHAP summaries at the server to explain federated fraud detection models trained across multiple banks. This setup supports regulatory auditing but requires careful consideration of information leakage from aggregated explanations. Such server-side strategies are more appropriate when global auditing, monitoring, or regulatory transparency is prioritized over strict local confidentiality.

Hybrid FedXAI combines client-side explanation generation with secure aggregation or selective sharing mechanisms, balancing privacy and model-level interpretability. Kalakoti et al. [4] propose a hybrid architecture in which clients compute SHAP values locally and securely aggregate them using secure multiparty computation. This enables server-side explanations that closely approximate centralized SHAP results without exposing client-level explanations or data. Hybrid approaches provide a practical compromise between local privacy preservation and model-level interpretable system behavior.

Across these integration levels, the nature of shared artifacts varies substantially. Standard FL exchanges gradients or model weights, whereas FedXAI may additionally share SHAP statistics, explanation summaries, fuzzy rules, attention weights, or semantic metadata. For instance, fuzzy rule-based FedXAI [49] exchanges symbolic rule parameters, while SemFedXAI [67] incorporates ontology-derived semantic information into aggregation.

In conclusion, axis D provides guidance on selecting both the integration locality and the type of exchanged explanatory artifacts according to system-level privacy, communication, and trust requirements.

4.5 Axis E: FL Setting and Heterogeneity↩︎

Axis E situates FedXAI methods within specific FL settings and heterogeneity assumptions. The feasibility, reliability, and interpretation of explanations are strongly influenced by client scale, availability, and data distribution. This axis helps practitioners align explainability strategies with the operational characteristics and constraints of the underlying federated environment.

In cross-device settings, a large number of resource-constrained and intermittently available clients participate in training.

Bilal et al. [57] study explainable federated intrusion detection in IoT networks, where lightweight models and SHAP/LIME explanations must operate under severe communication and computational constraints. In such environments, explanation latency and overhead are critical design considerations. Accordingly, cross-device FedXAI systems often favor lightweight and low-overhead explainability mechanisms.

Cross-silo FL involves a smaller number of stable, institution-level clients, making it well-suited for explainability-intensive applications. Medical imaging [70] and healthcare prediction [5] exemplify this setting, where Grad-CAM and SHAP explanations are used to support clinical trust and regulatory compliance. The relative stability of clients enables richer explanations and more complex models.

This setting is therefore more compatible with computationally intensive or high-fidelity explainability techniques.

Hierarchical FL introduces intermediate aggregation layers, such as edge servers or regional coordinators, complicating both learning and explainability. Patni and Lee [54] integrate XAI into hierarchical FL to explain resource management and communication efficiency, while Pervez et al. [75] apply multi-level explainability in hierarchical medical federations. Here, explanations may exist at client, edge, and global levels simultaneously. Hierarchical settings may additionally require explainability mechanisms operating across multiple federation levels simultaneously.

Across all settings, non-IID data distributions, partial client participation, and streaming or continual data significantly affect explanation stability and reliability. Incremental and streaming scenarios, such as federated class-incremental learning [76], further challenge the consistency of explanations over time.

In conclusion, axis E provides practical guidance for selecting explainability mechanisms that remain reliable under different federated deployment conditions and heterogeneity assumptions.

4.6 Operationalizing the Taxonomy for FedXAI Design↩︎

The proposed taxonomy can be used to systematically identify suitable explainability strategies based on application requirements, federated settings, privacy constraints, and system-level objectives.

Rather than viewing explainability as a standalone post-hoc component, the taxonomy conceptualizes FedXAI as a multidimensional design space spanning explainability objectives, XAI mechanisms, integration strategies, and federated deployment conditions.

By jointly considering explainability objectives, XAI mechanisms, integration strategies, and federated settings, the taxonomy enables systematic comparison of existing approaches and supports informed design choices for new FedXAI systems.

In practice, the taxonomy may be operationalized sequentially by: (i) identifying the primary role of explainability (axis A), (ii) selecting an appropriate XAI family (axis B), (iii) determining the explanation scope and target (axis C), (iv) choosing the integration level and exchanged artifacts (axis D), and (v) adapting the design to the target FL setting and heterogeneity assumptions (axis E).

Table 3 illustrates representative mappings of existing FedXAI methods onto the proposed taxonomy axes.

The table highlights how representative FedXAI methods differ in terms of explainability objectives, integration strategies, and federated deployment settings.

Although the taxonomy axes are conceptually orthogonal, practical FedXAI systems often exhibit recurring cross-axis correlations driven by domain constraints, privacy requirements, and communication limitations. For example, cross-device systems often rely on lightweight post-hoc explainers, whereas cross-silo healthcare applications typically employ richer visualization-based or hybrid explainability mechanisms.

The taxonomy can further guide the selection of explainability mechanisms according to application and system requirements. Applications with strict communication or privacy constraints may favor lightweight instance-level explainers, whereas high-stakes domains such as healthcare may require richer or privacy-preserving explanation mechanisms. Similarly, personalization-oriented systems may benefit from attention-based or architecture-integrated explainability, whereas coordination-driven FL systems may incorporate explanation signals directly into clustering, aggregation, or client selection procedures.

A healthcare-oriented cross-silo FL system requiring clinically interpretable predictions and strong privacy guarantees may favor visualization-based or hybrid explainability approaches with client-side integration, such as Grad-CAM or secure SHAP aggregation. In contrast, resource-constrained cross-device IoT environments may prefer lightweight post-hoc explainers with minimal communication overhead. Similarly, systems emphasizing adaptive aggregation or personalization may select attention-based or explanation-driven coordination mechanisms.

Overall, the proposed taxonomy provides not only a structured view of the current FedXAI landscape, but also a practical foundation for designing future FedXAI systems under diverse architectural, privacy, and application constraints.

5 FedXAI Methodological Approaches↩︎

Building upon the multi-axis taxonomy introduced in Section 4, this section reviews and categorizes existing FedXAI methods according to how they instantiate different roles of explainability within the FL lifecycle. Rather than organizing works by specific XAI techniques, we focus on methodological patterns that reveal how explainability is integrated, exploited, and operationalized in federated environments.

5.1 Post-hoc FedXAI: Explainability after Federated Training↩︎

Post-hoc explainability represents the most prevalent methodological paradigm in the current FedXAI literature. In this setting, FL is performed using conventional optimization and aggregation schemes–most commonly variants of Federated Averaging–while explainability mechanisms are applied after the training process, without directly influencing model optimization or aggregation dynamics. This approach enables the deployment of highly expressive black-box models, particularly deep neural networks, while augmenting them with interpretability mechanisms as an external layer.

A representative example of post-hoc FedXAI is provided by Kalakoti et al. [4], who propose a FedXAI-based IDS framework in a horizontal FL (HFL) setting for IoT botnet detection. Their framework uses SHAP to explain the server model by securely aggregating SHAP values from client-side models, without sharing client data with the server.

Let \(\mathcal{M}_{c_i}\) denote the trained model of client \(c_i\), and let \(\boldsymbol{\phi}_{c_i}(x) \in \mathbb{R}^d\) denote the SHAP values generated for a point of interest \(x\). Leveraging the additive nature of SHAP values, the explanation of the server model \(\mathcal{M}_s\) is obtained by aggregating explanations from the individual client models:

\[\boldsymbol{\phi}_s(x) = \frac{1}{|N|} \sum_{c_i \in N} \boldsymbol{\phi}_{c_i}(x_j),\] where \(N\) denotes the set of participating clients.

To preserve the privacy of individual clients’ SHAP values, the authors employ a secure multi-party computation (SMPC) protocol based on secret sharing. This allows the server to compute aggregated SHAP values without knowing the individual SHAP values of each client. The authors evaluate this approximation by comparing the securely aggregated client-based explanations with server-based explanations, generated when the server has direct access to the data from all participating clients. Their results show that securely aggregated client-side explanations can approximate the feature attributions of the server model without relying on client data.

A dominant instantiation of post-hoc FedXAI involves client-side explanation generation, where each participating client independently computes explanations for its local predictions using model-agnostic XAI techniques. This design aligns naturally with privacy-preserving requirements, as both raw data and explanation computations remain confined to the client side. Numerous studies in intrusion detection and cybersecurity adopt this paradigm. For instance, Kalakoti et al. [4] integrate SHAP, LIME, and Integrated Gradients into a federated LSTM-based IDS, enabling each IoT client to interpret detection outcomes on the client side. Similarly, Taheri et al. [55] and Harshitha et al. [56] employ post-hoc SHAP- and LIME-based explanations at the client level to justify intrusion alerts in connected vehicle and network security scenarios. These works demonstrate that client-side post-hoc explanations are relatively easy to deploy and effective for operational transparency, debugging, and local trust establishment.

Beyond cybersecurity, post-hoc client-level explainability has been extensively explored in healthcare and medical imaging. Raza et al. [69] adapt Grad-CAM for one-dimensional ECG signals within a federated transfer learning framework, allowing clinicians to interpret arrhythmia classification decisions within each participating institution. Similar visualization-based approaches are adopted in federated medical imaging systems for brain tumor classification [70], skin cancer diagnosis [72], [74], and viral disease detection [59], [60], where Grad-CAM and saliency maps highlight clinically relevant image regions without exposing sensitive patient data. Collectively, these studies illustrate that post-hoc client-side explanations are particularly well suited for high-stakes domains requiring instance-level transparency and human validation.

While client-side explanations enhance institution-specific interpretability, they provide limited insight into the model-level behavior of the federated model. To address this limitation, several works extend post-hoc FedXAI toward server-side or model-level explanations by aggregating explanation artifacts rather than raw data or gradients. In this paradigm, clients compute client-side explanation vectors–typically SHAP values–which are then transmitted to the server and combined to produce model-level feature importance profiles. Bilal et al. [57] demonstrate this approach in an IoT intrusion detection setting, showing that aggregated SHAP and LIME explanations approximate centralized interpretations while preserving data locality. Similarly, Oki et al. [3] employ Kernel SHAP to analyze how FL recovers the feature relevance patterns of a non-distributed IDS, providing a model-level explanation of performance gains under FL.

In healthcare applications, Kumar et al. [5] integrate SHAP-based model-level explanations into a federated liver disease prediction framework, enabling clinicians to assess whether the global federated model’s reasoning aligns with established medical knowledge. Comparable model-level explanation strategies are reported in federated flood prediction [63] and smart agriculture [51], where SHAP-based summaries are used to interpret feature relevance across geographically distributed clients. These studies highlight that explanation aggregation serves as a practical compromise between privacy preservation and system-level interpretability.

Corbucci et al. [79] further formalize explanation aggregation in horizontal server-based FL by proposing a SHAP variant for explaining black-box federated models without requiring server access to clients’ training data. In their approach, each client builds a SHAP explainer using its private data and computes feature-attribution vectors for the instances to be explained; these client-side explanations are then averaged to approximate the explanation of the global model. Using the Adult and CoverType tabular datasets, the authors show that the aggregated client-based explanations closely match simulated server-based explanations obtained with full training-data access, suggesting that SHAP aggregation can offer a practical privacy–utility trade-off for post-hoc FedXAI. They also identify important directions for future work, including evaluation under non-IID data, larger client populations, peer-to-peer FL, and stronger privacy-preserving mechanisms.

A more principled instantiation of post-hoc FedXAI is introduced by Ducange et al. [80], who propose the concept of Federated SHAP (FedSHAP) to generate privacy-preserving and consistent SHAP explanations in FL. Instead of directly aggregating client-side SHAP values, their approach focuses on the federated construction of a representative background dataset, which is a critical component for computing reliable Shapley values. For tabular data, they employ a Federated Fuzzy C-Means (FedFCM) clustering algorithm, where clients collaboratively compute federated cluster centroids that summarize the distributed data without sharing raw samples [80]. These centroids form a compact and privacy-preserving background dataset for KernelSHAP. For image data, they introduce a Federated GAN (FedGAN) to synthesize realistic background images, enabling GradientSHAP explanations while preventing exposure of sensitive visual data [80].

This methodology satisfies three key desiderata of FedXAI: (i) privacy preservation, since only aggregated statistics or model updates are exchanged; (ii) explanation consistency, as all entities rely on a common federated background dataset; and (iii) accuracy, since the generated explanations closely approximate those obtained in a centralized setting. Compared to simple client-side SHAP aggregation schemes, FedSHAP provides a more theoretically grounded, modality-aware, and systematic framework for post-hoc explainability in federated environments.

However, transmitting explanation artifacts introduces new privacy and security considerations, as explanations may encode sensitive information about local data distributions. To mitigate this risk, several works incorporate secure aggregation mechanisms for explanations. Kalakoti et al. [4] employ Secure Multiparty Computation (SMPC) to aggregate client-side SHAP values, enabling the server to approximate model-level explanations without direct access to local explanation vectors. In a complementary direction, Yazdinejad et al. [64] combine SHAP with homomorphic encryption and differential privacy in a federated threat detection framework, explicitly addressing confidentiality risks associated with both model updates and explanation sharing.

Despite their practical appeal, post-hoc FedXAI approaches exhibit inherent limitations. Since explainability is decoupled from the federated optimization process, explanations remain passive diagnostic tools that cannot influence convergence behavior, aggregation robustness, or personalization. Moreover, explanation inconsistency across clients and across federated rounds–exacerbated by non-IID data distributions–poses challenges for trustworthiness and regulatory validation, as highlighted in comparative healthcare studies [48]. These limitations motivate more tightly integrated FedXAI methodologies, in which explainability actively shapes learning and coordination mechanisms, as discussed in the following subsections.

5.2 Explainability-aware FL: XAI inside the FL Loop↩︎

While post-hoc FedXAI frameworks treat explainability as an external interpretability layer, a growing body of work integrates XAI directly into the FL process, allowing explanation signals to actively influence learning dynamics, aggregation strategies, personalization, and system-level coordination. In these explainability-aware approaches, explainability is embedded inside the FL loop, shaping how models are trained, aggregated, and deployed rather than merely explaining their outputs.

A first and prominent methodological pattern in explainability-aware FedXAI is the use of explanation signals to guide aggregation. Instead of relying solely on data volume or local loss for weighting client updates, these methods exploit feature attribution scores or explanation-derived relevance measures to modulate aggregation behavior. Saleem et al. [62] propose a Weighted Explainable Federated Learning (WFL-XAI) framework for privacy-preserving and scalable energy optimization in autonomous vehicular networks. The framework integrates XAI techniques, including SHAP and LIME, to enhance transparency and interpretability in AI-driven energy optimization decisions. In addition, the framework introduces a weighted federated aggregation mechanism in which client contributions are dynamically adjusted according to local model performance and client data volume, enabling a more accurate and fair federated model construction in heterogeneous vehicular environments.

For the aggregation layer, clients send local model parameters and performance metrics to the central server. The client contribution weight is computed as

\[Q_k = \frac{R_k^2 \cdot n_k}{\sum_{i=1}^{K} R_i^2 \cdot n_i}, \label{eq:wfl95weight95Weighted2025Saleem}\tag{1}\] where \(Q_k\) is the aggregation weight for client \(k\), \(R_k^2\) denotes the coefficient of determination of client \(k\), \(n_k\) is the number of samples at client \(k\), and \(K\) is the total number of participating clients.

The global model is then aggregated as \[W^{*} = \sum_{k=1}^{K} Q_k \theta_k, \label{eq:wfl95agg95Weighted2025Saleem}\tag{2}\] where \(W^{*}\) denotes the global weight matrix and \(\theta_k\) represents the local model parameters of client \(k\). This weighted aggregation mechanism ensures that higher-quality local models exert greater influence on the global model.

A related explanation-guided aggregation strategy is employed in the smart grid and solar energy forecasting domain. Ali et al. [7] propose Explainable Clustered Federated Learning (XCFL), which integrates SHAP-based feature contribution analysis into CFL for solar power forecasting. The framework improves both model performance and interpretability by incorporating feature-level contributions into cluster and global aggregation processes. In contrast to conventional FedAvg, XCFL performs weighted aggregation using feature contribution scores extracted from XAI tools, allowing more informative features and clusters to contribute more significantly to the global model.

Beyond direct aggregation weighting, explainability is also utilized for client clustering under heterogeneous data distributions. In XCFL [7], photovoltaic clients are grouped using Mean-Shift clustering based on similar data distributions and feature characteristics. The clustered training process enables localized models to adapt to different climatic, geographic, and sensor conditions, while SHAP explanations provide interpretable insights into feature contributions at instance-level, cluster-level, and model-level. The authors show that explanation-driven clustering improves robustness and generalization under non-IID settings.

In XCFL [7], clients are first grouped into clusters and local models are trained independently inside each cluster. SHAP-based feature contribution scores are then used to perform hierarchical weighted aggregation at both cluster and global levels. The cluster-level aggregation is formulated as \[\omega_{m}^{(t+1)} = \sum_{k \in C_m} \sum_{f=1}^{F} \left( \frac{\delta_f^{(k)}}{\sum_{k' \in C_m}\delta_f^{(k')}} \right) \left( 1-\sum_{\substack{j=1 \\ j\neq f}}^{F} a_j \right) \omega_k^{(t)}, \label{eq:xcfl95cluster95agg}\tag{3}\] where \(m\) denotes the cluster index, \(C_m\) is the set of clients belonging to cluster \(m\), \(\delta_f^{(k)}\) denotes the SHAP-based contribution score of feature \(f\) for client \(k\), \(a_j\) is the weight assigned to feature \(j\), and \(\omega_k^{(t)}\) represents the local model parameters of client \(k\) at communication round \(t\).

The server subsequently performs global aggregation across clusters as \[\omega^{t+1} = \sum_{k \in C_m} \sum_{f=1}^{F} \frac{\delta_f^{(k)}}{\sum_{m \in C_m}\delta_{fm}} \left( 1-\sum_{\substack{j=1 \\ j\neq i}}^{F} a_j \right) \omega_m^{t}, \label{eq:xcfl95global95agg}\tag{4}\] where \(\omega_m^{(t)}\) denotes the cluster-level model parameters and \(\delta_{fm}\) represents the aggregated feature contribution score for feature \(f\) within cluster \(m\). This hierarchical SHAP-guided aggregation mechanism allows XCFL to prioritize informative features and clusters during model fusion, thereby improving forecasting accuracy, interpretability, and robustness in heterogeneous federated environments.

Several explainability-aware FedXAI frameworks integrate explanation analysis to support adaptive personalization and robustness against unreliable or malicious clients. In connected vehicular networks, Taheri et al. [55] combine SHAP- and LIME-based explanations with robust aggregation mechanisms to enhance both interpretability and resilience to adversarial behavior. Explanation patterns are analyzed alongside prediction outcomes to identify influential features and adjust aggregation behavior, enabling adaptive trust management across vehicles.

Similarly, FLEX-IDS [58] incorporates post-hoc explanations into an explainability-aware federated IDS framework that dynamically selects among multiple aggregation strategies (e.g., FedAvg, FedProx, FedNova, FedPer). Although explanations do not directly modify gradients, they inform robustness analysis and client influence control, illustrating a hybrid form of explainability-aware personalization where XAI supports adaptive decision-making within the federated loop.

Explainability-aware methodologies also extend beyond model-centric decisions to system-level coordination, particularly in Hierarchical FL (HFL). In such architectures, explainability is used to interpret and guide orchestration decisions rather than model predictions.

Patni and Lee [54] propose an explainable AI-empowered resource management framework for HFL, where SHAP and LIME explanations are applied to resource prediction models governing communication scheduling and node selection. Explainability enables transparent justification of system-level decisions across cloud-edge-device hierarchies, improving communication efficiency and convergence reliability. A complementary system-level integration is observed in the FLaaS-based architecture proposed by Corcuera Bárcena et al. [47], where explainable models are natively embedded into federated services to support governance, monitoring, and trust in beyond-5G/6G network environments.

A more advanced form of explainability-aware FedXAI incorporates domain knowledge into the learning and aggregation process itself. Amato and Branco [67] introduce SemFedXAI, a semantic framework that integrates ontologies and knowledge graphs into FL pipelines. In this approach, explainability directly influences aggregation through semantic weighting mechanisms, aligning feature relevance with clinically meaningful concepts. By elevating explanations from numerical attributions to concept-level reasoning, SemFedXAI mitigates non-IID effects and enhances interpretability in healthcare applications.

5.3 Interpretable-by-Design FedXAI: FL of Transparent Models↩︎

Interpretable-by-design FedXAI constitutes a distinct methodological paradigm from post-hoc and explainability-aware approaches. In this setting, explainability is not provided by auxiliary interpretation mechanisms, nor used as a control signal within the learning loop; instead, transparency is an intrinsic property of the learned model itself. The model representation–typically expressed as rules or tree-based fuzzy structures–is inherently human-understandable, enabling direct inspection of the decision logic without relying on external explainers.

A central challenge in interpretable-by-design FedXAI lies in reconciling symbolic or rule-based model representations with FL protocols originally designed for gradient-based optimization. As a result, these approaches often require bespoke training and aggregation mechanisms that differ substantially from standard FedAvg-style procedures.

Fuzzy Rule-Based Systems (FRBSs), particularly Takagi-Sugeno-Kang Fuzzy Rule-Based Systems (TSK-FRBSs), represent one of the most widely studied inherently interpretable models in federated settings. Corcuera Bárcena et al. [46] propose a federated approach for learning explainable TSK-FRBSs in regression tasks. In their framework, each client independently learns a local TSK-FRBS from private data and transmits the learned fuzzy rules to a central server for aggregation. Unlike FedAvg-based approaches that aggregate gradients or model tensors, the server aggregates rule-based models by resolving conflicts among rules with identical antecedents and different consequents, while preserving the interpretability of the global TSK-FRBS.

In [46], each fuzzy rule is assigned a rule weight \(RW_k\) based on its fuzzy confidence \(Conf_k\) and fuzzy support \(Supp_k\): \[RW_k= 2\times \frac{Supp_k \times Conf_k}{Supp_k+Conf_k}.\]

At the server side, conflicting rules, i.e., rules with the same antecedent but different consequents, are resolved through weighted aggregation. Let \(CR\) be the set of conflicting rules, and let \(\Gamma_l\) and \(rw_l\) denote the consequent coefficient vector and rule weight of the \(l\)-th rule in \(CR\). The merged consequent is computed as \[\Gamma= \frac{ \sum_{l=1}^{|CR|} \Gamma_l \cdot rw_l }{ \sum_{l=1}^{|CR|} rw_l },\]

and the merged rule weight is \[rw= \frac{1}{|CR|} \sum_{l=1}^{|CR|} rw_l.\]

This aggregation strategy preserves the linguistic structure and interpretability of the global TSK-FRBS while avoiding raw data sharing among clients.

Subsequent works by Corcuera Bárcena et al. [47], [49] further extend this idea by embedding federated fuzzy models into system-level architectures, including FL as-a-Service (FLaaS) frameworks for beyond-5G/6G networks. These studies show that interpretable fuzzy models can be trained federatively while maintaining human-readable fuzzy rules and achieving performance comparable to centralized learning.

Fuzzy Regression Trees (FRTs) extend the interpretable-by-design paradigm by organizing decision logic hierarchically, enabling implicit feature selection and improved scalability. In [49], Corcuera Bárcena et al. introduce an FL algorithm for FRTs that relies on sharing aggregated sufficient statistics–such as weighted sums of squares–rather than raw samples or gradients. The server uses these statistics to construct a global fuzzy tree through a federated tree induction procedure analogous to centralized training.

This statistics-based federation mechanism highlights a key insight of interpretable-by-design FedXAI: privacy preservation and interpretability can be jointly achieved by carefully designing what information is exchanged during federation. By avoiding iterative gradient exchange, federated FRTs maintain transparency while achieving competitive predictive performance.

Several works explicitly compare interpretable-by-design models with black-box models augmented by post-hoc explainability under identical federated conditions. Ducange et al. [48] conduct a comprehensive healthcare case study on Parkinson’s disease progression prediction, contrasting federated TSK-FRBSs with federated multilayer perceptrons explained via SHAP. While the neural model achieves slightly higher predictive accuracy, the interpretable-by-design approach exhibits superior transparency, explanation consistency across clients, and ease of clinical interpretation. This comparative analysis underscores that trustworthiness in FedXAI cannot be assessed through accuracy alone, but must also account for model-level interpretability interpretability and stability.

Beyond algorithmic contributions, practical adoption of interpretable-by-design FedXAI depends on appropriate tooling and software abstractions. Daole et al. [50] address this gap through OpenFL-XAI, an extension of the Intel OpenFL framework that enables FL of rule-based and fuzzy models. Rather than exchanging neural network weights, OpenFL-XAI supports the transmission and aggregation of symbolic rule representations encoded as tensors. This contribution demonstrates that explainability in FL is not solely a modeling concern, but also an infrastructural challenge requiring tailored orchestration and aggregation logic.

Interpretable-by-design models have also been integrated into system-level federated architectures. In FLaaS environments for beyond-5G/6G networks [47], inherently explainable fuzzy models are deployed as shared services, enabling transparency, accountability, and trust in distributed AI-driven network management. In these settings, interpretability supports not only human understanding but also service governance and operational monitoring across heterogeneous stakeholders.

This section has reviewed FedXAI methodologies through a role-oriented lens, focusing on how explainability is instantiated and operationalized within the FL lifecycle rather than on specific XAI techniques. The surveyed literature reveals a clear methodological progression from loosely coupled post-hoc explainability toward tightly integrated and intrinsically transparent FL paradigms.

Post-hoc FedXAI approaches, reviewed in Section 5.1, dominate current applications due to their architectural simplicity and compatibility with existing FL frameworks. By decoupling learning from explanation, these methods enable the deployment of expressive black-box models while satisfying basic transparency requirements through client-side or aggregated explanations. This paradigm has proven particularly effective in high-stakes domains such as healthcare, cybersecurity, and finance, where instance-level explanations support human validation and regulatory compliance. However, the passive nature of post-hoc explanations limits their influence on learning dynamics, robustness, and personalization. Moreover, explanation instability under non-IID data distributions and the potential privacy leakage associated with explanation sharing pose fundamental challenges to trustworthiness and governance.

Explainability-aware FedXAI methods, discussed in Section 5.2, represent a conceptual shift in which explainability evolves from a diagnostic artifact into an active control signal within the FL loop. By leveraging explanation signals to guide aggregation, clustering, personalization, and system-level coordination, these approaches explicitly acknowledge that transparency can improve not only interpretability but also learning efficiency and robustness. Explanation-guided weighting, clustering, and resource management demonstrate that explanation vectors can serve as high-level semantic representations of client data, offering principled mechanisms for managing heterogeneity and uncertainty. Nevertheless, this tighter integration introduces additional complexity, computational overhead, and new attack surfaces, particularly related to the integrity and privacy of explanation signals themselves.

Interpretable-by-design FedXAI approaches, reviewed in Section 5.3, constitute the strongest and most principled form of federated explainability. By embedding transparency directly into the model structure–through fuzzy rule-based systems, fuzzy regression trees, and other symbolic representations–these methods eliminate reliance on post-hoc explainers and provide globally consistent, human-readable decision logic. Crucially, they demonstrate that privacy preservation and explainability can be jointly achieved through careful redesign of federated training and aggregation protocols, often by exchanging aggregated statistics or symbolic parameters rather than gradients. While interpretable-by-design approaches offer superior explanation consistency and auditability, their applicability is currently constrained to structured-data domains and requires bespoke federated infrastructures and aggregation mechanisms.

Taken together, the three methodological paradigms reveal an inherent trade-off between model expressiveness, system complexity, and explainability guarantees. Post-hoc methods favor flexibility and ease of deployment, explainability-aware approaches balance transparency with adaptive coordination, and interpretable-by-design models prioritize trust and governance at the cost of reduced expressiveness and increased system specialization. Importantly, these paradigms should not be viewed as mutually exclusive. Several recent works suggest hybrid directions that combine interpretable cores with explainability-aware coordination or selectively apply post-hoc explanations atop intrinsically transparent models.

From a broader perspective, the evolution of FedXAI methodologies reflects a shift from viewing explainability as an optional add-on toward recognizing it as a first-class design constraint in distributed learning systems. As FL continues to expand into regulated, safety-critical, and large-scale environments, future FedXAI research must address open challenges related to explanation consistency, privacy-preserving explanation aggregation, robustness against explanation manipulation, and standardized evaluation protocols. Ultimately, achieving trustworthy federated AI will require principled integration of explainability across algorithmic, system, and governance layers rather than isolated methodological advances.

6 Evaluation of FedXAI↩︎

Despite the rapid growth of FedXAI, its evaluation remains largely ad hoc and fragmented. Most existing studies primarily report predictive performance and provide qualitative explanation visualizations, while lacking a unified and principled evaluation methodology tailored to federated explainability. As highlighted in early conceptual works on FedXAI [1], the absence of standardized evaluation protocols constitutes a fundamental research gap that limits reproducibility, comparability, and real-world trustworthiness. This section formalizes the FedXAI evaluation gap by clarifying what should be evaluated, how evaluation is currently conducted, and why existing benchmarks are insufficient.

6.1 What Should Be Evaluated in FedXAI?↩︎

Evaluation in FedXAI must extend beyond classical performance metrics and explicitly account for explainability, privacy, robustness, and system-level costs induced by federation.

Most FedXAI studies continue to assess predictive performance using task-specific metrics such as accuracy, F1-score, RMSE, or AUC, comparing centralized, local-only, and federated settings [53], [63], [68], [70], [76]. While necessary, predictive performance alone is insufficient, as models with similar accuracy may exhibit substantially different explanation behaviors across clients and federated rounds [3], [4].

A defining goal of FedXAI is to generate explanations that remain meaningful under data decentralization and heterogeneity. Consequently, explanation quality must be evaluated explicitly, typically along dimensions such as fidelity, stability, sparsity, and consistency. Several works assess explanation fidelity using attribution stability or insertion–deletion analysis in image and signal domains [52], [75], while others report cross-round or cross-client consistency of SHAP- or Grad-CAM-based explanations [7], [48], [76]. However, these metrics are rarely standardized, and their sensitivity to non-IID data distributions remains an open challenge.

Although FL prevents raw data sharing, explanations themselves may leak sensitive information about local data distributions or rare features. This risk is explicitly acknowledged in privacy-preserving FedXAI frameworks that combine explainability with differential privacy or secure aggregation [4], [64], [66]. Nevertheless, systematic quantification of explanation-induced privacy leakage is still largely absent from FedXAI evaluation practices.

Federated systems are vulnerable to adversarial behaviors such as poisoning, backdoor insertion, and malicious client updates. In FedXAI, robustness evaluation must therefore consider not only predictive degradation under attack, but also manipulation or distortion of explanations. Recent intrusion-detection studies highlight the need to evaluate explanation reliability under adversarial participation [55], [58], yet explanation-aware attack models remain underexplored.

Explainability introduces additional overhead in federated systems, including client-side explanation computation, explanation aggregation, and increased communication payloads. Several works explicitly report communication and computational overhead associated with SHAP, Grad-CAM, or ensemble-based explainability [52], [54], [57]. However, cost metrics are rarely analyzed jointly with explanation quality, obscuring practical trade-offs in resource-constrained deployments.

6.2 Metrics and Evaluation Protocols↩︎

Current FedXAI evaluations rely on heterogeneous metrics drawn from centralized XAI, FL, and application-specific traditions, limiting cross-study comparability.

Stability metrics quantify how explanations evolve across federated rounds or differ between clients. Several studies measure distances between feature-attribution vectors (e.g., SHAP value distributions) to assess explanation drift under non-IID data [3], [56], [57]. Stability is particularly critical in cross-silo and safety-critical domains, where inconsistent explanations may erode trust.

Agreement metrics assess consistency between different explanation methods (e.g., SHAP vs.LIME), between clients, or between instance-level and model-level explanations. Such analyses are used to justify explanation reliability in intrusion detection, healthcare, and energy systems [52], [55], [57]. Low agreement may indicate either model uncertainty or explanation instability induced by federation.

For image-based and signal-based tasks, explanation fidelity is often evaluated using insertion and deletion curves, which quantify the impact of salient regions on model predictions [75]. While widely adopted, these metrics were originally designed for centralized models and may not fully capture the specific dynamics of federated explanation settings.

A comprehensive evaluation of explanation quality in FL is provided by Ducange et al. [80], who explicitly quantify how closely federated SHAP explanations approximate their centralized counterparts. They measure explanation accuracy by computing the Frobenius norm between the SHAP attribution matrices obtained under different background generation strategies and those produced using the full centralized training data.

This metric directly captures the discrepancy between federated and centralized explanations, offering a principled and quantitative way to assess explanation fidelity in federated settings. Their results show that Federated SHAP significantly reduces the explanation error compared to random or locally perturbed background datasets, sometimes by up to a factor of two or three, depending on the data modality.

Moreover, they demonstrate that explanation quality remains stable even when using compact synthetic backgrounds (e.g., a few thousand GAN-generated images), highlighting an effective trade-off between computational efficiency and explanation accuracy. This work thus establishes a concrete quantitative benchmark for evaluating explanation quality in FedXAI and represents one of the most rigorous validation frameworks currently available in the literature.

A smaller subset of works explicitly consider calibration alongside explainability, arguing that well-calibrated predictions with unstable explanations remain unsafe in high-stakes applications [5], [52]. Joint evaluation of calibration and explanation quality represents a promising yet underdeveloped direction in FedXAI.

Some studies introduce metrics tailored to federated explainability, such as cross-client explanation distance, explanation entropy, semantic consistency, or convergence of explanations across rounds [7], [48], [67]. However, these metrics are rarely reused across papers, preventing the emergence of a shared evaluation standard. Table 4 summarizes a core evaluation suite for FedXAI, highlighting the key dimensions, commonly used metrics, and representative studies.

6.3 The Benchmarking Gap in FedXAI↩︎

Despite extensive empirical experimentation, the FedXAI literature lacks dedicated benchmarks explicitly designed for federated explainability. Most studies rely on centralized public datasets that are artificially partitioned to simulate federated clients [53], [65], [70], [74]. While this approach improves reproducibility, it fails to capture real-world federation dynamics such as evolving data streams, institutional biases, asynchronous participation, and explanation leakage risks.

Moreover, most datasets do not provide ground-truth explanations, forcing researchers to rely on proxy metrics such as stability, sparsity, or fidelity without validating whether explanations align with domain knowledge [7], [52], [75]. As emphasized in survey and conceptual works [1], this limitation hinders systematic comparison and obscures whether reported explanations genuinely support human understanding and decision-making.

These observations highlight the urgent need for federated explainability benchmarks that jointly define realistic client heterogeneity, standardized evaluation protocols, privacy-aware explanation constraints, and domain-relevant explanation targets. Without such benchmarks, progress in FedXAI evaluation will remain fragmented, slowing adoption in regulated and safety-critical domains.

7 Datasets and Data Characteristics↩︎

Dataset characteristics in FedXAI differ fundamentally from those in centralized XAI benchmarks, where data homogeneity and unified annotation schemas are often implicitly assumed.

In FedXAI, data are not only distributed across clients, but also intrinsically heterogeneous, privacy-constrained, and often collected under distinct operational or institutional contexts. As a result, dataset characteristics directly influence the design of FL protocols, the choice of explainability techniques, and the interpretation of generated explanations. Rather than being a neutral input, data properties in FedXAI actively shape what types of explanations are feasible, stable, and meaningful.

FedXAI has been applied across a wide range of data modalities, including medical imaging, physiological time-series, tabular records, network traffic, and multimodal sensor data. Each modality affords different forms of explainability and constrains how explanations can be communicated to human stakeholders. In medical imaging applications, such as brain tumor classification [70], skin cancer detection [72], [74], eye disease diagnosis [73], and leukemia diagnosis [75], datasets consist of high-dimensional images with strong spatial structure. Consequently, explainability is predominantly visual and region-based, relying on saliency maps, Grad-CAM, occlusion sensitivity, or attention heatmaps to highlight diagnostically relevant areas. In these settings, explanations are tightly coupled to spatial coherence and clinical plausibility rather than feature ranking.

In contrast, tabular and network-centric datasets used in cybersecurity and finance–such as encrypted traffic classification [76], IoT intrusion detection [4], [57], [65], CPSS threat detection [64], and financial fraud detection [6], [61]–naturally lend themselves to feature-attribution explanations. Here, explainability is expressed through ranked feature contributions (e.g., packet statistics, protocol fields, transaction attributes), typically generated via SHAP or LIME. The dataset structure enables both instance-level explanations (per alert or transaction) and model-level summaries that support auditing and policy compliance. Time-series datasets, including ECG monitoring [69], [71] and smart grid load forecasting [53], occupy an intermediate position, where explanations often highlight influential temporal segments or lagged variables rather than static features or spatial regions.

A defining property of datasets in FedXAI is statistical heterogeneity across clients, commonly referred to as non-IID data. This heterogeneity arises from multiple sources, including institutional differences (e.g., hospitals using different imaging devices or protocols [48], [70]), geographical and environmental variation (e.g., solar farms and smart grids under different climate conditions [7], [53]), device-level diversity in IoT and vehicular networks [52], [62], and severe class imbalance in rare-event detection tasks such as fraud and intrusion detection [61], [65]. Importantly, non-IID data affect not only predictive performance but also the stability and consistency of explanations.

Several studies explicitly demonstrate that explanation patterns vary across clients under heterogeneous data distributions. For example, in federated intrusion detection, SHAP-based feature importance profiles differ significantly between client-side and federated models when data are skewed [3], [4]. Similarly, in healthcare applications, client-specific data characteristics lead to variations in saliency maps or feature attributions, raising concerns about whether a single model-level explanation accurately reflects local decision logic [5], [48]. These observations highlight that dataset heterogeneity is an explainability challenge in its own right, motivating the use of clustered FL [7], weighted aggregation strategies [62], or semantic enrichment [67] to mitigate explanation divergence.

Despite the practical motivation of FedXAI, a substantial portion of the literature relies on simulated federated settings constructed by partitioning centralized public datasets into artificial clients. This approach is common in healthcare [5], [70], [74], cybersecurity [57], [65], energy forecasting [7], [53], and finance [6], [61]. While simulation enables reproducibility and controlled experimentation, it often fails to capture real-world federation dynamics such as evolving data streams, cross-institutional biases, asynchronous participation, and concept drift. As noted in several studies, explanation quality observed under simulated conditions may overestimate the robustness and consistency achievable in operational deployments [3], [4].

Another limitation concerns the lack of ground-truth explanations in most datasets. Except for a few medical imaging tasks where expert-annotated regions are implicitly available, the majority of FedXAI datasets do not provide explicit rationales against which explanations can be quantitatively validated. Consequently, many works evaluate explainability qualitatively or through proxy metrics such as stability, sparsity, or fidelity [7], [52], [75]. While these metrics are informative, they do not fully address whether explanations align with domain knowledge or support correct human decision-making, underscoring a key gap in current FedXAI benchmarks.

The reviewed literature indicates that dataset characteristics are inseparable from FedXAI methodology. Data modality constrains the choice of explainability techniques, client heterogeneity affects explanation stability and aggregation, and dataset realism determines the external validity of reported interpretability claims. As a result, rigorous FedXAI evaluation requires explicit documentation of dataset properties, including modality, client partitioning strategy, degree and source of non-IID distributions, and availability of explanation-relevant annotations. Failure to account for these factors risks conflating algorithmic explainability with dataset-specific artifacts.

To facilitate systematic comparison across studies, Table 5 summarizes representative dataset characteristics used in FedXAI research, organized by application domain, data modality, heterogeneity type, and explainability implications. This structured overview highlights recurring patterns and open challenges in dataset design and selection for trustworthy and interpretable federated AI systems.

8 Application Domains and Use Cases↩︎

8.1 Healthcare and Medical Imaging↩︎

Healthcare is one of the most prominent application domains of FedXAI, driven by strict data privacy regulations and the need for clinically meaningful explanations [81]. A representative line of work focuses on ECG-based cardiac monitoring, where FL enables collaborative arrhythmia classification without centralizing sensitive patient data, while explainability supports clinical interpretability. Studies such as Raza et al. [69] and Manocha et al. [71] demonstrate how post-hoc attribution methods, adapted to one-dimensional signals, highlight salient temporal segments of ECG recordings, allowing clinicians to assess the physiological relevance of model predictions. These works illustrate a recurring pattern in healthcare FedXAI, where explainability primarily serves to justify predictions at the signal level rather than to audit model behavior.

Recent advances further extend FedXAI toward multimodal and privacy-preserving clinical decision support systems. Qazi et al. [82] propose a federated deep learning framework integrating Vision Transformers (ViT), DINOv2-based self-supervised learning, and FedProx to support personalized medical image analysis under heterogeneous and non-IID healthcare environments. Their framework combines differential privacy and elliptic curve cryptography (ECC) for secure federated communication, while Grad-CAM and LIME provide instance-level explainability for clinical transparency across tuberculosis, diabetic retinopathy, and brain tumor diagnosis tasks.

Chorney and Wang [83] investigate federated transfer learning for ECG analysis under realistic clinical settings with heterogeneous and non-IID data distributed across institutions. Their framework employs autoencoders to map ECG signals with different dimensions and lead configurations into a shared latent space, enabling privacy-preserving federated training across heterogeneous datasets. The study highlights the practical challenges of deploying federated healthcare models beyond idealized centralized assumptions and demonstrates the importance of handling heterogeneous medical data in realistic FedXAI environments.

Beyond time-series analysis, medical imaging constitutes a major use case of FedXAI, where visual interpretability is often indispensable. Several studies address collaborative diagnosis tasks such as brain tumor classification [70], skin cancer detection [72], [74], eye disease diagnosis [73], and leukemia detection [75] using federated deep learning combined with visual explanation techniques. In these settings, Grad-CAM, saliency maps, and attention-based mechanisms are employed to highlight spatial regions that drive classification decisions, enabling clinicians to verify whether models attend to pathologically relevant areas. The integration of explainability is critical for building trust, particularly when models are trained across multiple institutions with heterogeneous data distributions.

Recent healthcare FedXAI frameworks further integrate advanced architectures and hybrid strategies to address data scarcity and generalization challenges. Examples include the use of GAN-based data augmentation and Vision Transformers for viral disease detection [59], [60], where explainability is provided through attention mechanisms and post-hoc attribution methods. Collectively, healthcare applications demonstrate that FedXAI can reconcile cross-institutional collaboration with privacy constraints while delivering explanations that are aligned with clinical reasoning.

A recent example is SHAP-FL proposed by Düsing and Cimiano [84], which addresses explanation inconsistency across healthcare institutions in federated sepsis onset prediction. The framework combines federated model training with a histogram-based background dataset synthesis mechanism that enables clients to generate more consistent SHAP explanations without sharing sensitive patient records. Using multicentric ICU data, the authors demonstrate that the synthesized federated background dataset improves both the fidelity and clinical plausibility of instance-level explanations across participating institutions. This work highlights an important emerging direction in healthcare FedXAI: harmonizing explanations across heterogeneous medical centers while preserving privacy.

However, most studies remain simulation-based, and systematic validation of explanation consistency and clinical utility across institutions remains an open challenge.

8.2 Cybersecurity, Intrusion Detection, and IoT↩︎

Cybersecurity and IDSs represent one of the most active and practically relevant domains of FedXAI. Network traffic data are highly sensitive, frequently encrypted, and distributed across heterogeneous administrative domains, making centralized learning impractical. At the same time, IDS outputs directly inform mitigation and response actions, rendering explainability a functional requirement rather than a purely diagnostic feature.

Several works integrate explainability into federated IDS pipelines to improve transparency and adaptability under non-IID data. Carillo et al. [76] combine FL with class-incremental learning for encrypted traffic classification, using attribution- and instance-based explanations to analyze how decision boundaries evolve across federated updates. Similarly, SHAP- and LIME-based explanations are widely used to justify intrusion alerts and audit feature relevance in edge-IoT environments [4], [52], [65]. In these systems, explainability supports both instance-level alert interpretation and model-level auditing.

A recent example is XAI-EdgeSFL proposed by Zhao et al. [85], which integrates XAI with adaptive intrusion-resilient split FL for consumer healthcare IoT ecosystems. The framework combines lightweight edge-side Temporal Convolutional Networks with server-side classifiers, while employing SHAP, LIME, and Integrated Gradients to provide feature-level explanations for intrusion detection and health monitoring decisions. In addition to enhancing interpretability, the system introduces adaptive aggregation mechanisms to improve robustness against cyber threats under resource-constrained edge environments. This work illustrates an emerging direction in FedXAI that jointly addresses transparency, security, privacy preservation, and edge intelligence within healthcare-oriented IoT infrastructures.

FedXAI has also been applied to broader cyber-physical and industrial contexts. Yazdinejad et al. [64] employ SHAP-based explanations to interpret threat detection decisions across cyber, physical, and social subsystems, while Patel et al. [68] use visual explanations to support fault diagnosis in semiconductor manufacturing. In addition, explainability has been used as an analytical tool to understand FL dynamics themselves, revealing how aggregation improves feature relevance consistency compared to isolated local training [3].

Overall, cybersecurity-oriented FedXAI emphasizes robustness, transparency, and operational trust. However, open challenges include explanation reliability under adversarial manipulation, the high computational cost of attribution methods, and the lack of standardized evaluation protocols for explanation robustness in federated IDS deployments.

8.3 Energy, Smart Grid, and Transportation↩︎

Energy systems, smart grids, and intelligent transportation networks form a key application domain of FedXAI, where privacy-preserving collaboration, operational efficiency, and interpretability are jointly required. Data in these domains are inherently distributed across geographically dispersed and independently operated entities, while AI-driven decisions often have direct economic and safety implications.

In smart grid applications, FL has been combined with explainable deep models for load forecasting, enabling collaborative training across grid nodes without exposing consumption data [53]. Explainability, typically implemented via SHAP or LIME, allows operators to identify the temporal and environmental factors driving demand fluctuations, supporting planning and anomaly detection. In renewable energy forecasting, explainability has been embedded more deeply into the learning process itself. Explainable Clustered FL (XCFL) [7] uses SHAP-based feature relevance not only for interpretation but also to guide client clustering and aggregation under strong geographical heterogeneity.

Transportation-oriented FedXAI further highlights the functional role of explainability in dynamic and resource-constrained environments. In vehicular networks and smart mobility systems, FL frameworks have been combined with explainability techniques to support privacy-preserving and adaptive decision-making for energy management, trajectory prediction, and intelligent transportation services.

Abdullah et al. [86] propose a FedXAI framework for autonomous electric vehicle energy management in smart cities, where FL enables decentralized optimization of vehicular energy consumption while XAI techniques enhance transparency and trustworthiness in AI-driven decisions. Their framework employs a hierarchical FL architecture using real-world vehicular telemetry data and integrates explainability mechanisms to interpret the influence of traffic density, energy consumption, and driving conditions on model predictions.

Similarly, weighted FL frameworks leverage SHAP- and LIME-based explanations to interpret energy consumption predictions and to adjust client contributions based on reliability and context [62].

Beyond energy optimization, federated and explainable approaches have also been explored for transportation-related IoT security and malicious traffic detection. Bilal et al. [57] integrate FL with SHAP and LIME for interpretable intrusion detection across distributed IoT and vehicular-edge environments, demonstrating how explainability can support transparent security monitoring while preserving data locality.

Across these applications, explainability supports both transparency and coordination, enabling federated systems to operate effectively under heterogeneous, non-IID, and rapidly changing transportation conditions. Nonetheless, scalability, communication overhead, and real-time explanation generation remain important open challenges in large-scale intelligent transportation deployments.

8.4 Environment and Agriculture↩︎

Environmental monitoring and smart agriculture are well suited to FedXAI due to the distributed nature of data sources, strong ownership constraints, and the high societal impact of predictive decisions. FL enables collaboration across farms, regions, or agencies without centralizing sensitive environmental data, while explainability is essential for trust and actionable insight.

In smart agriculture, FedXAI frameworks employ SHAP-based feature attribution to reveal how environmental factors such as soil moisture, temperature, rainfall, and nutrient levels influence predictions related to yield estimation and irrigation planning [51]. Explainability plays a decision-support role by translating model outputs into agronomically meaningful insights. In environmental risk modeling, FL combined with SHAP-based explanations has been applied to flood prediction in transboundary river basins [63]. In this context, explainability highlights the relative contribution of upstream rainfall and hydrological dynamics, supporting policy-making and disaster preparedness.

Across these applications, explainability serves to align model behavior with physical and environmental understanding rather than to justify individual predictions. Open challenges include handling long-term temporal dependencies, validating explanations against domain-specific scientific models, and designing benchmarks that capture realistic cross-region dependencies and climate variability.

8.5 Finance↩︎

Financial fraud detection is a regulation-driven application domain where FedXAI is particularly compelling. Financial institutions are prohibited from sharing raw transaction data, yet are required to provide transparent and auditable explanations for automated decisions. FedXAI directly addresses this tension by enabling collaborative learning with built-in interpretability [87].

Several studies integrate SHAP-based explainability into federated fraud detection pipelines, allowing institutions to collaboratively train models while interpreting transaction-level risk factors [61]. In these systems, explainability supports regulatory compliance, post-hoc investigation, and internal risk assessment, rather than serving as a purely diagnostic tool. Feature-level explanations enable auditors and analysts to assess whether predictions are grounded in plausible financial behavior.

More conceptual analyses further emphasize that privacy-preserving learning alone is insufficient for trustworthy fraud detection, as accountability and interpretability are increasingly mandated by regulators [6]. Across the financial domain, explainability functions as a trust-enabling mechanism that bridges technical fraud models and non-technical stakeholders. However, challenges remain in ensuring explanation consistency across institutions, mitigating privacy leakage through feature attributions, and validating FedXAI systems in real-world inter-bank deployments.

9 Open Challenges and Future Directions (Roadmap)↩︎

FedXAI has progressed from a conceptual vision to a rapidly expanding body of applied research across domains such as healthcare, cybersecurity, smart grids, and intelligent transportation systems. Despite these advances, the literature reveals that FedXAI is still in an early, pre-paradigmatic stage, characterized by fragmented methodologies, domain-specific solutions, and limited theoretical grounding. The combination of FL constraints with explainability requirements introduces challenges that are not encountered when these paradigms are studied in isolation, making the formulation of a coherent roadmap both necessary and timely.

A fundamental open challenge concerns the privacy of explanations themselves. While FL prevents direct data sharing, many FedXAI frameworks implicitly treat explanations as non-sensitive artifacts. In practice, however, feature attributions, saliency maps, and explanation statistics can encode sensitive information about local data distributions, rare events, or client-specific patterns. Several studies exchange or aggregate SHAP-based information to enable model-level interpretability or explanation-driven optimization, thereby introducing new leakage channels that are not covered by classical FL threat models. Although differential privacy and cryptographic mechanisms have been incorporated into some FedXAI pipelines, empirical results indicate a significant trade-off between explanation fidelity, predictive performance, and computational overhead [64], [66]. Foundational work explicitly highlights the absence of a formal notion of explanation privacy in FedXAI, underscoring the need for principled privacy models tailored to explanatory artifacts rather than model parameters alone [1].

Closely related is the largely unexplored issue of explanation integrity and manipulation. FedXAI systems are vulnerable not only to model poisoning but also to explanation poisoning, in which adversarial clients deliberately distort explanations while preserving high predictive accuracy. This threat is particularly severe in approaches where explanations actively influence learning dynamics, such as explanation-weighted aggregation, clustering, or resource management strategies [7], [54]. While some security-oriented frameworks combine robust aggregation with explainability, explanation integrity is rarely treated as an explicit adversarial objective [55], [58]. Addressing this gap requires new robustness mechanisms that jointly reason about prediction behavior and explanatory consistency.

Another major challenge arises from the interaction between non-IID data and explanation stability. non-IID data distributions are inherent to FL, yet FedXAI further demands that explanations remain meaningful, comparable, and stable across heterogeneous clients. Empirical evidence shows that feature attributions can vary substantially across clients and federated rounds, even when global performance converges. A small number of studies propose quantitative measures, such as attribution distance or explanation consistency, to analyze this phenomenon [3], [57]. However, most works still rely on qualitative inspection, especially in complex multimodal or medical settings [51], [70]. Future research should formalize explanation drift and stability as core evaluation dimensions and explore explanation-level personalization strategies that complement predictive personalization.

The problem becomes more pronounced in continual and streaming FedXAI scenarios. Real-world federated systems must adapt to evolving data distributions, emerging classes, and concept drift, yet the majority of FedXAI evaluations assume static datasets and limited training horizons. Preliminary results in federated class-incremental learning demonstrate that explainability can reveal catastrophic forgetting and bias shifts, but they also expose the fragility of explanations under continual updates [76]. Conceptual studies identify streaming data and long-term accountability as central unresolved challenges in FedXAI [1]. Consequently, future systems must move beyond explaining instantaneous predictions and instead support explanations of model evolution and temporal change.

Scalability and efficiency remain persistent bottlenecks. Post-hoc explainability techniques such as SHAP and LIME are computationally expensive and often dominate the runtime and communication cost of FedXAI pipelines, particularly in cross-device and edge-based deployments. Several works mitigate these costs through lightweight models, pruning, or selective explanation strategies, but these solutions are largely heuristic [53], [57]. The integration of cryptographic privacy mechanisms further exacerbates computational and communication overhead [64], [66]. This motivates research into explanation-efficient designs, including compressed or sparse explanations, on-demand interpretability, and architectural choices that reduce reliance on heavy post-hoc methods.

An alternative and promising research direction involves interpretable-by-design models in federated settings. Federated fuzzy rule-based systems and fuzzy regression trees demonstrate that strong interpretability, low communication cost, and competitive performance can be jointly achieved [46], [49]. Tooling efforts such as OpenFL-XAI further show that such models can be operationalized in real federated workflows [50]. Nonetheless, these approaches face challenges related to rule complexity, aggregation conflicts, and scalability to high-dimensional or unstructured data. Hybrid architectures that combine deep representation learning with federated symbolic or rule-based explanation layers represent a compelling avenue for future research.

A notable gap across the literature is the limited emphasis on human-centered evaluation. Many FedXAI studies assess interpretability through visual plausibility or attribution rankings, yet few investigate whether these explanations improve human decision-making, calibration, or trust in real workflows. This limitation is particularly critical in high-stakes domains such as medical imaging and clinical decision support, where explanations are often cited as a prerequisite for deployment [69]–[71]. Future work should incorporate task-based human evaluations, role-specific explanation design, and longitudinal studies of trust and reliance.

Finally, the lack of standardization in benchmarks, metrics, and evaluation protocols remains a major barrier to progress. There is no consensus on how to jointly assess accuracy, privacy, robustness, efficiency, and explanation quality in FedXAI systems. While some studies introduce ad hoc FedXAI-specific metrics such as attribution stability or calibration measures, these are rarely comparable across works [52], [75]. Foundational surveys explicitly identify this fragmentation as a key open challenge [1]. Establishing standardized benchmarks, reference non-IID partitions, and unified reporting guidelines is essential for advancing FedXAI toward a mature and reproducible research field.

In summary, the future of FedXAI will be shaped less by incremental performance gains and more by the community’s ability to secure explanations, ensure their integrity under adversarial and non-IID conditions, scale them efficiently, and validate their utility in human-centered decision-making contexts. Addressing these challenges will determine whether FedXAI evolves into a foundational paradigm for trustworthy federated intelligence or remains a collection of domain-specific case studies.

10 Conclusion↩︎

Federated Explainable Artificial Intelligence (FedXAI) has emerged as a critical research direction at the intersection of privacy-preserving learning and trustworthy artificial intelligence. By jointly addressing the decentralization constraints of Federated Learning (FL) and the transparency requirements of explainable AI, FedXAI seeks to enable collaborative intelligence in domains where both data confidentiality and human interpretability are non-negotiable. This survey has provided a comprehensive and structured analysis of this emerging paradigm, highlighting how explainability is no longer a peripheral add-on, but an increasingly integral component of federated system design.

A central contribution of this work is the introduction of a multi-axis taxonomy that systematically organizes the FedXAI literature along five orthogonal dimensions: the role of explainability within the FL lifecycle, the family of XAI techniques employed, the scope and target of explanations, the integration level and exchanged artifacts, and the underlying FL setting and heterogeneity assumptions. This taxonomy enables principled comparison across heterogeneous approaches and reveals fundamental design trade-offs between transparency, privacy, scalability, and model expressiveness. By moving beyond single-axis categorizations, the proposed framework captures the diversity of FedXAI methodologies ranging from post-hoc interpretability to explanation-aware optimization and interpretable-by-design federated models.

Through an extensive methodological review, this survey has identified three dominant paradigms in FedXAI: post-hoc explainability applied to federated models, explainability-aware FL where explanations actively shape coordination and aggregation, and interpretable-by-design approaches that embed transparency directly into the model structure. Each paradigm offers distinct advantages and limitations. Post-hoc methods provide flexibility and ease of deployment but remain passive and vulnerable to instability under non-IID data. Explainability-aware approaches demonstrate that explanation signals can improve robustness, personalization, and heterogeneity management, albeit at the cost of increased system complexity and new security risks. Interpretable-by-design models offer the strongest transparency guarantees and governance benefits, yet currently face scalability and applicability constraints in unstructured data domains.

The survey further highlights that evaluation remains a major bottleneck for the maturation of FedXAI. Existing studies rely on heterogeneous metrics, simulated federations, and qualitative assessments that limit reproducibility and cross-paper comparison. In particular, explanation quality, stability under non-IID data, privacy leakage from explanations, robustness against manipulation, and system-level overhead are rarely evaluated in a unified manner. The lack of standardized benchmarks and evaluation protocols underscores the need for community-wide efforts toward reproducible and application-relevant FedXAI assessment.

From an application perspective, the reviewed literature demonstrates that FedXAI is most mature in healthcare, cybersecurity, energy systems, and finance, where regulatory constraints and high-stakes decision-making necessitate both privacy and interpretability. However, most deployments remain experimental and simulation-based, indicating a gap between methodological innovation and real-world operational validation. Bridging this gap will require closer integration of domain knowledge, human-centered evaluation, and system-level design considerations.

In conclusion, FedXAI represents a paradigm shift in the design of distributed intelligent systems, reframing explainability as a first-class constraint alongside privacy and performance. While significant progress has been made, the field remains fragmented and faces open challenges related to explanation security, non-IID stability, scalability, and evaluation standardization. Addressing these challenges will be essential for transforming FedXAI from a collection of promising techniques into a foundational framework for trustworthy, transparent, and privacy-preserving federated intelligence.

Acknowledgements↩︎

This work has been partly funded by the Italian Ministry of University and Research (MUR) within the framework of the PNRR - M4C2 - Investimento 1.3, Partenariato Esteso PE00000013 - “FAIR - Future Artificial Intelligence Research” - Spoke 1 “Human-centered AI” under the NextGeneration EU programme, and the framework of the FoReLab and CrossLab projects (Departments of Excellence).

Data Availability↩︎

No new data were generated or analyzed in this study. All data discussed in this article are derived from previously published studies.

References↩︎