[Paper Review] Interplay between Federated Learning and Explainable Artificial Intelligence: a Scoping Review

Luis M. López-Ramos, Florian Leiser+7 more|arXiv (Cornell University)|Nov 7, 2024

Privacy-Preserving Technologies in Data4 citations

TL;DR

This scoping review investigates the interplay between Federated Learning (FL) and Explainable AI (XAI), identifying a critical research gap: few studies quantify how FL affects model explanations. It finds that while most works focus on feature relevance explanations, FL aggregation dilutes node-specific interpretability patterns, and standardized reporting practices remain rare.

ABSTRACT

Contains basic and extraction data about papers included in a scoping review about the interplay between federated learning and explainable artificial intelligence.

Motivation & Objective

• To map and analyze the current state of research on the joint application of Federated Learning (FL) and Explainable AI (XAI).
• To identify methodological and reporting gaps in studies combining FL and XAI, particularly regarding transparency and reproducibility.
• To investigate how FL training affects model interpretability and post-hoc explanations, especially in terms of global vs. local explanation fidelity.
• To highlight the lack of standardized reporting practices, including FL library usage and data characteristic documentation.
• To call for more rigorous, quantitative research to assess the impact of FL on model structure and explainability

Proposed method

• Conducted a scoping review of peer-reviewed publications that explicitly address both FL and XAI, focusing on joint interplay in model interpretability or post-hoc explanations.
• Screened 37 studies that met inclusion criteria, with a focus on horizontal FL (HFL) setups involving 10 or fewer data centers.
• Categorized studies based on whether they emphasized interpretability (e.g., algorithmic transparency) or explanation methods (e.g., feature relevance).
• Evaluated reporting quality by assessing adherence to standards such as TRIPOD+AI and MINIMAR for data and model reporting.
• Analyzed the use of established FL libraries (e.g., Flower, PySyft) versus custom implementations, and assessed transparency in methodology description.
• Identified inconsistencies in terminology, particularly around 'explainability', 'interpretability', and 'XAI', and advocated for standardized definitions

Figure 1: Schematic overview of a main research question: do explanations differ between federated models and centralized models?

Experimental results

Research questions

• RQ1 How does federated learning influence the quality, fidelity, and consistency of model explanations compared to centralized training?
• RQ2 What proportion of FL and XAI studies use established FL libraries, and how does this affect reproducibility and transparency?
• RQ3 To what extent do studies report data characteristics, model performance, and explanation metrics in line with reporting guidelines?
• RQ4 What are the dominant types of explanation methods used in FL-XAI research, and how do they compare to interpretability-focused approaches?
• RQ5 What are the key methodological and reporting gaps that hinder reproducibility and responsible deployment of FL-XAI systems?

Key findings

• Only one study in the review explicitly and quantitatively analyzed the influence of FL on model explanations, revealing a significant research gap in this area.
• Aggregation of interpretability metrics across FL nodes often dilutes node-specific patterns, leading to generalized global insights at the cost of local interpretability fidelity.
• Eight papers incorporated explanation methods as components of the FL algorithm, suggesting potential for integrating XAI into the FL training loop.
• A minority of studies used established FL libraries (e.g., Flower, PySyft), with most either developing custom implementations or lacking implementation details.
• Many studies failed to report data characteristics or adhere to reporting guidelines such as TRIPOD+AI or MINIMAR, undermining reproducibility and auditability.
• There is a lack of consensus in terminology, with inconsistent use of terms like 'explainability', 'interpretability', and 'XAI', which hinders comparability and methodological clarity

Figure 2: Flow diagram indicating the number of papers identified, excluded in the different phases, and included in the review.