[論文レビュー] Privacy Implications of Explainable AI in Data-Driven Systems

Fatima Ezzeddine|arXiv (Cornell University)|Jun 22, 2024

Privacy-Preserving Technologies in Data被引用数 5

ひとこと要約

論文は、データ駆動システムにおける説明可能AI（XAI）とプライバシーの緊張関係を分析し、説明からの潜在的なリークリスクを概説し、研究質問と初期結果とともにプライバシー保護アプローチを探る。

ABSTRACT

Machine learning (ML) models, demonstrably powerful, suffer from a lack of interpretability. The absence of transparency, often referred to as the black box nature of ML models, undermines trust and urges the need for efforts to enhance their explainability. Explainable AI (XAI) techniques address this challenge by providing frameworks and methods to explain the internal decision-making processes of these complex models. Techniques like Counterfactual Explanations (CF) and Feature Importance play a crucial role in achieving this goal. Furthermore, high-quality and diverse data remains the foundational element for robust and trustworthy ML applications. In many applications, the data used to train ML and XAI explainers contain sensitive information. In this context, numerous privacy-preserving techniques can be employed to safeguard sensitive information in the data, such as differential privacy. Subsequently, a conflict between XAI and privacy solutions emerges due to their opposing goals. Since XAI techniques provide reasoning for the model behavior, they reveal information relative to ML models, such as their decision boundaries, the values of features, or the gradients of deep learning models when explanations are exposed to a third entity. Attackers can initiate privacy breaching attacks using these explanations, to perform model extraction, inference, and membership attacks. This dilemma underscores the challenge of finding the right equilibrium between understanding ML decision-making and safeguarding privacy.

研究の動機と目的

• データとモデルが機微である場合のプライバシーリスクを認識しつつ、透明性のツールとしてXAIの研究を推進する。
• XAIの出力（例えば feature importance や counterfactuals）などが情報を漏らし、プライバシー攻撃を可能にする方法を特定する。
• 説明が露出した場合のプライバシーを保護する際の効果的なプライバシー保護技術（例：Differential Privacy）を評価する。
• 高品質な説明と強力なプライバシー保護のバランスを取る手法を提案する。
• プライバシー保護されたXAIとプライバシー保護されたMLアプローチの比較の視点を提示する。

提案手法

• ポストホック説明、feature importance、counterfactual explanations を含むXAIの概念を調査・統合する。
• 説明に関連するプライバシー攻撃（membership inference、model extraction、model inversion など）を分析する。
• Differential Privacy や Federated Learning などのデータおよびモデルのプライバシー技術と、それらがパフォーマンスに与える影響を論じる。
• 強化学習（RL）ベースの CF生成とプライバシー対応CF生成、および勾配ベースの FI 計算への DP統合を潜在的な道筋として提案する。
• RLによるCF生成の探索、知識蒸留攻撃、およびVertical Split Learningの懸念に関する初期結果を示す。

Figure 1: Scenario of privacy attacks where MLaaS provides explanations alongside the prediction

実験結果

リサーチクエスチョン

• RQ1 RQ1: How effectively do privacy-preserving techniques (e.g., differential privacy) guard against information leakage when explanations are provided?
• RQ2 RQ2: Can high-quality XAI explanations be produced while maintaining privacy to mitigate attacks?
• RQ3 RQ3: Which approach is more effective for safeguarding privacy in XAI systems, privacy-preserving XAI or privacy-preserving ML?
• RQ4 RQ4: How do DP and other privacy methods affect the explainability and utility of XAI outputs like CFs and FI?

主な発見

• Initial exploration shows CFs can reveal decision boundaries and information about the model when exposed as explanations.
• Demonstrated that knowledge distillation can facilitate model extraction attacks in the presence of CFs via a new attack approach.
• Found that integrating DP into CF generation pipelines can help produce private CFs that deviate from confidential data properties.
• Vertical Split Learning (VSL) studies indicate gradient-based explanations can leak original feature information across domains.
• Analysis suggests trade-offs between DP-induced privacy and the level of explainability achieved by SHAP and related methods.
• The work presents a roadmap for defense mechanisms that enable high-quality explanations with privacy protections.