[Paper Review] From Robustness to Explainability and Back Again

Xuanxiang Huang, João Marques‐Silva|arXiv (Cornell University)|Jun 5, 2023

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper proposes a novel method to scale formal explainability for deep neural networks by leveraging robustness tools as oracles, reducing explanation computation to at most one robustness query per feature. It establishes a tight duality between adversarial robustness and formal explanations, enabling efficient computation of distance-restricted abductive explanations for networks with hundreds of ReLU units—demonstrating a significant scalability improvement over prior formal methods.

ABSTRACT

Formal explainability guarantees the rigor of computed explanations, and so it is paramount in domains where rigor is critical, including those deemed high-risk. Unfortunately, since its inception formal explainability has been hampered by poor scalability. At present, this limitation still holds true for some families of classifiers, the most significant being deep neural networks. This paper addresses the poor scalability of formal explainability and proposes novel efficient algorithms for computing formal explanations. The novel algorithm computes explanations by answering instead a number of robustness queries, and such that the number of such queries is at most linear on the number of features. Consequently, the proposed algorithm establishes a direct relationship between the practical complexity of formal explainability and that of robustness. To achieve the proposed goals, the paper generalizes the definition of formal explanations, thereby allowing the use of robustness tools that are based on different distance norms, and also by reasoning in terms of some target degree of robustness. Preliminary experiments validate the practical efficiency of the proposed approach.

Motivation & Objective

• To address the critical scalability bottleneck in formal explainability of deep neural networks and other complex classifiers.
• To establish a rigorous, formal relationship between adversarial robustness and formal abductive/contrastive explanations.
• To generalize formal explanation definitions to incorporate norm-based distance bounds (e.g., l_p norms) and target robustness levels.
• To enable the reuse of existing robustness tools as oracles for computing formal explanations, thereby improving practical scalability.
• To demonstrate that the number of required robustness queries is linear in the number of features, significantly reducing computational overhead.

Proposed method

• Generalize formal explanations to include a distance threshold ε and an l_p norm, enabling robustness-aware explanation definitions.
• Reformulate the problem of computing formal explanations as a sequence of robustness queries using an adversarial example oracle.
• Design an algorithm that computes one ε-AXp (ε-Abductive eXplanation) per feature by testing feature deletions, ensuring at most n queries for n features.
• Leverage the duality between robustness and explanation: if a network is robust under ε, no ε-AXp exists; otherwise, ε-AXp identifies minimal feature sets causing misprediction.
• Use existing SMT-based robustness tools (e.g., Marabou) as oracles to answer adversarial queries efficiently.
• Implement a prototype in Perl and evaluate on ACASXU networks using Marabou as the robustness oracle with a 4800-second timeout per query.

Experimental results

Research questions

• RQ1 Can formal explainability be made scalable for deep neural networks by reusing robustness tools?
• RQ2 How can formal explanation definitions be generalized to incorporate distance constraints (e.g., l_p norms and ε)?
• RQ3 What is the relationship between the number of robustness queries and the number of features in computing formal explanations?
• RQ4 Can the duality between adversarial robustness and formal explanation be exploited to improve efficiency?
• RQ5 To what extent does the choice of ε and feature deletion order affect the performance of explanation computation?

Key findings

• The proposed method computes formal explanations using at most one robustness query per feature, achieving linear complexity in the number of features.
• The approach enables formal explanation for deep neural networks with several hundred ReLU units—representing more than an order of magnitude improvement over prior formal methods, which were limited to tens of units.
• For ACASXU networks, the method successfully computed ε-AXp explanations within the 4800-second timeout, with performance highly dependent on ε and the number of dropped features.
• Larger ε values or higher numbers of dropped features increased runtime due to larger search spaces, but the method remained practical for networks with hundreds of units.
• When ε was too small (e.g., 0.05), no ε-AXp existed, confirming that robustness implies absence of explanations, and that smaller ε values are less likely to yield explanations.
• The method demonstrated that explanation size and number of explanations are directly related to the robustness threshold ε, with higher ε values yielding more explanations.