[Paper Review] Towards Explainable Neural-Symbolic Visual Reasoning

Adrien Bennetot, Jean-Luc Laurent+2 more|arXiv (Cornell University)|Sep 19, 2019

Explainable Artificial Intelligence (XAI)Computer Science22 references34 citations

TL;DR

The paper proposes a neural-symbolic explainable model that directly builds a knowledge base from data, constrains learning with symbolic reasoning, and uses it to reveal and correct biases in neural networks, demonstrated on image captioning.

ABSTRACT

Many high-performance models suffer from a lack of interpretability. There has been an increasing influx of work on explainable artificial intelligence (XAI) in order to disentangle what is meant and expected by XAI. Nevertheless, there is no general consensus on how to produce and judge explanations. In this paper, we discuss why techniques integrating connectionist and symbolic paradigms are the most efficient solutions to produce explanations for non-technical users and we propose a reasoning model, based on definitions by Doran et al. [2017] (arXiv:1710.00794) to explain a neural network's decision. We use this explanation in order to correct bias in the network's decision rationale. We accompany this model with an example of its potential use, based on the image captioning method in Burns et al. [2018] (arXiv:1803.09797).

Motivation & Objective

• Motivate interpretability in AI and highlight the need for explanations that are understandable to non-technical users.
• Propose a neural-symbolic architecture that couples a black-box neural model with an intrinsic knowledge base (KB) derived from data.
• Show how the KB can reveal biases in the model and guide learning to improve robustness and performance.
• Demonstrate the approach by adapting existing image captioning methods to expose reasoning and bias in predictions.

Proposed method

• Extend the neural-symbolic framework of Doran et al. (2017) by directly populating the KB from data instead of using an externally supplied KB.
• Extract symbolic rules from the first black-box and reflect them in a second black box to constrain learning (e.g., via loss functions, initialization, or hyperparameters).
• Define a reasoning component (the reasoner) that answers queries using the KB and links explanations to the final output.
• Generalize loss functions from Burns et al. (2018) to handle bias-prone words via a Confusion Loss and a Confidence Loss, encouraging neutrality when evidence is weak and specificity when evidence is strong.
• Combine L_CE (cross-entropy) with L_Confusion and L_Confidence in a unified loss to balance context usage and bias-aware explanations.
• Apply the model to image captioning with a base captioning network (Vinyals et al. 2014) and derive a bias-detecting word set B_word to guide the KB.
• Explain predictions in natural language via the reasoner, highlighting potential biases and the learning process.

Experimental results

Research questions

• RQ1 How can a knowledge base be derived directly from neural network learning data to support faithful explanations?
• RQ2 Can neural-symbolic reasoning reduce bias in vision-language tasks and provide interpretable, natural language explanations of model decisions?
• RQ3 How can loss functions be designed to make the model be confident when evidence exists and confused when it does not, revealing biases?
• RQ4 What are the trade-offs between maintaining performance and achieving interpretability in a neural-symbolic Captioning/recognition system?

Key findings

• The proposed model can detect provenance of biases in a neural network and provide unbiased predictions when context is weak.
• The approach yields explanations in natural language that reflect the model’s loss-based optimization process.
• Extracting and using a data-driven KB can constrain learning and improve performance while making reasoning transparent.
• Bias-prone word sets (B_word) can be leveraged to encourage the model to hesitate among interchangeable terms, reducing overgeneralization.
• The method demonstrates how to couple high-level symbolic reasoning with a neural predictor to produce faithful explanations without sacrificing accuracy.