[Paper Review] An Interpretable Reasoning Network for Multi-Relation Question Answering

Mantong Zhou, Minlie Huang+1 more|arXiv (Cornell University)|Jan 15, 2018

Topic Modeling34 references82 citations

TL;DR

The paper proposes an Interpretable Reasoning Network (IRN) that answers multi-relation KB questions with hop-by-hop reasoning, making intermediate relations and entities observable for analysis and manual intervention, achieving state-of-the-art results on two datasets.

ABSTRACT

Multi-relation Question Answering is a challenging task, due to the requirement of elaborated analysis on questions and reasoning over multiple fact triples in knowledge base. In this paper, we present a novel model called Interpretable Reasoning Network that employs an interpretable, hop-by-hop reasoning process for question answering. The model dynamically decides which part of an input question should be analyzed at each hop; predicts a relation that corresponds to the current parsed results; utilizes the predicted relation to update the question representation and the state of the reasoning process; and then drives the next-hop reasoning. Experiments show that our model yields state-of-the-art results on two datasets. More interestingly, the model can offer traceable and observable intermediate predictions for reasoning analysis and failure diagnosis, thereby allowing manual manipulation in predicting the final answer.

Motivation & Objective

• Motivate and address multi-relation question answering over knowledge bases (KBQA).
• Introduce an interpretable, hop-by-hop reasoning process that selects relations at each step.
• Enable traceable intermediate predictions to facilitate reasoning analysis and failure diagnosis.
• Show state-of-the-art performance on benchmark datasets and demonstrate robustness and interpretability.

Proposed method

• Proposes an Interpretable Reasoning Network (IRN) with input, reasoning, and answer modules.
• Reasoning module at each hop predicts a relation, updates the question and state, and determines when to stop via a Terminal relation.
• Input module encodes the question and updates q^h by subtracting the predicted relation (q^h = q^{h-1} - M_rq r_hat^h).
• Reasoning module computes g_j^h via softmax over relations using q^{h-1} and s^{h-1}, then forms r_hat^h as a weighted sum of relation embeddings.
• State s^h is updated as s^{h-1} + M_rs r_hat^h to encode accumulated reasoning.
• Answer module predicts an entity a^h from the state using e^h = M_se s^h and o_i^h from softmax over entity embeddings.
• Uses a multi-task loss combining relation prediction and entity prediction (with a balancing parameter lambda).
• Introduces multitask KB representation training via a constraint inspired by TransE: M_se (e_s + r) ≈ e_o, aiding robustness with incomplete paths.
• Handles conjunctive questions by running parallel IRNs and intersecting their results.

Experimental results

Research questions

• RQ1 Can an interpretable, hop-by-hop reasoning network effectively answer multi-relation KB questions?
• RQ2 Does exposing intermediate relations and entities improve traceability, failure diagnosis, and potential manual guidance?
• RQ3 Can joint supervision from KB embeddings and QA data boost performance on multi-hop questions?
• RQ4 Is the approach robust to incomplete KBs and unseen relations?

Key findings

• IRN achieves state-of-the-art results on two KBQA datasets (PathQuestion and WorldCup2014) for multi-hop and conjunctive questions.
• IRN provides traceable intermediate predictions (relations and entities) at each hop, enabling reasoning analysis and failure diagnosis.
• Auxiliary KB embedding training via a multitask objective improves robustness to incomplete knowledge bases and helps predict missing links.
• Compared to baselines, IRN shows strong performance on single-hop, multi-hop, and conjunctive questions and offers greater interpretability than many memory-based models.
• The framework supports manual manipulation of intermediate predictions to improve final answers, demonstrated by higher accuracy when ground-truth intermediate steps are provided.