[Paper Review] Reading Comprehension using Entity-based Memory Network

Xun Wang, Katsuhito Sudoh+4 more|arXiv (Cornell University)|Dec 12, 2016

Topic Modeling26 references3 citations

TL;DR

This paper proposes an entity-based memory network (EntityMNN) that enhances reading comprehension by representing text through dynamically updated entity states in a memory pool, enabling fine-grained reasoning over long contexts. The model achieves state-of-the-art performance on multiple question-answering benchmarks, including 76.1% and 76.6% average accuracy on MC160 and MC500 of the MCTest dataset, respectively, outperforming prior methods without relying on complex semantic features.

ABSTRACT

This paper introduces a novel neural network model for question answering, the \emph{entity-based memory network}. It enhances neural networks' ability of representing and calculating information over a long period by keeping records of entities contained in text. The core component is a memory pool which comprises entities' states. These entities' states are continuously updated according to the input text. Questions with regard to the input text are used to search the memory pool for related entities and answers are further predicted based on the states of retrieved entities. Compared with previous memory network models, the proposed model is capable of handling fine-grained information and more sophisticated relations based on entities. We formulated several different tasks as question answering problems and tested the proposed model. Experiments reported satisfying results.

Motivation & Objective

• To improve neural network reasoning over long text by modeling entities as core information units rather than entire sentences.
• To address the limitation of sentence-level memory networks in capturing fine-grained, relation-based information in text.
• To develop a scalable, general-purpose QA framework that can be applied across diverse NLP tasks without heavy reliance on external knowledge or complex feature engineering.
• To evaluate the model's effectiveness on diverse datasets, including bAbI, MCTest, and large movie reviews, to demonstrate robustness and generalization.

Proposed method

• The model constructs a memory pool composed of entity states, where each entity's representation is updated incrementally based on input text.
• Input sentences are parsed to extract entities and their relations, which are then used to update the corresponding entity states in the memory pool.
• For question answering, the model retrieves relevant entities by matching the question vector to entity states using a similarity-based retrieval mechanism.
• The answer is predicted by a response module that aggregates information from retrieved entity states, enabling reasoning over entity-level relationships.
• The model is trained end-to-end using gradient descent to minimize the cross-entropy loss between predicted and ground-truth answers.
• The approach supports both open-domain and multiple-choice QA by treating each answer option as a hypothesis to be verified via entity state analysis.

Experimental results

Research questions

• RQ1 Can modeling text through entity states improve reasoning performance in question-answering tasks compared to sentence-level memory networks?
• RQ2 How well does the entity-based memory network handle fine-grained relations and long-context comprehension in reading comprehension tasks?
• RQ3 Can the model generalize across diverse datasets without requiring extensive feature engineering or external knowledge?
• RQ4 Does the entity-based approach outperform existing state-of-the-art models on standard benchmarks like MCTest and IMDB?

Key findings

• The entity-based memory network achieved an average accuracy of 76.1% on the MC160 subset and 76.6% on the MC500 subset of the MCTest dataset, outperforming prior state-of-the-art methods.
• On the Large Movie Review Dataset, the model achieved 97.2% accuracy, surpassing previous SOTA models that used more complex architectures and features.
• The model demonstrated strong performance on the bAbI dataset, indicating its effectiveness in handling diverse reasoning tasks, including those requiring multi-hop inference.
• The results show that entity-level reasoning enables better capture of fine-grained information and complex relations, improving performance even without external knowledge or rich semantic features.
• The model is robust across different languages and tasks, with potential for easy adaptation to low-resource or pro-drop languages with further extension.