[Paper Review] LOREN: Logic Enhanced Neural Reasoning for Fact Verification.

Jiangjie Chen, Qiaoben Bao+5 more|arXiv (Cornell University)|Dec 25, 2020

Topic Modeling34 references4 citations

TL;DR

LOREN proposes a logic-enhanced neural fact verification model that decomposes statements into logical reasoning units, treats each as a question-answering task using neural embeddings, and aggregates predictions via a logic-guided joint reasoner. It achieves 73.43% FEVER score, outperforming prior methods with improved interpretability through intermediate reasoning traces.

ABSTRACT

Given a natural language statement, how to verify whether it is supported, refuted, or unknown according to a large-scale knowledge source like Wikipedia? Existing neural-network-based methods often regard a sentence as a whole. While we argue that it is beneficial to decompose a statement into multiple verifiable logical points. In this paper, we propose LOREN, a novel approach for fact verification that integrates both Logic guided Reasoning and Neural inference. The key insight of LOREN is that it decomposes a statement into multiple reasoning units around the central phrases. Instead of directly validating a single reasoning unit, LOREN turns it into a question-answering task and calculates the confidence of every single hypothesis using neural networks in the embedding space. They are aggregated to make a final prediction using a neural joint reasoner guided by a set of three-valued logic rules. LOREN enjoys the additional merit of interpretability -- it is easy to explain how it reaches certain results with intermediate results and why it makes mistakes. We evaluate LOREN on FEVER, a public benchmark for fact verification. Experiments show that our proposed LOREN outperforms other previously published methods and achieves 73.43% of the FEVER score.

Motivation & Objective

• To improve fact verification by decomposing natural language statements into smaller, verifiable logical units centered on key phrases.
• To address the limitation of existing neural methods that treat entire statements as monolithic units without interpretability.
• To integrate symbolic logic rules with neural inference for more robust and explainable predictions.
• To enable model interpretability by exposing intermediate reasoning steps and error analysis.
• To achieve state-of-the-art performance on the FEVER benchmark while maintaining logical consistency and transparency.

Proposed method

• LOREN decomposes a given statement into multiple reasoning units based on central phrases to isolate verifiable claims.
• Each reasoning unit is transformed into a question-answering task, where neural networks compute confidence scores in the embedding space.
• A neural joint reasoner aggregates individual hypothesis predictions using a set of three-valued logic rules (support, refute, unknown).
• The logic rules guide the aggregation process to ensure consistency and logical coherence across multiple claims.
• The model leverages pre-trained sentence embeddings to encode statements and evidence, enabling effective similarity and confidence scoring.
• Intermediate predictions are exposed to provide interpretability, allowing users to trace reasoning and diagnose errors.

Experimental results

Research questions

• RQ1 Can decomposing a natural language statement into smaller logical units improve fact verification performance?
• RQ2 How effective is combining neural inference with symbolic logic rules in enhancing prediction accuracy and robustness?
• RQ3 To what extent does the proposed method improve interpretability compared to end-to-end neural models?
• RQ4 Can the model generalize to the FEVER benchmark while maintaining high accuracy and logical consistency?
• RQ5 What role do intermediate reasoning steps play in identifying and explaining model errors?

Key findings

• LOREN achieves a FEVER score of 73.43%, outperforming previously published methods on the benchmark.
• The decomposition of statements into logical units enables more accurate and fine-grained verification of claims.
• The integration of three-valued logic rules with neural confidence scores improves prediction consistency and reduces erroneous classifications.
• The model's intermediate reasoning traces provide clear explanations for predictions, enhancing interpretability and error diagnosis.
• The approach demonstrates that combining neural reasoning with symbolic logic leads to better performance and transparency than purely neural methods.