[Paper Review] Faithful Reasoning Using Large Language Models

Antonia Creswell, Murray Shanahan|arXiv (Cornell University)|Aug 30, 2022

Topic ModelingComputer Science38 citations

TL;DR

The paper presents a faithfulness-driven reasoning framework (Selection-Inference with a halter and a value-based beam search) that produces interpretable reasoning traces and improves multi-step question answering on ProofWriter and EntailmentBankQA datasets.

ABSTRACT

Although contemporary large language models (LMs) demonstrate impressive question-answering capabilities, their answers are typically the product of a single call to the model. This entails an unwelcome degree of opacity and compromises performance, especially on problems that are inherently multi-step. To address these limitations, we show how LMs can be made to perform faithful multi-step reasoning via a process whose causal structure mirrors the underlying logical structure of the problem. Our approach works by chaining together reasoning steps, where each step results from calls to two fine-tuned LMs, one for selection and one for inference, to produce a valid reasoning trace. Our method carries out a beam search through the space of reasoning traces to improve reasoning quality. We demonstrate the effectiveness of our model on multi-step logical deduction and scientific question-answering, showing that it outperforms baselines on final answer accuracy, and generates humanly interpretable reasoning traces whose validity can be checked by the user.

Motivation & Objective

• Motivate the need for interpretable, faithful multi-step reasoning in large language models (LLMs).
• Develop a forward-chaining architecture (Selection-Inference) that mirrors logical validity in its reasoning trace.
• Introduce a halting mechanism to decide when to stop and output answers in a way dependent on the trace rather than model priors.
• Incorporate a value-function-guided beam search to find high-quality reasoning traces.
• Evaluate the system on Reasoning datasets to demonstrate improved final-answer accuracy and trace validity.

Proposed method

• Two fine-tuned LMs form the backbone: a Selection model and an Inference model, creating a sequence of reasoning steps.
• A halter LM decides after each step whether the current inference suffices to answer, returning an answer only if it relies on the reasoning trace.
• A value-function LM scores partial traces to guide beam search over reasoning traces, selecting top traces to continue.
• Statements are selected from the context using sentence labels to prevent hallucination; the inference step predicts an entailment from the selected statements.
• Reasoning traces are defined and evaluated for validity, connectivity, and correctness with respect to logical entailment.
• The system uses a two-stage Halter to either produce an answer or declare Unknown if the trace is insufficient.
• Beam search explores a tree of traces by expanding with P candidate steps and pruning to the top B traces at each depth.

Experimental results

Research questions

• RQ1 Can a forward-chaining reasoning system with selection and inference steps produce valid, checkable reasoning traces?
• RQ2 Does a halting mechanism improve precision by outputting Answer or Unknown based on trace sufficiency?
• RQ3 Does a value-function-guided beam search improve final answer accuracy and trace quality over baselines?
• RQ4 Do faithful traces reduce hallucinations and improve use of context compared with prior approaches?

Key findings

• The SI+Halter+Search configuration achieves high final answer accuracy on PW (88.1%) and EB (78.1%) relative to baselines.
• SI yields higher-quality reasoning traces than baselines, with lower rates of halting on spurious facts on EntailmentBankQA and ProofWriter datasets.
• The halter reliably predicts when it knows the answer, enabling filtering of Unknown cases and improving precision substantially.
• The value-function-guided beam search improves performance, especially on deeper reasoning (PW depth-5) and EB with distractors.
• Baseline models tend to hallucinate more and rely less on the provided context and traces; SI+Halter demonstrates better trace validity and trace-driven answering.