[Paper Review] Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs

Sen Yang, Xin Li+3 more|arXiv (Cornell University)|Nov 16, 2023

Natural Language Processing Techniques4 citations

TL;DR

This paper proposes CaRing, a neuro-symbolic framework that combines a large language model (LLM) for knowledge extraction with an LLM-free symbolic solver for deliberative reasoning, ensuring causal and reliable reasoning proofs. By using customized meta-interpreters to execute formalized knowledge, CaRing achieves near-perfect accuracy (98.80% on 7B, 100% on 34B) and over triple the reasoning proof similarity compared to CoT baselines on ProofWriter, with significant gains on GSM8K and PrOntoQA.

ABSTRACT

Two lines of approaches are adopted for complex reasoning with LLMs. One line of work prompts LLMs with various reasoning structures, while the structural outputs can be naturally regarded as intermediate reasoning steps. Another line of work adopt LLM-free declarative solvers to do the reasoning task, rendering higher reasoning accuracy but lacking interpretability due to the black-box nature of the solvers. Aiming to resolve the trade-off between answer accuracy and interpretability, we present a simple extension to the latter line of work. Specifically, we showcase that the intermediate search logs generated by Prolog interpreters can be accessed and interpreted into human-readable reasoning proofs. As long as LLMs correctly translate problem descriptions into Prolog representations, the corresponding reasoning proofs are ensured to be causal and reliable. On two logical reasoning and one arithmetic reasoning datasets, our framework obtains significant improvements in terms of both answer accuracy and reasoning proof accuracy. Our code is released at https://github.com/DAMO-NLP-SG/CaRing

Motivation & Objective

• To address the lack of causality and reliability in reasoning proofs generated by large language models (LLMs), which often produce correct answers with erroneous or hallucinated intermediate steps.
• To develop a neuro-symbolic framework that separates knowledge representation (via LLMs) from deliberative reasoning (via symbolic solvers), ensuring deterministic and traceable inference.
• To produce reasoning proofs that are structurally and causally aligned with gold-standard proofs, particularly in complex, multi-premise reasoning tasks.
• To evaluate the method on diverse reasoning datasets, including logical reasoning (ProofWriter, PrOntoQA) and natural-language arithmetic reasoning (GSM8K), to assess generalization and robustness.

Proposed method

• A two-stage neuro-symbolic framework: first, an LLM-based symbolic representation generator (SymGen) translates natural language problems into formal, machine-readable knowledge representations.
• Second, a symbolic inference engine (SymInfer) performs deliberative reasoning on the formalized knowledge using customized meta-interpreters that support traceable execution and flexible search strategies.
• The symbolic solver operates in isolation from the LLM during reasoning, preventing hallucinations and ensuring deterministic, causal execution of inference steps.
• Meta-interpreters are implemented to track reasoning paths, enabling full traceability and support for diverse search strategies such as depth-first or best-first search.
• Reasoning proofs are represented as directed acyclic graphs (DAGs), and similarity is measured using normalized graph edit distance between predicted and gold reasoning graphs.
• The method is evaluated across multiple LLM sizes (7B, 13B, 34B) and datasets, with inference cost kept minimal via optimized LLM serving (e.g., vLLM).

Experimental results

Research questions

• RQ1 Can a neuro-symbolic approach significantly improve the reliability and causality of reasoning proofs generated by LLMs, especially when intermediate steps are erroneous despite correct answers?
• RQ2 How effective is an LLM-free symbolic solver in maintaining reasoning traceability and structural fidelity when processing natural language problems translated into formal logic?
• RQ3 To what extent can a hybrid neuro-symbolic system generalize across datasets with varying linguistic complexity, such as synthetic logical reasoning (ProofWriter) and real-world arithmetic reasoning (GSM8K)?
• RQ4 Does decoupling knowledge representation from reasoning execution lead to measurable improvements in both answer accuracy and reasoning proof similarity compared to end-to-end LLM prompting?

Key findings

• On ProofWriter, CaRing achieved 98.80% accuracy and 83.17% reasoning proof similarity using the 7B model, outperforming the CoT baseline by nearly double in accuracy (52.00%) and more than triple in proof similarity (24.12%).
• On PrOntoQA, CaRing reached 100% accuracy with the 34B model, demonstrating strong performance on a dataset with simplified, natural-language logical statements.
• On GSM8K, CaRing achieved 42.22% accuracy and 22.91% reasoning proof similarity with the 34B model, nearly doubling the proof similarity of the strongest CoT baseline (13.04%).
• The method maintained high performance across all model sizes, with accuracy increasing from 7B to 34B, indicating scalability and robustness.
• Despite the inherent ambiguity in natural language, CaRing generalized effectively to GSM8K, suggesting that LLMs’ generalization ability complements symbolic reasoning in real-world settings.
• The inference cost remained low due to optimized LLM serving, with only a marginal increase in output length despite the longest reasoning traces.