[Paper Review] Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thought

Xiaoyu Tan, Yongxin Deng+6 more|arXiv (Cornell University)|Jul 18, 2024

Topic Modeling4 citations

TL;DR

This paper proposes Thought-Like-Pro, a self-driven framework that enhances large language models' reasoning by imitating verified, Prolog-generated reasoning trajectories. By using a symbolic Prolog engine to produce correct logical inferences and converting them into natural language Chain-of-Thought (CoT) prompts, the method enables LLMs to learn robust, generalizable reasoning patterns, significantly improving performance on out-of-distribution reasoning tasks with minimal catastrophic forgetting.

ABSTRACT

Large language models (LLMs) have shown exceptional performance as general-purpose assistants, excelling across a variety of reasoning tasks. This achievement represents a significant step toward achieving artificial general intelligence (AGI). Despite these advancements, the effectiveness of LLMs often hinges on the specific prompting strategies employed, and there remains a lack of a robust framework to facilitate learning and generalization across diverse reasoning tasks. To address these challenges, we introduce a novel learning framework, THOUGHT-LIKE-PRO In this framework, we utilize imitation learning to imitate the Chain-of-Thought (CoT) process which is verified and translated from reasoning trajectories generated by a symbolic Prolog logic engine. This framework proceeds in a self-driven manner, that enables LLMs to formulate rules and statements from given instructions and leverage the symbolic Prolog engine to derive results. Subsequently, LLMs convert Prolog-derived successive reasoning trajectories into natural language CoT for imitation learning. Our empirical findings indicate that our proposed approach substantially enhances the reasoning abilities of LLMs and demonstrates robust generalization across out-of-distribution reasoning tasks.

Motivation & Objective

• To address the lack of a generalizable framework for improving reasoning in large language models across diverse tasks.
• To reduce error propagation in autoregressive reasoning by grounding reasoning steps in a formally verified symbolic logic engine.
• To bridge the gap between symbolic reasoning and neural language models through imitation learning of verified CoT trajectories.
• To enable LLMs to generalize to out-of-distribution reasoning tasks by learning from logically correct, Prolog-verified inference paths.
• To mitigate catastrophic forgetting during domain-specific fine-tuning using model averaging.

Proposed method

• Utilize open-source LLMs to generate Prolog facts, rules, and queries from natural language instructions and demonstrations.
• Employ a symbolic Prolog engine to perform logical inference and verify the correctness of reasoning trajectories.
• Select only verified reasoning paths for conversion into natural language Chain-of-Thought (CoT) sequences.
• Translate Prolog-derived reasoning steps into structured, natural language CoT prompts for supervised fine-tuning (SFT).
• Apply model averaging during fine-tuning to prevent catastrophic forgetting and improve generalization.
• Use a self-driven loop: LLM generates logic → Prolog verifies → CoT is generated → LLM is fine-tuned on verified CoT.

Experimental results

Research questions

• RQ1 Can symbolic verification via Prolog improve the reliability and correctness of LLM reasoning trajectories?
• RQ2 Can imitation learning from verified Prolog reasoning paths enhance the generalization of LLMs on out-of-distribution reasoning tasks?
• RQ3 How does the integration of symbolic logic with neural models affect reasoning accuracy and robustness?
• RQ4 To what extent can self-driven, Prolog-augmented CoT generation reduce error propagation in autoregressive reasoning?
• RQ5 Does model averaging effectively mitigate catastrophic forgetting during task-specific fine-tuning in this hybrid framework?

Key findings

• The proposed Thought-Like-Pro framework significantly enhances the reasoning capabilities of LLMs across diverse reasoning tasks.
• The method demonstrates robust generalization to out-of-distribution reasoning tasks, outperforming standard CoT and few-shot prompting.
• Verified reasoning trajectories from the Prolog engine eliminate incorrect inference paths, reducing error propagation in LLMs.
• The use of model averaging during fine-tuning effectively mitigates catastrophic forgetting, preserving performance on previously learned tasks.
• The framework successfully converts Prolog-based logical inference into natural language CoT, enabling effective imitation learning by LLMs.
• Empirical results confirm that LLMs fine-tuned on Prolog-verified CoT sequences achieve higher accuracy and consistency in complex logic and arithmetic reasoning tasks.