[Paper Review] Boosting Language Models Reasoning with Chain-of-Knowledge Prompting

Jianing Wang, Qiushi Sun+2 more|arXiv (Cornell University)|Jun 10, 2023

Topic Modeling4 citations

TL;DR

This paper introduces Chain-of-Knowledge (CoK) prompting, a method that enhances large language models' reasoning by eliciting explicit, structured knowledge triples (evidentiary facts) alongside textual explanations. By integrating external knowledge bases and applying an F²-Verification strategy to assess factuality and faithfulness, CoK improves reasoning reliability and performance across commonsense, factual, symbolic, and arithmetic tasks, outperforming standard in-context learning and chain-of-thought prompting.

ABSTRACT

Recently, Chain-of-Thought (CoT) prompting has delivered success on complex reasoning tasks, which aims at designing a simple prompt like ``Let's think step by step'' or multiple in-context exemplars with well-designed rationales to elicit Large Language Models (LLMs) to generate intermediate reasoning steps. However, the generated rationales often come with mistakes, making unfactual and unfaithful reasoning chains. To mitigate this brittleness, we propose a novel Chain-of-Knowledge (CoK) prompting, where we aim at eliciting LLMs to generate explicit pieces of knowledge evidence in the form of structure triple. This is inspired by our human behaviors, i.e., we can draw a mind map or knowledge map as the reasoning evidence in the brain before answering a complex question. Benefiting from CoK, we additionally introduce a F^2-Verification method to estimate the reliability of the reasoning chains in terms of factuality and faithfulness. For the unreliable response, the wrong evidence can be indicated to prompt the LLM to rethink. Extensive experiments demonstrate that our method can further improve the performance of commonsense, factual, symbolic, and arithmetic reasoning tasks.

Motivation & Objective

• To address the brittleness of chain-of-thought (CoT) prompting, where LLMs generate unfactual or unfaithful reasoning chains.
• To improve reasoning reliability in LLMs by explicitly eliciting structured knowledge evidence in the form of knowledge triples.
• To develop an F²-Verification method that evaluates reasoning chains for factuality (alignment with ground-truth knowledge) and faithfulness (consistency with textual explanations).
• To enable a rethinking mechanism that identifies and corrects unreliable reasoning by highlighting erroneous evidence.
• To demonstrate the effectiveness of CoK prompting across diverse reasoning benchmarks, including commonsense, factual, symbolic, and arithmetic reasoning tasks.

Proposed method

• CoK prompting combines two components: Evidence Triples (CoK-ET), which are structured knowledge facts in the form of (subject, relation, object), and Explanation Hints (CoK-EH), which provide textual justifications for the reasoning chain.
• In-context exemplars are constructed by sampling labeled examples and appending a 'Let’s think step by step' prompt to elicit reasoning, then retrieving and manually annotating relevant knowledge triples from an external knowledge base.
• The F²-Verification strategy quantifies two reliability metrics: factuality (matching between reasoning evidence and ground-truth knowledge) and faithfulness (consistency between evidence and textual explanation and final answer).
• For unreliable outputs, the F²-verification identifies incorrect evidence, triggering a rethinking process that prompts the LLM to revise its reasoning.
• The method is applied in a zero-shot few-shot setting, without fine-tuning, by conditioning the LLM on CoK-structured in-context demonstrations.
• The rethinking process uses the identified faulty evidence to guide the LLM toward generating corrected reasoning chains, improving final answer accuracy.

Figure 1 : Comparison of three prompting methods: (a) ICL, (b) Chain-of-Thought (CoT), and (c) Chain-of-Knowledge (CoK) solving a StrategyQA question.

Experimental results

Research questions

• RQ1 Can structured knowledge triples improve the factuality and faithfulness of LLM reasoning compared to standard chain-of-thought prompting?
• RQ2 How effective is the F²-verification strategy in detecting unreliable reasoning chains based on evidence quality?
• RQ3 Does CoK prompting lead to measurable performance gains across diverse reasoning tasks such as commonsense, factual, symbolic, and arithmetic reasoning?
• RQ4 Can the rethinking mechanism guided by F²-verification effectively correct erroneous reasoning and improve answer accuracy?
• RQ5 To what extent does CoK prompting reduce hallucinated or inconsistent reasoning in LLMs without parameter updates?

Key findings

• CoK prompting significantly improves performance on commonsense reasoning tasks, outperforming standard in-context learning and chain-of-thought prompting.
• On factual reasoning benchmarks, CoK achieves higher accuracy by reducing unfactual reasoning chains through explicit knowledge triple integration.
• The F²-verification method successfully identifies unreliable reasoning chains, with a focus on detecting both factually incorrect evidence and inconsistent explanations.
• The rethinking process guided by F²-verification leads to improved answer accuracy by correcting erroneous reasoning paths.
• In arithmetic and symbolic reasoning tasks, CoK demonstrates superior robustness and reliability, particularly in maintaining logical consistency between evidence and conclusions.
• Empirical results show consistent gains across multiple datasets, including StrategyQA, demonstrating the method’s generalization across reasoning types.

Figure 2 : The proposed framework. We first construct exemplars with chain-of-knowledge (CoK) prompts. Then, the CoK prompts can be indicated to let the LLM generate reasoning chains, including evidence triples, explanation hints, and the final answer. At last, we estimate the reliability of reasoni