[论文解读] GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Framework

Hannah Sansford, Nicholas Richardson+2 more|arXiv (Cornell University)|Jul 15, 2024

Plant-based Medicinal Research被引用 6

一句话总结

GraphEval 将 LLM 输出表示为知识图谱，并使用 NLI 检测每个三元组的幻觉，从而提高检测准确性并实现解释；它还为部分幻觉纠正引入 GraphCorrect。

ABSTRACT

Methods to evaluate Large Language Model (LLM) responses and detect inconsistencies, also known as hallucinations, with respect to the provided knowledge, are becoming increasingly important for LLM applications. Current metrics fall short in their ability to provide explainable decisions, systematically check all pieces of information in the response, and are often too computationally expensive to be used in practice. We present GraphEval: a hallucination evaluation framework based on representing information in Knowledge Graph (KG) structures. Our method identifies the specific triples in the KG that are prone to hallucinations and hence provides more insight into where in the response a hallucination has occurred, if at all, than previous methods. Furthermore, using our approach in conjunction with state-of-the-art natural language inference (NLI) models leads to an improvement in balanced accuracy on various hallucination benchmarks, compared to using the raw NLI models. Lastly, we explore the use of GraphEval for hallucination correction by leveraging the structure of the KG, a method we name GraphCorrect, and demonstrate that the majority of hallucinations can indeed be rectified.

研究动机与目标

• Address the lack of explainability in hallucination detection for LLM outputs grounded in given context.
• Propose a KG-based preprocessing step to identify inconsistent triples before standard evaluation models.
• Show that GraphEval improves balanced accuracy on benchmark datasets.
• Demonstrate the potential of GraphCorrect for correcting hallucinations while preserving original text quality.

提出的方法

• Construct a knowledge graph from LLM output via prompting strategies (CoT and in-context learning).
• For each KG triple, apply an out-of-the-box NLI model to assess factual consistency against the grounding context.
• Classify the example as inconsistent if any triple exceeds a 0.5 inconsistency probability.
• Provide explainability by returning inconsistent triples; use as a preprocessing step for existing hallucination detectors.
• Optionally extend to GraphCorrect by correcting identified hallucinations per triple and reassembling the output.

Figure 1: A visualisation of the GraphEval approach. First, the LLM output is fed into the KG construction prompt to produce the KG depicted on the right. Next, each individual triple in the KG is fed into an out-of-the-box hallucination detection method, such as an NLI model, and compared to the pr

实验结果

研究问题

• RQ1 Can KG-based preprocessing improve the accuracy of existing hallucination detection models?
• RQ2 To what extent can per-triple grounding checks reveal the exact location of hallucinations within an LLM output?
• RQ3 Is it feasible to correct hallucinations by targeting only inconsistent KG triples (GraphCorrect) while maintaining overall text fidelity?

主要发现

• GraphEval consistently improves balanced accuracy when paired with NLI models across SummEval, QAGS-C, and QAGS-X (average improvement ~6.2 points).
• The framework identifies specific KG triples that are inconsistent with the grounding context, enhancing explainability of detected hallucinations.
• GraphCorrect outperforms a direct prompting baseline in correcting hallucinations across most benchmarks, especially for longer/structurally complex outputs.
• KG-based preprocessing reduces computational burden by enabling a single LLM pass for KG construction and using cheaper NLI checks thereafter.
• When output length is short (e.g., QAGS-X), benefits of GraphEval are diminished, highlighting its strength on longer, more complex outputs.