[Paper Review] CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answering

Nirmalie Wiratunga, Ramitha Abeyratne+7 more|arXiv (Cornell University)|Apr 4, 2024

Artificial Intelligence in Law4 citations

TL;DR

This paper proposes CBR-RAG, a retrieval-augmented generation framework that integrates Case-Based Reasoning (CBR) into LLM-based legal question answering to improve factual accuracy and context relevance. By using CBR’s indexing, similarity matching, and case reuse mechanisms to enrich LLM prompts with semantically similar past cases, the method achieves a 1.94% average improvement in answer similarity over baseline models, with Hybrid AnglEBERT at k=3 yielding the best performance.

ABSTRACT

Retrieval-Augmented Generation (RAG) enhances Large Language Model (LLM) output by providing prior knowledge as context to input. This is beneficial for knowledge-intensive and expert reliant tasks, including legal question-answering, which require evidence to validate generated text outputs. We highlight that Case-Based Reasoning (CBR) presents key opportunities to structure retrieval as part of the RAG process in an LLM. We introduce CBR-RAG, where CBR cycle's initial retrieval stage, its indexing vocabulary, and similarity knowledge containers are used to enhance LLM queries with contextually relevant cases. This integration augments the original LLM query, providing a richer prompt. We present an evaluation of CBR-RAG, and examine different representations (i.e. general and domain-specific embeddings) and methods of comparison (i.e. inter, intra and hybrid similarity) on the task of legal question-answering. Our results indicate that the context provided by CBR's case reuse enforces similarity between relevant components of the questions and the evidence base leading to significant improvements in the quality of generated answers.

Motivation & Objective

• To address the lack of factual grounding and provenance in LLM-generated legal answers by integrating structured case-based reasoning into retrieval-augmented generation.
• To evaluate how different embedding representations (general and domain-specific) and similarity comparison strategies (inter, intra, hybrid) affect retrieval quality in legal QA.
• To demonstrate that CBR-RAG enhances LLM output by providing richer, more contextually relevant prompts derived from semantically similar legal cases.
• To identify optimal configurations for embedding models and retrieval parameters in legal RAG systems, with a focus on improving answer fidelity and reducing hallucination.

Proposed method

• CBR-RAG integrates the CBR cycle’s retrieval, indexing, and similarity-matching stages into the RAG pipeline to structure and enrich the retrieval of legal cases.
• The system uses case representations based on embeddings from BERT, LegalBERT, and AnglEBERT to encode legal questions and cases for similarity computation.
• Three similarity strategies are evaluated: inter-similarity (between question and case), intra-similarity (within question or case components), and hybrid similarity combining both.
• k-Nearest Neighbors (k-NN) with k ∈ {1, 3, 5, ..., 37} is used for retrieval, with k=3 selected as optimal based on F1-score evaluation.
• Generated answers are evaluated via cosine similarity between LLM outputs and reference answers using the Mistral-7B model as the generator.
• Hybrid embeddings are constructed by combining multiple embedding types (e.g., BERT + LegalBERT + AnglEBERT) with learned weights to improve fine-grained similarity matching.

Figure 1: CBR-RAG

Experimental results

Research questions

• RQ1 How does integrating Case-Based Reasoning (CBR) into the retrieval stage of RAG improve the quality of LLM-generated answers in legal question-answering?
• RQ2 What is the impact of different embedding models (general vs. domain-specific) on retrieval performance in legal RAG systems?
• RQ3 Which similarity comparison strategy—inter, intra, or hybrid—yields the most effective case retrieval for legal QA tasks?
• RQ4 How does the number of retrieved cases (k) influence the factual accuracy and semantic quality of generated answers?
• RQ5 Can hybrid embeddings combining multiple models outperform individual embeddings in case retrieval and answer generation for legal LLMs?

Key findings

• The Hybrid AnglEBERT model with [0.25, 0.40, 0.35] weights and k=3 achieved the highest F1-score in retrieval, establishing it as the optimal configuration.
• CBR-RAG with k=3 and full case context in the prompt outperformed both k=1 and baseline No-RAG, achieving a 1.94% average increase in cosine similarity to ground truth answers.
• Hybrid AnglEBERT at k=3 significantly outperformed both the No-RAG baseline and hybrid LegalBERT at k=3 with p < 0.05 (one-tailed t-test), confirming statistical significance.
• Including the full case in the prompt led to better performance than using only the support text in most hybrid embedding variants.
• AnglEBERT’s contrastive pre-training approach yielded superior performance compared to BERT and LegalBERT, even without fine-tuning on the ALQA corpus.
• The results suggest that contrastive learning for embeddings is more effective than standard self-supervised masking, especially in low-resource legal domains.

Figure 2: Ten most frequent legal acts in the casebase are listed on the left, and the legal act frequency distribution appears on the right.