[Paper Review] Hengqin-RA-v1: Advanced Large Language Model for Diagnosis and Treatment of Rheumatoid Arthritis with Dataset based Traditional Chinese Medicine

Yishen Liu, Shengda Luo+7 more|arXiv (Cornell University)|Jan 5, 2025

Traditional Chinese Medicine StudiesMedicine3 citations

TL;DR

Hengqin-RA-v1 is the first large language model specifically fine-tuned for Traditional Chinese Medicine (TCM)-based diagnosis and treatment of rheumatoid arthritis (RA), leveraging the novel HQ-GCM-RA-C1 dataset derived from classical Chinese medical texts and modern clinical studies. It achieves a 54% pass rate on the Chinese TCM examination—significantly outperforming other LLMs, including augmented versions of GPT-3.5 and Huatuo-2-7B—demonstrating superior accuracy and domain-specific expertise in TCM for RA.

ABSTRACT

Large language models (LLMs) primarily trained on English texts, often face biases and inaccuracies in Chinese contexts. Their limitations are pronounced in fields like Traditional Chinese Medicine (TCM), where cultural and clinical subtleties are vital, further hindered by a lack of domain-specific data, such as rheumatoid arthritis (RA). To address these issues, this paper introduces Hengqin-RA-v1, the first large language model specifically tailored for TCM with a focus on diagnosing and treating RA. We also present HQ-GCM-RA-C1, a comprehensive RA-specific dataset curated from ancient Chinese medical literature, classical texts, and modern clinical studies. This dataset empowers Hengqin-RA-v1 to deliver accurate and culturally informed responses, effectively bridging the gaps left by general-purpose models. Extensive experiments demonstrate that Hengqin-RA-v1 outperforms state-of-the-art models, even surpassing the diagnostic accuracy of TCM practitioners in certain cases.

Motivation & Objective

• To address the lack of domain-specific, high-quality Chinese medical corpora for rheumatoid arthritis (RA) in Traditional Chinese Medicine (TCM).
• To overcome the limitations of general-purpose large language models (LLMs) in handling culturally nuanced, clinically precise TCM diagnostics and treatment recommendations.
• To develop a specialized LLM, Hengqin-RA-v1, that matches or exceeds human expert performance in TCM-based RA diagnosis and treatment planning.
• To create and release HQ-GCM-RA-C1, the first comprehensive, curated Chinese corpus for RA in TCM, including classical texts, exam questions, and clinical data.
• To establish a benchmark for evaluating LLMs in TCM-specific medical reasoning, particularly in classical Chinese and complex diagnostic logic.

Proposed method

• The model is trained using a progressive pipeline: starting from the HQ-GCM-RA-C1 dataset, followed by data segmentation, conversation set generation, full fine-tuning, and LoRA adaptation.
• Incremental fine-tuning is applied to selectively update parameters while preserving prior knowledge, enabling continual adaptation to new data.
• Instance-oriented and entity-relationship-oriented retrieval enhancements are integrated to improve contextual relevance and diagnostic reasoning.
• TCM diagnostic logic is refined using structured medical records and a sliding window mechanism to maintain coherence and accuracy in long-context reasoning.
• The model is evaluated using a standardized Chinese TCM examination, with performance measured as the ratio of correct answers to total questions (accuracy).
• Data augmentation and domain-specific pre-training are applied to enhance model generalization and reduce bias in Chinese medical language understanding.

Figure 1: The progressive training workflow of Hengqin-RA-v1 starts with HQ-GCM-RA-C1 , followed by Data Segmentation Conversation Set Generation, Full Fine-Tuning, and LoRA. Incremental Fine-Tuning then adjusts select parameters while preserving knowledge, branching into Instance-Oriented and Entit

Experimental results

Research questions

• RQ1 Can a large language model fine-tuned on a curated, TCM-specific dataset for rheumatoid arthritis achieve superior diagnostic and treatment recommendation performance compared to general-purpose LLMs?
• RQ2 To what extent does the inclusion of classical Chinese medical texts and modern clinical data in the training corpus improve the model's accuracy and cultural contextual understanding in TCM?
• RQ3 How does Hengqin-RA-v1 compare to human TCM practitioners and other state-of-the-art LLMs in diagnosing and treating RA using TCM principles?
• RQ4 What is the impact of data augmentation and specialized fine-tuning techniques on the model's ability to pass standardized TCM examinations?
• RQ5 Can a domain-specific LLM trained on a comprehensive, multi-source Chinese corpus reduce language and cultural biases present in general-purpose models?

Key findings

• Hengqin-RA-v1 achieved a 54% pass rate on the Chinese TCM examination, significantly outperforming other LLMs, including GPT-3.5 (+) at 37% and Huatuo-2-7B (*) at 37%.
• The model surpassed all evaluated Chinese and non-Chinese LLMs, including data-augmented variants, demonstrating the effectiveness of its specialized training and domain-specific data.
• Hengqin-RA-v1 demonstrated diagnostic accuracy exceeding that of some human TCM practitioners in specific test cases, indicating high reliability in clinical reasoning.
• The HQ-GCM-RA-C1 dataset, which includes classical texts, exam questions, and modern clinical studies, was instrumental in enabling the model’s high performance and contextual accuracy.
• The study reveals that language complexity, including classical Chinese, homophones, and polysemy, remains a major challenge for LLMs, even when trained on domain-specific corpora.
• The results underscore the critical role of curated, high-quality, and culturally grounded datasets in advancing specialized LLMs for traditional medicine and reducing bias in medical AI.

Figure 2: Enhancing TCM diagnostic and treatment logic involves a data processing pipeline starting with Raw Data as the initial input. The data is segmented (Raw Data- $1$ to Raw Data- $N$ ) and combined with a System Prompt at the Input stage. This input undergoes task-specific processing in the T