[Paper Review] Med42-v2: A Suite of Clinical LLMs

Christophe Clément, Praveen K Kanithi+3 more|arXiv (Cornell University)|Aug 12, 2024

Health and Medical Research Impacts9 citations

TL;DR

Med42-v2 presents a suite of clinical LLMs based on Llama3, fine-tuned with medical data and multi-stage alignment, outperforming base Llama3 and GPT-4 on medical benchmarks.

ABSTRACT

Med42-v2 introduces a suite of clinical large language models (LLMs) designed to address the limitations of generic models in healthcare settings. These models are built on Llama3 architecture and fine-tuned using specialized clinical data. They underwent multi-stage preference alignment to effectively respond to natural prompts. While generic models are often preference-aligned to avoid answering clinical queries as a precaution, Med42-v2 is specifically trained to overcome this limitation, enabling its use in clinical settings. Med42-v2 models demonstrate superior performance compared to the original Llama3 models in both 8B and 70B parameter configurations and GPT-4 across various medical benchmarks. These LLMs are developed to understand clinical queries, perform reasoning tasks, and provide valuable assistance in clinical environments. The models are now publicly available at \href{https://huggingface.co/m42-health}{https://huggingface.co/m42-health}.

Motivation & Objective

• Address limitations of generic LLMs in healthcare by building clinically-aligned models.
• Fine-tune Llama3/3.1 with specialized medical data for clinical usefulness.
• Apply multi-stage preference alignment to meet real-world healthcare prompts.
• Demonstrate superior performance over base models and GPT-4 across medical benchmarks.

Proposed method

• Two-stage training: clinical instruction fine-tuning followed by preference alignment.
• Data: curated medical and biomedical data plus a 26.5% general-domain subset to improve linguistic/generalization.
• Training approach: auto-regressive loss with backpropagation on output tokens only, 8192-token chunks, system/assistant/user prompt format.
• Preference alignment using UltraFeedback and Snorkel-DPO datasets with iterative reweighting and Direct Preference Optimization (DPO).
• Hyperparameters provided for fine-tuning (8B/70B) and alignment stages; released models on HuggingFace.
• Evaluation with Eleuther AI evaluation harness across standard medical benchmarks.

Experimental results

Research questions

• RQ1 How do Med42-v2 models perform on standard medical benchmarks compared to base Llama3 and GPT-4?
• RQ2 Does the two-stage process (instruction fine-tuning + multi-stage preference alignment) improve clinical utility and safety in responses?
• RQ3 How do 8B and 70B parameter configurations compare in medical reasoning and factual accuracy?
• RQ4 What is the impact of medical-domain vs. general-domain data mix on real-world clinical query handling?

Key findings

• Med42-v2 outperforms the original Llama3 models at 8B and 70B and GPT-4 across multiple medical benchmarks.
• Larger models (70B) show stronger performance and can surpass GPT-4 on several datasets in zero-shot evaluation.
• Clinical instruction and alignment contribute to improved understanding and reasoning in medical contexts, with safer and more useful clinical responses.
• Alignment via UltraFeedback and Snorkel-DPO enable scalable, stable tuning without full reinforcement learning loops.
• The results are based on zero-shot evaluations; real-world clinical utility needs further evaluation.