[Paper Review] Dr. Assistant: Enhancing Clinical Diagnostic Inquiry via Structured Diagnostic Reasoning Data and Reinforcement Learning

Yue Guo, Fanfu Wang+10 more|arXiv (Cornell University)|Jan 20, 2026

Clinical Reasoning and Diagnostic Skills0 citations

TL;DR

Dr. Assistant introduces a Clinical Diagnostic Reasoning Data (CDRD) format and a two-stage SFT+RL training pipeline to improve clinical diagnostic reasoning and multi-turn inquiry, achieving strong performance vs open-source models and competitive results with GPT-5.

ABSTRACT

Clinical Decision Support Systems (CDSSs) provide reasoning and inquiry guidance for physicians, yet they face notable challenges, including high maintenance costs and low generalization capability. Recently, Large Language Models (LLMs) have been widely adopted in healthcare due to their extensive knowledge reserves, retrieval, and communication capabilities. While LLMs show promise and excel at medical benchmarks, their diagnostic reasoning and inquiry skills are constrained. To mitigate this issue, we propose (1) Clinical Diagnostic Reasoning Data (CDRD) structure to capture abstract clinical reasoning logic, and a pipeline for its construction, and (2) the Dr. Assistant, a clinical diagnostic model equipped with clinical reasoning and inquiry skills. Its training involves a two-stage process: SFT, followed by RL with a tailored reward function. We also introduce a benchmark to evaluate both diagnostic reasoning and inquiry. Our experiments demonstrate that the Dr. Assistant outperforms open-source models and achieves competitive performance to closed-source models, providing an effective solution for clinical diagnostic inquiry guidance.

Motivation & Objective

• Capture abstract clinical diagnostic reasoning logic via a structured data format (CDRD).
• Develop a two-stage training pipeline (SFT then RL) to build diagnostic reasoning and inquiry skills.
• Create a benchmark for evaluating diagnostic reasoning and clinical inquiry in CDSS contexts.
• Demonstrate that Dr. Assistant outperforms open-source models and rivals closed-source models on the benchmark.

Proposed method

• Define CDRD as a triple-tuple (S, E, D) representing core symptom, diagnostic evidence, and differential diagnoses.
• Construct a three-stage pipeline to build CDRD from clinical guidelines with LLM synthesis and physician refinement.
• Generate QA data for supervised fine-tuning (SFT) from CDRD through answer formulation and question synthesis.
• Create multi-turn inquiry data for RL via a dual-agent (physician and patient) dialogue grounded in CDRD and patient profiles.
• Train Dr. Assistant with SFT on P_SFT, followed by RL using a composite reward for clinical reasoning, inquiry, and fidelity to CDRD (R_div and R_comp).
• Evaluate using a dedicated ICD-10-based reasoning benchmark with 242 patient profiles and 147 inquiry rounds.

Experimental results

Research questions

• RQ1 Can a structured diagnostic reasoning data format (CDRD) improve abstract reasoning alignment with clinical guidelines?
• RQ2 Does a two-stage SFT+RL training regime enhance diagnostic inquiry quality and reasoning fidelity compared to baseline models?
• RQ3 How does Dr. Assistant perform on dynamic diagnostic reasoning and inquiry benchmarks relative to open-source and closed-source LLMs?
• RQ4 What is the impact of fidelity penalties to CDRD on output quality and safety of clinical inferences?

Key findings

• Dr. Assistant achieves the highest ICD-Recall among the evaluated models, outperforming several open-source medical models and approaching closed-source GPT-5 performance.
• In ICD-Recall benchmarking, Dr. Assistant improves by about 13.59% relative over HuatuoGPT-o1-72B.
• Dr. Assistant shows competitive or superior results to several open-source models and approaches GPT-5 on the ICD-Recall metric.
• Physician evaluators report higher satisfaction with Dr. Assistant’s inquiries compared with open-source baselines, with notable improvements over Med42-v2-8B and HuatuoGPT-o1-72B.
• Ablation studies indicate both SFT and RL contribute substantially, with RL providing the largest relative performance gains; the R_div fidelity term also meaningfully improves output quality.