[Paper Review] FIT: a Fast and Accurate Framework for Solving Medical Inquiring and Diagnosing Tasks.

Weijie He, Xiaohao Mao+3 more|arXiv (Cornell University)|Dec 2, 2020

Machine Learning in Healthcare37 references4 citations

TL;DR

FIT is a fast, accurate reinforcement learning framework for medical self-diagnosis that uses an information-theoretic reward and a multimodal variational autoencoder with a two-step sampling strategy to efficiently select symptoms and predict diseases. It reduces computational cost significantly while outperforming existing RL baselines in simulated and real-world medical datasets.

ABSTRACT

Automatic self-diagnosis provides low-cost and accessible healthcare via an agent that queries the patient and makes predictions about possible diseases. From a machine learning perspective, symptom-based self-diagnosis can be viewed as a sequential feature selection and classification problem. Reinforcement learning methods have shown good performance in this task but often suffer from large search spaces and costly training. To address these problems, we propose a competitive bipartite framework, called FIT, which uses an information-theoretic reward to determine what data to collect next. FIT improves over previous information-based approaches by using a multimodal variational autoencoder (MVAE) model and a two-step sampling strategy for disease prediction. Furthermore, we propose novel methods to substantially reduce the computational cost of FIT to a level that is acceptable for practical online self-diagnosis. Our results in two simulated datasets show that FIT can effectively deal with large search space problems, outperforming existing RL baselines. Moreover, using several public medical datasets, we show that FIT is a competitive alternative in various real-world settings.

Motivation & Objective

• To address the high computational cost and large search space challenges in reinforcement learning-based medical self-diagnosis.
• To improve accuracy and efficiency in symptom selection and disease prediction using information-theoretic principles.
• To reduce training and inference costs to enable practical deployment in online self-diagnosis systems.
• To develop a scalable framework that performs well across diverse medical datasets and real-world scenarios.

Proposed method

• FIT employs a competitive bipartite framework that models symptom selection as a sequential decision-making process using reinforcement learning.
• It uses an information-theoretic reward function to guide the agent in selecting the most informative symptoms at each step.
• A multimodal variational autoencoder (MVAE) is used to model the joint distribution of symptoms and diseases, capturing complex dependencies.
• A two-step sampling strategy improves disease prediction by refining uncertainty estimates and focusing on high-impact symptoms.
• Novel computational optimization techniques are introduced to drastically reduce inference time and memory usage.
• The framework is trained and evaluated on both simulated datasets and public medical datasets to ensure robustness and generalization.

Experimental results

Research questions

• RQ1 Can a reinforcement learning-based framework achieve high diagnostic accuracy while maintaining low computational cost in medical self-diagnosis?
• RQ2 How does the use of an information-theoretic reward improve symptom selection efficiency compared to standard RL approaches?
• RQ3 To what extent does the MVAE-based modeling of symptom-disease relationships enhance prediction performance?
• RQ4 Can the proposed two-step sampling strategy reduce uncertainty and improve diagnostic confidence?
• RQ5 How does FIT perform in comparison to existing RL baselines across diverse real-world medical datasets?

Key findings

• FIT outperforms existing reinforcement learning baselines in terms of diagnostic accuracy on two simulated datasets with large search spaces.
• The framework achieves significant reductions in computational cost, making it suitable for real-time online self-diagnosis applications.
• The use of a multimodal variational autoencoder improves modeling of symptom-disease relationships, leading to more accurate predictions.
• The information-theoretic reward mechanism enables efficient and targeted symptom selection, reducing the number of queries needed for diagnosis.
• FIT demonstrates strong generalization performance across multiple public medical datasets, confirming its competitiveness in real-world settings.
• The two-step sampling strategy enhances prediction reliability by refining uncertainty estimates during the diagnostic process.