[Paper Review] Machine learning based disease diagnosis: A comprehensive review

Md Manjurul Ahsan, Zahed Siddique|arXiv (Cornell University)|Dec 31, 2021

COVID-19 diagnosis using AI4 citations

TL;DR

This comprehensive review analyzes machine learning (ML) and deep learning (DL) applications in disease diagnosis from 2012 to 2021, identifying key trends, algorithms, and data types. It finds that convolutional neural networks (CNNs) are the most widely used and effective method, achieving over 90% accuracy in disease detection, with major focus on cancer, diabetes, heart disease, and COVID-19.

ABSTRACT

Globally, there is a substantial unmet need to diagnose various diseases effectively. The complexity of the different disease mechanisms and underlying symptoms of the patient population presents massive challenges to developing the early diagnosis tool and effective treatment. Machine Learning (ML), an area of Artificial Intelligence (AI), enables researchers, physicians, and patients to solve some of these issues. Based on relevant research, this review explains how Machine Learning (ML) and Deep Learning (DL) are being used to help in the early identification of numerous diseases. To begin, a bibliometric study of the publication is given using data from the Scopus and Web of Science (WOS) databases. The bibliometric study of 1216 publications was undertaken to determine the most prolific authors, nations, organizations, and most cited articles. The review then summarizes the most recent trends and approaches in Machine Learning-based Disease Diagnosis (MLBDD), considering the following factors: algorithm, disease types, data type, application, and evaluation metrics. Finally, the paper highlights key results and provides insight into future trends and opportunities in the MLBDD area.

Motivation & Objective

• To identify the most prevalent diseases targeted by ML-based diagnosis research between 2012 and 2021.
• To analyze the most frequently used machine learning and deep learning algorithms in medical diagnosis.
• To evaluate the performance of ML models using standard evaluation metrics across diverse disease types.
• To highlight critical challenges such as data imbalance, model interpretability, and ethical concerns in clinical ML applications.
• To guide future research by identifying gaps in model explainability, fairness, and robustness in real-world healthcare deployment.

Proposed method

• Conducted a bibliometric analysis of 1,216 publications from Scopus and Web of Science to map trends in authorship, country of origin, institutions, and citation impact.
• Systematically reviewed ML and DL techniques applied to disease diagnosis, focusing on algorithm type, disease category, data modality (e.g., imaging, tabular), and application context.
• Classified and compared performance metrics such as accuracy, sensitivity, specificity, and AUC-ROC across studies to assess diagnostic reliability.
• Evaluated the role of deep learning architectures—particularly Convolutional Neural Networks (CNNs)—in image-based diagnosis (e.g., X-ray, MRI).
• Assessed challenges including data imbalance, lack of model interpretability, and ethical concerns such as bias and accountability in automated diagnosis.
• Proposed future research directions, including explainable AI (XAI), handling multiclass and imbalanced datasets, and secure cloud-based deployment for scalable healthcare systems.

Experimental results

Research questions

• RQ1 Which diseases are most frequently studied in ML-based disease diagnosis research between 2012 and 2021?
• RQ2 Which machine learning and deep learning algorithms are most commonly used in disease diagnosis, and what explains their popularity?
• RQ3 How do evaluation metrics such as accuracy, AUC-ROC, and F1-score vary across different disease types and data modalities?
• RQ4 What are the key challenges—such as data imbalance, model interpretability, and fairness—that hinder the clinical adoption of ML-based diagnostic systems?
• RQ5 What future research directions are most critical for advancing reliable, ethical, and scalable ML-based disease diagnosis systems?

Key findings

• Convolutional Neural Networks (CNNs) are the most widely used and effective deep learning architecture in ML-based disease diagnosis, particularly for medical imaging applications.
• ML-based diagnostic models achieve over 90% accuracy in detecting diseases such as breast cancer, Alzheimer’s, heart disease, and pneumonia, especially when using imaging data.
• The most studied diseases include breast cancer, diabetes, heart disease, kidney disease, Alzheimer’s, Parkinson’s, and more recently, COVID-19 and pneumonia due to the pandemic.
• Despite widespread use, most studies fail to address data imbalance issues, even though such imbalance is common in real-world clinical datasets.
• Model interpretability remains a major gap—few studies incorporate explainable AI (XAI) techniques, despite growing demand for transparent and trustworthy diagnostic systems.
• Future research should prioritize multiclass classification with highly imbalanced and missing data, integration of XAI, and secure, cloud-based deployment to support scalable and ethical ML in healthcare.