[Paper Review] AI-based Data Preparation and Data Analytics in Healthcare: The Case of Diabetes

Marianna Maranghi, Aris Anagnostopoulos+14 more|arXiv (Cornell University)|Jun 13, 2022

Artificial Intelligence in Healthcare4 citations

TL;DR

This paper presents an AI-driven framework for cleaning, modeling, and analyzing the large-scale AMD diabetes database (600,000 patients, 15 years) using ontology-based data preparation, machine learning, and NLP techniques. Key contributions include a domain-specific OWL ontology, robust data cleaning across heterogeneous sources, and predictive models for retinopathy, HbA1c trajectory clustering, and short-term cardiovascular risk prediction using PubMedBERT and temporal encoding, demonstrating the potential of AI to enhance clinical decision-making in diabetes care.

ABSTRACT

The Associazione Medici Diabetologi (AMD) collects and manages one of the largest worldwide-available collections of diabetic patient records, also known as the AMD database. This paper presents the initial results of an ongoing project whose focus is the application of Artificial Intelligence and Machine Learning techniques for conceptualizing, cleaning, and analyzing such an important and valuable dataset, with the goal of providing predictive insights to better support diabetologists in their diagnostic and therapeutic choices.

Motivation & Objective

• To address the challenge of inconsistent, heterogeneous, and large-scale real-world diabetes data from 320 Italian clinics.
• To develop a shared, standardized, and semantically consistent knowledge base from the AMD database using an OWL-based domain ontology.
• To enable advanced machine learning analytics by preparing high-quality, clean, and interoperable data for predictive modeling.
• To support clinicians with AI-driven insights into disease progression, treatment response, and risk prediction in type 2 diabetes.

Proposed method

• Developed a formal OWL ontology to model clinical concepts and relationships in diabetes care, ensuring semantic consistency across data sources.
• Designed a two-part database schema integrating patient data and metadata (e.g., value ranges, national codes) to support data integrity and interpretation.
• Performed extensive data cleaning across 300+ clinics, resolving inconsistencies, missing values, and semantic discrepancies due to varying EMR software versions and usage.
• Applied Gaussian Mixture Models and Temporal K-means to cluster patients based on 10-year HbA1c trajectories, capturing longitudinal glycemic control patterns.
• Formulated short-term cardiovascular risk prediction as a next-sentence prediction task using PubMedBERT, with temporal encoding of event timestamps and drug similarity via pharmacological principles.
• Replaced numerical lab values with categorical labels (e.g., normal, high) and encoded event sequences to predict the probability of future high-risk cardiovascular events within a 6-month window.

Experimental results

Research questions

• RQ1 How can a large, heterogeneous, real-world diabetes dataset from multiple clinics be standardized and semantically unified for advanced analytics?
• RQ2 Can AI-based data preparation techniques effectively resolve inconsistencies and missing data across 15 years of multi-center EMR records?
• RQ3 What patient subgroups can be identified based on longitudinal HbA1c trajectories, and how do they align with clinical expectations?
• RQ4 Can NLP-based models like PubMedBERT with temporal encoding improve short-term prediction of high-risk cardiovascular events in diabetic patients?
• RQ5 To what extent do AI-derived clusters of HbA1c trajectories correlate with clinical outcomes such as comorbidities or treatment regimens?

Key findings

• The development of a formal OWL ontology enabled consistent interpretation and integration of clinical data across 320 diabetes centers, forming a shared knowledge base.
• Extensive data cleaning resolved semantic discrepancies caused by varying EMR software versions and local data entry practices, significantly improving data quality.
• Preliminary HbA1c trajectory clustering using Gaussian Mixture Models and Temporal K-means revealed distinct patient subgroups, with early HbA1c values near diagnosis year showing meaningful clinical differentiation.
• The model for short-term cardiovascular risk prediction achieved improved performance by incorporating temporal context through age and event timestamp encoding, outperforming standard sequence modeling.
• The use of PubMedBERT with domain-specific event encoding and drug similarity mapping enabled effective sequence modeling for predicting future high-risk events within a 6-month window.
• The integration of longitudinal risk factor exposure scores into patient status vectors enhanced predictive modeling by capturing cumulative clinical risk over time.