[Paper Review] Mobile Artificial Intelligence Technology for Detecting Macula Edema and Subretinal Fluid on OCT Scans: Initial Results from the DATUM alpha Study

Stephen G. Odaibo, Mikelson MomPremier+4 more|arXiv (Cornell University)|Feb 8, 2019

Retinal Imaging and AnalysisMedicine27 references3 citations

TL;DR

This study evaluates Fluid Intelligence, a cloud-based mobile AI app that detects macula edema and subretinal fluid on OCT scans by analyzing smartphone-captured images of OCT displays. In a multicenter retrospective analysis of 283 OCT scans, the AI achieved a weighted average sensitivity of 89.3% and specificity of 81.23%, demonstrating feasibility as a screening tool for underserved ophthalmic care settings.

ABSTRACT

Artificial Intelligence (AI) is necessary to address the large and growing deficit in retina and healthcare access globally. And mobile AI diagnostic platforms running in the Cloud may effectively and efficiently distribute such AI capability. Here we sought to evaluate the feasibility of Cloud-based mobile artificial intelligence for detection of retinal disease. And to evaluate the accuracy of a particular such system for detection of subretinal fluid (SRF) and macula edema (ME) on OCT scans. A multicenter retrospective image analysis was conducted in which board-certified ophthalmologists with fellowship training in retina evaluated OCT images of the macula. They noted the presence or absence of ME or SRF, then compared their assessment to that obtained from Fluid Intelligence, a mobile AI app that detects SRF and ME on OCT scans. Investigators consecutively selected retinal OCTs, while making effort to balance the number of scans with retinal fluid and scans without. Exclusion criteria included poor scan quality, ambiguous features, macula holes, retinoschisis, and dense epiretinal membranes. Accuracy in the form of sensitivity and specificity of the AI mobile App was determined by comparing its assessments to those of the retina specialists. At the time of this submission, five centers have completed their initial studies. This consists of a total of 283 OCT scans of which 155 had either ME or SRF ("wet") and 128 did not ("dry"). The sensitivity ranged from 82.5% to 97% with a weighted average of 89.3%. The specificity ranged from 52% to 100% with a weighted average of 81.23%. CONCLUSION: Cloud-based Mobile AI technology is feasible for the detection retinal disease. In particular, Fluid Intelligence (alpha version), is sufficiently accurate as a screening tool for SRF and ME, especially in underserved areas. Further studies and technology development is needed.

Motivation & Objective

• Address the global shortage of retina specialists, particularly in rural and underserved regions.
• Evaluate the feasibility of distributing AI diagnostic capability through mobile devices and cloud computing.
• Assess the accuracy of a mobile AI app (Fluid Intelligence) in detecting subretinal fluid and macula edema on OCT scans compared to expert ophthalmologists.
• Determine whether smartphone-captured images of OCT displays can yield sufficient diagnostic accuracy for clinical screening.
• Explore the potential of mobile AI as a scalable, low-cost solution to improve early detection of retinal pathology.

Proposed method

• Developed a mobile AI application (Fluid Intelligence) for iOS that runs inference on cloud-hosted machine learning models.
• Users capture a smartphone image of an OCT scan displayed on a monitor or printed page, which is uploaded to a cloud-based AI engine.
• The AI model processes the image to detect the presence or absence of macula edema and subretinal fluid using deep learning techniques.
• A noSQL database stored images sent to the cloud for inference and audit purposes.
• Trained the model on a diverse dataset of OCT scans, with ongoing updates enabled through cloud-based retraining.
• Conducted a multicenter retrospective analysis with board-certified retina specialists serving as the gold standard for comparison.

Experimental results

Research questions

• RQ1 Can a mobile AI app using smartphone-captured images of OCT scans achieve clinically acceptable sensitivity and specificity for detecting macula edema and subretinal fluid?
• RQ2 Is the diagnostic accuracy of such a mobile AI system consistent across diverse clinical settings and reader expertise levels?
• RQ3 What is the impact of image quality, resolution, and artifacts (e.g., glare, blur) on AI performance in real-world clinical image capture?
• RQ4 Can cloud-hosted AI models be effectively updated and improved over time using new data without requiring local software updates?
• RQ5 To what extent can mobile AI reduce unnecessary referrals and missed diagnoses in primary and community eye care settings?

Key findings

• The Fluid Intelligence AI app achieved a weighted average sensitivity of 89.3% across five centers, indicating strong ability to detect true positive cases of macula edema or subretinal fluid.
• The weighted average specificity was 81.23%, with a wide range from 52% to 100% across centers, suggesting variability in false positive detection.
• Sensitivity ranged from 82.5% to 97% across centers, indicating consistent performance in identifying pathological cases.
• False positives were more common in images with features like large confluent drusen or pigment epithelial detachments, which may be trainable out with expanded datasets.
• Image quality issues such as low resolution or glare contributed to false positives, highlighting the need for improved input image standards.
• The study confirms that mobile AI via smartphone-captured images of OCT scans is a feasible and scalable method for distributing diagnostic capability in underserved areas.