[Paper Review] OralCam: Enabling Self-Examination and Awareness of Oral Health Using a Smartphone Camera

Yuan Liang, Hsuan-Wei Fan+8 more|arXiv (Cornell University)|Jan 16, 2020

Dental Research and COVID-1952 references5 citations

TL;DR

OralCam is a smartphone-based mobile health application that enables lay users to self-examine five common oral conditions—periodontal disease, caries, soft deposit, dental calculus, and discoloration—using smartphone-captured intraoral images. It combines deep learning with user-provided contextual inputs (e.g., pain, habits) to deliver hierarchical, probabilistic, and explainable results, achieving a mean detection sensitivity of 0.787 on a dataset of 3,182 expert-annotated images.

ABSTRACT

Due to a lack of medical resources or oral health awareness, oral diseases are often left unexamined and untreated, affecting a large population worldwide. With the advent of low-cost, sensor-equipped smartphones, mobile apps offer a promising possibility for promoting oral health. However, to the best of our knowledge, no mobile health (mHealth) solutions can directly support a user to self-examine their oral health condition. This paper presents OralCam, the first interactive app that enables end-users' self-examination of five common oral conditions (diseases or early disease signals) by taking smartphone photos of one's oral cavity. OralCam allows a user to annotate additional information (e.g. living habits, pain, and bleeding) to augment the input image, and presents the output hierarchically, probabilistically and with visual explanations to help a laymen user understand examination results. Developed on our in-house dataset that consists of 3,182 oral photos annotated by dental experts, our deep learning based framework achieved an average detection sensitivity of 0.787 over five conditions with high localization accuracy. In a week-long in-the-wild user study (N=18), most participants had no trouble using OralCam and interpreting the examination results. Two expert interviews further validate the feasibility of OralCam for promoting users' awareness of oral health.

Motivation & Objective

• Address the global gap in oral health awareness and access to care by enabling self-examination using widely available smartphone technology.
• Overcome the lack of mHealth tools that support direct self-assessment of oral conditions beyond clinical settings.
• Design a system that provides understandable, trustworthy, and actionable feedback to non-expert users.
• Integrate multimodal inputs (images, user-reported symptoms, annotations) to improve detection accuracy and contextual relevance.
• Develop a deep learning framework that localizes and classifies oral conditions with visual explainability for end users.

Proposed method

• Built an in-house dataset of 3,182 intraoral images annotated by dental experts for five common oral conditions.
• Implemented a deep learning model trained on the annotated dataset to detect and localize oral conditions in smartphone-captured images.
• Extended the input modality by allowing users to annotate images and complete a survey on symptoms (e.g., pain, bleeding, habits), which serve as model priors.
• Designed a hierarchical, probabilistic output interface that presents results with visual localization and confidence scores to enhance user comprehension.
• Incorporated heatmap visualizations to show model attention regions, improving transparency and user trust in predictions.
• Conducted a week-long in-the-wild user study (N=18) and expert interviews to evaluate usability, interpretability, and clinical feasibility.

Experimental results

Research questions

• RQ1 Can a smartphone-based mobile app enable non-expert users to self-examine common oral conditions with acceptable accuracy and usability?
• RQ2 How does incorporating user-reported contextual information (e.g., pain, habits) improve the performance of deep learning models in oral condition detection?
• RQ3 To what extent can visual explainability (e.g., heatmaps, localization) enhance user trust and understanding of AI-generated oral health assessments?
• RQ4 What are the key technical and usability challenges in deploying AI-based oral self-examination in real-world settings?
• RQ5 How do dental experts perceive the clinical feasibility and reliability of AI-driven self-examination tools like OralCam?

Key findings

• The deep learning model achieved an average detection sensitivity of 0.787 across five oral conditions on the in-house dataset of 3,182 expert-annotated images.
• In a week-long in-the-wild user study with 18 participants, most users reported no difficulty in using OralCam or interpreting the results.
• Dental experts rated the system’s detection performance highly, with average scores of 4.2 and 4.8 out of 5, indicating strong clinical acceptability.
• The model’s performance was significantly affected by image quality—poor focus and improper lighting led to false positives and missed detections.
• Experts recommended capturing multiple views and improving lighting to reduce errors, especially for shadowed or poorly lit regions.
• Users expressed a need for explanation beyond localization, such as reasoning (e.g., 'flagged due to a black notch'), suggesting a need for explainable AI in future iterations.