[Paper Review] I Ate This: A Photo-based Food Journaling System with Expert Feedback

Sanjeev B. Goyal, Qi Liu+4 more|arXiv (Cornell University)|Feb 20, 2017

Nutritional Studies and Diet6 references3 citations

TL;DR

I Ate This is a smartphone-based food journaling system that uses photo capture and expert dietitian feedback to improve dietary awareness and behavior change. Users submit meal photos with minimal input, which dietitians evaluate for nutritional quality, portion size, and health impact, providing personalized feedback to support long-term healthy eating habits.

ABSTRACT

What we eat is one of the most frequent and important health decisions we make in daily life, yet it remains notoriously difficult to capture and understand. Effective food journaling is thus a grand challenge in personal health informatics. In this paper we describe a system for food journaling called I Ate This, which is inspired by the Remote Food Photography Method (RFPM). I Ate This is simple: you use a smartphone app to take a photo and give a very basic description of any food or beverage you are about to consume. Later, a qualified dietitian will evaluate your photo, giving you feedback on how you did and where you can improve. The aim of I Ate This is to provide a convenient, visual and reliable way to help users learn from their eating habits and nudge them towards better choices each and every day. Ultimately, this incremental approach can lead to long-term behaviour change. Our goal is to bring RFPM to a wider audience, through APIs that can be incorporated into other apps.

Motivation & Objective

• To address the low adherence and high burden of traditional food journaling methods in personal health informatics.
• To develop a lightweight, visual food logging system that reduces user effort while improving accuracy compared to manual calorie tracking.
• To integrate expert dietitian feedback into a scalable mobile system to enhance user engagement and behavior change.
• To explore the integration of continuous glucose monitoring (CGM) data with food photos for real-time blood glucose impact feedback.
• To enable broader adoption of the Remote Food Photography Method (RFPM) through API-based integration into third-party apps.

Proposed method

• Users capture food photos via a smartphone app and provide a minimal text description, with optional inputs for portion size, time, location, and self-rating on a 5-point health scale.
• Dietitians evaluate photos through a back-office interface that supports three core steps: matching the food to a nutrition database, tagging for nutritional quality (e.g., good vs. poor carb), and scoring the meal on a 5-point health scale.
• The system supports two feedback layers: quantitative (aggregated nutritional data for alerts on calories, sugar, salt) and qualitative (personalized messages and visual markers from dietitians).
• Integration with Continuous Glucose Monitoring (CGM) devices enables visual feedback—color-coding photos as green (in-range glucose) or red (out-of-range) based on postprandial glucose response.
• The system is designed with modular APIs to allow integration into other health and wellness applications.
• A database of restaurant meals is being developed to allow menu-based logging without photo capture, enhancing usability for dining-out scenarios.

Experimental results

Research questions

• RQ1 Can a photo-based food journaling system with expert feedback improve user adherence and accuracy compared to traditional manual logging?
• RQ2 How effective is expert dietitian feedback in supporting long-term dietary behavior change in a remote, scalable format?
• RQ3 To what extent can CGM-integrated feedback enhance self-management of blood glucose in individuals with diabetes?
• RQ4 Can automated tagging and scoring systems based on dietitian evaluations be used to train future AI models for food recognition and nutrition assessment?
• RQ5 What role does visual, personalized feedback play in increasing user engagement and motivation in self-monitoring of dietary intake?

Key findings

• The I Ate This system reduces the burden of food logging by replacing manual data entry with photo capture and minimal user input.
• Expert dietitian feedback significantly improves the accuracy and reliability of food intake assessment compared to self-reported or automated systems.
• Integration with CGM devices enables real-time, visual feedback on postprandial glucose responses, helping users identify food impacts on blood sugar.
• The system supports scalable, remote nutrition coaching, demonstrating potential for wider deployment in chronic disease management, especially diabetes.
• User self-rating and expert scoring show strong alignment, indicating that the 5-point health scale is a reliable and interpretable metric for dietary quality.
• The system’s API-first design enables integration into third-party apps, expanding its reach beyond standalone use.