[Paper Review] NUTRIVISION: A System for Automatic Diet Management in Smart Healthcare

Madhumita Veeramreddy, Ashok Kumar Pradhan+3 more|arXiv (Cornell University)|Sep 30, 2024

Mobile Health and mHealth ApplicationsHealth Professions3 citations

TL;DR

NutriVision is a smart healthcare system that leverages Faster R-CNN-based computer vision and machine learning to automatically detect food items from smartphone images, estimate portion sizes, and deliver personalized nutritional analysis and diet recommendations. It outperforms YOLO, Mask R-CNN, and conventional CNNs in food detection accuracy, achieving superior F1 scores, precision, and recall, while integrating user-specific health data for tailored dietary guidance.

ABSTRACT

Maintaining health and fitness through a balanced diet is essential for preventing non communicable diseases such as heart disease, diabetes, and cancer. NutriVision combines smart healthcare with computer vision and machine learning to address the challenges of nutrition and dietary management. This paper introduces a novel system that can identify food items, estimate quantities, and provide comprehensive nutritional information. NutriVision employs the Faster Region based Convolutional Neural Network, a deep learning algorithm that improves object detection by generating region proposals and then classifying those regions, making it highly effective for accurate and fast food identification even in complex and disorganized meal settings. Through smartphone based image capture, NutriVision delivers instant nutritional data, including macronutrient breakdown, calorie count, and micronutrient details. One of the standout features of NutriVision is its personalized nutritional analysis and diet recommendations, which are tailored to each user's dietary preferences, nutritional needs, and health history. By providing customized advice, NutriVision helps users achieve specific health and fitness goals, such as managing dietary restrictions or controlling weight. In addition to offering precise food detection and nutritional assessment, NutriVision supports smarter dietary decisions by integrating user data with recommendations that promote a balanced, healthful diet. This system presents a practical and advanced solution for nutrition management and has the potential to significantly influence how people approach their dietary choices, promoting healthier eating habits and overall well being. This paper discusses the design, performance evaluation, and prospective applications of the NutriVision system.

Motivation & Objective

• To address the limitations of manual food logging in existing diet tracking systems by enabling fully automated, image-based nutritional assessment.
• To improve the accuracy and reliability of food detection and portion estimation in complex, real-world meal settings.
• To deliver personalized dietary recommendations based on individual health profiles, including BMI, dietary preferences, and medical conditions.
• To reduce user burden and increase adherence by eliminating manual data entry and providing real-time, actionable feedback.
• To enhance nutritional decision-making through integration of user feedback and dynamic adaptation of recommendations.

Proposed method

• The system employs Faster R-CNN, a deep learning object detection model, to identify and localize food items in user-captured images with high precision.
• Food volume is estimated from image features and converted into weight using learned regression models to enable portion quantification.
• Nutritional information—macronutrients, calories, and micronutrients—is automatically extracted using a comprehensive food database linked to detected items.
• Personalized recommendations are generated by analyzing user-specific data, including BMI, health history, and dietary preferences, using a feedback-driven adaptation engine.
• The system incorporates a user feedback loop that dynamically refines future recommendations based on user responses to meal suggestions.
• Evaluation is conducted using standard object detection metrics (precision, recall, F1-score) across multiple training epochs to benchmark performance against YOLO, Mask R-CNN, and conventional CNNs.

Experimental results

Research questions

• RQ1 How does Faster R-CNN compare to other deep learning models (YOLO, Mask R-CNN, conventional CNNs) in detecting food items from real-world meal images?
• RQ2 To what extent can the system accurately estimate food portion sizes from single smartphone images without external references?
• RQ3 How effective is the integration of user health data (e.g., BMI, dietary preferences) in generating personalized and actionable dietary recommendations?
• RQ4 Can a feedback-driven recommendation engine improve the relevance and user adherence to dietary suggestions over time?
• RQ5 What is the impact of automated nutritional analysis on user engagement and long-term dietary behavior change?

Key findings

• Faster R-CNN achieved the highest F1 score across all evaluation metrics, demonstrating superior balance between precision and recall in food detection.
• The F1-score vs. epochs graph confirmed that Faster R-CNN consistently outperformed YOLO, Mask R-CNN, and conventional CNNs in convergence and accuracy.
• Faster R-CNN showed the highest precision and recall values over time, indicating reliable and consistent detection performance in complex meal scenarios.
• The system demonstrated significant improvement in detection accuracy compared to baseline models, particularly in disorganized and cluttered meal settings.
• Personalized recommendations based on user health profiles and feedback were shown to enhance user engagement and adaptability of the system.
• The system successfully reduced reliance on manual food logging and enabled real-time nutritional feedback, supporting healthier dietary choices.