[Paper Review] Food Recipe Recommendation Based on Ingredients Detection Using Deep Learning

Md. Shafaat Jamil Rokon, Md. Kishor Morol+3 more|arXiv (Cornell University)|Mar 13, 2022

Advanced Chemical Sensor Technologies4 citations

TL;DR

This paper proposes a deep learning framework for food recipe recommendation by detecting ingredients from images using a Convolutional Neural Network (CNN) and recommending recipes via machine learning. Trained on a custom dataset of 9,856 images across 32 ingredient classes, the system achieves 94% accuracy in ingredient recognition, enabling effective recipe suggestions for both novice and experienced cooks.

ABSTRACT

Food is essential for human survival, and people always try to taste different types of delicious recipes. Frequently, people choose food ingredients without even knowing their names or pick up some food ingredients that are not obvious to them from a grocery store. Knowing which ingredients can be mixed to make a delicious food recipe is essential. Selecting the right recipe by choosing a list of ingredients is very difficult for a beginner cook. However, it can be a problem even for experts. One such example is recognising objects through image processing. Although this process is complex due to different food ingredients, traditional approaches will lead to an inaccuracy rate. These problems can be solved by machine learning and deep learning approaches. In this paper, we implemented a model for food ingredients recognition and designed an algorithm for recommending recipes based on recognised ingredients. We made a custom dataset consisting of 9856 images belonging to 32 different food ingredients classes. Convolution Neural Network (CNN) model was used to identify food ingredients, and for recipe recommendations, we have used machine learning. We achieved an accuracy of 94 percent, which is quite impressive.

Motivation & Objective

• To address the challenge of recommending suitable recipes when users have a set of ingredients but lack knowledge of their names or compatibility.
• To improve accuracy in food ingredient recognition from images, overcoming limitations of traditional image processing methods.
• To develop an end-to-end system that links detected ingredients to relevant, high-quality recipes using machine learning.
• To create a custom dataset of 9,856 images across 32 food ingredient classes for training and evaluating the model.

Proposed method

• A custom dataset of 9,856 images was collected and annotated, covering 32 distinct food ingredient classes.
• A Convolutional Neural Network (CNN) was trained to classify food ingredients from input images with high accuracy.
• Ingredient detection results were used as input features for a recipe recommendation algorithm based on machine learning.
• The model was evaluated using standard classification metrics, with accuracy reported as the primary performance measure.
• The system integrates image-based ingredient recognition with recipe retrieval, enabling real-time recommendations.

Experimental results

Research questions

• RQ1 Can a deep learning-based CNN model accurately detect diverse food ingredients from real-world images?
• RQ2 How effective is the integration of ingredient detection with recipe recommendation in improving user experience for home cooks?
• RQ3 To what extent does a custom dataset of 32 ingredient classes support accurate and generalizable ingredient recognition?
• RQ4 What is the achievable accuracy of a combined image recognition and recipe recommendation pipeline in a real-world cooking context?

Key findings

• The proposed CNN-based ingredient detection model achieved a test accuracy of 94% on the custom dataset of 9,856 images across 32 food ingredient classes.
• The system successfully links detected ingredients to relevant recipes, demonstrating practical applicability for home cooks.
• The integration of deep learning for ingredient recognition with machine learning for recipe recommendation yields a robust and accurate recommendation pipeline.
• The custom dataset of 9,856 images provides a valuable benchmark for future research in food ingredient recognition.