[Paper Review] ChineseFoodNet: A large-scale Image Dataset for Chinese Food Recognition

Xin Chen, Yu Zhu+3 more|arXiv (Cornell University)|May 8, 2017

Advanced Chemical Sensor Technologies34 references88 citations

TL;DR

The paper introduces ChineseFoodNet, a large-scale dataset of 185,628 images across 208 Chinese dish categories, benchmarks multiple CNNs, and proposes TastyNet, a two-step data fusion ensemble that improves top-1 accuracy over individual models.

ABSTRACT

In this paper, we introduce a new and challenging large-scale food image dataset called "ChineseFoodNet", which aims to automatically recognizing pictured Chinese dishes. Most of the existing food image datasets collected food images either from recipe pictures or selfie. In our dataset, images of each food category of our dataset consists of not only web recipe and menu pictures but photos taken from real dishes, recipe and menu as well. ChineseFoodNet contains over 180,000 food photos of 208 categories, with each category covering a large variations in presentations of same Chinese food. We present our efforts to build this large-scale image dataset, including food category selection, data collection, and data clean and label, in particular how to use machine learning methods to reduce manual labeling work that is an expensive process. We share a detailed benchmark of several state-of-the-art deep convolutional neural networks (CNNs) on ChineseFoodNet. We further propose a novel two-step data fusion approach referred as "TastyNet", which combines prediction results from different CNNs with voting method. Our proposed approach achieves top-1 accuracies of 81.43% on the validation set and 81.55% on the test set, respectively. The latest dataset is public available for research and can be achieved at https://sites.google.com/view/chinesefoodnet.

Motivation & Objective

• Create a large, diverse dataset of Chinese dishes including web and real-world photos to enable robust deep learning models for dish recognition.
• Benchmark a range of state-of-the-art CNN architectures on ChineseFoodNet to establish performance baselines.
• Develop and evaluate data-efficient labeling/cleaning strategies to build high-quality labels with reduced manual effort.
• Propose and validate an ensemble data-fusion approach (TastyNet) to improve recognition accuracy beyond single-model results.

Proposed method

• Collect and aggregate Chinese dish images from web sources and real-world photos, followed by semi-supervised labeling guided by metadata and keyword-based filtering.
• Clean data using size/entropy checks and deduplicate via 1,024-dimension AlexNet features and Euclidean distance.
• Fine-tune ImageNet-pretrained CNNs (SqueezeNet, VGG19-BN, ResNet variants, DenseNet variants) on ChineseFoodNet.
• Evaluate single CNNs to establish baselines for Top-1 and Top-5 accuracy on validation and test sets.
• Propose TastyNet: a two-step data fusion ensemble that averages predictions from multiple CNNs (ResNet152, DenseNet121/169/201, VGG19-BN) to achieve superior accuracy.
• Provide reproducible evaluation protocol including training details (SGD, learning rate schedule, data preprocessing) and public model checkpoints.

Experimental results

Research questions

• RQ1 What is the scale and category coverage required for effective Chinese dish recognition using deep learning?
• RQ2 How do diverse CNN architectures perform on ChineseFoodNet, and how close can they approach real-world recognition tasks?
• RQ3 Can ensemble data fusion (TastyNet) outperform single networks on ChineseFoodNet?
• RQ4 What practical data-labeling strategies can efficiently produce high-quality annotations for a large, noisy image dataset?

Key findings

• ChineseFoodNet contains 185,628 images in 208 categories, making it the largest public Chinese food recognition dataset at the time.
• Among single models, DenseNet201 achieves the best validation Top-1 (79.05%) and VGG19-BN the best test Top-1 (79.22%).
• Deeper/wider networks generally outperform shallower ones (e.g., ResNet152, DenseNet169/201 improve over shallower variants).
• The proposed TastyNet ensemble (ResNet152 + DenseNet121/169/201 + VGG19-BN) yields Top-1 81.55% on the test set and 81.43% on validation, surpassing single-model counterparts by up to ~2.38% (validation) and ~2.33% (test).
• Ensembling diverse architectures provides greater gains than ensembling similar networks, indicating complementary strengths across model families.