[论文解读] Mining Discriminative Food Regions for Accurate Food Recognition

Jianing Qiu, Frank P.-W. Lo+3 more|arXiv (Cornell University)|Jul 8, 2022

Advanced Chemical Sensor Technologies被引用 44

一句话总结

论文提出 PAR-Net，是一个三分支网络，通过对抗性抹除逐步挖掘判别性食物区域，并融合局部与全局特征实现端到端的食物识别，在多个数据集上达到最先进的结果，包括一个新的 Sushi-50 精细粒度数据集。

ABSTRACT

Automatic food recognition is the very first step towards passive dietary monitoring. In this paper, we address the problem of food recognition by mining discriminative food regions. Taking inspiration from Adversarial Erasing, a strategy that progressively discovers discriminative object regions for weakly supervised semantic segmentation, we propose a novel network architecture in which a primary network maintains the base accuracy of classifying an input image, an auxiliary network adversarially mines discriminative food regions, and a region network classifies the resulting mined regions. The global (the original input image) and the local (the mined regions) representations are then integrated for the final prediction. The proposed architecture denoted as PAR-Net is end-to-end trainable, and highlights discriminative regions in an online fashion. In addition, we introduce a new fine-grained food dataset named as Sushi-50, which consists of 50 different sushi categories. Extensive experiments have been conducted to evaluate the proposed approach. On three food datasets chosen (Food-101, Vireo-172, and Sushi-50), our approach performs consistently and achieves state-of-the-art results (top-1 testing accuracy of $90.4\%$, $90.2\%$, $92.0\%$, respectively) compared with other existing approaches. Dataset and code are available at https://github.com/Jianing-Qiu/PARNet

研究动机与目标

• 通过改进自动食物识别准确性来推动被动式饮食监控。
• 开发一种弱监督方法以在图像中发现判别性食物区域。
• 设计一个端到端可训练的架构，将全局与 mined 区域表示整合。

提出的方法

• 引入带有三个子网络的 PAR-Net：P-Net（全局图像分类）、A-Net（在抹除判别区域后进行分类）和 R-Net（对 mined 区域进行分类）。
• 使用对抗性抹除通过 CAM 热力图逐步挖掘判别区域。
• 在线计算 CAM 以突出判别区域并为 R-Net 生成区域裁剪。
• 将全图特征与 mined 区域特征连接后，通过额外的全连接层进行分类。
• 端到端训练，损失包括 P-Net、A-Net、R-Net 及拼接表示的组合损失。

实验结果

研究问题

• RQ1 判定挖掘判别区域是否能在全局图像特征基础上提升食物识别性能？
• RQ2 为了在不过度计算的前提下实现最大性能，应该挖掘多少个区域？
• RQ3 通过端到端训练整合全局与区域表示是否优于单网络基线？
• RQ4 基于在线 CAM 的区域挖掘在端到端优化的食物识别中是否有效？

主要发现

• PAR-Net 在 Food-101（1-crop: 89.3；10-crop: 90.4）、Vireo-172（1-crop: 89.6；10-crop: 90.2）和 Sushi-50（1-crop: 91.8；10-crop: 92.0）上持续达到最先进的 top-1 精度。
• 使用三个判别区域（T=3）在可控开销下提供强劲性能。
• 独立的全局、区域和抹除区域任务子网络优于共享权重或单一网络变体。
• 将全局图像特征与 mined 区域特征拼接比仅使用任一来源具有更高的准确性。