[论文解读] Efficient Deep Aesthetic Image Classification using Connected Local and Global Features

Xin Jin, Le Wu+6 more|arXiv (Cornell University)|Oct 7, 2016

Visual Attention and Saliency Detection参考文献 44被引用 5

一句话总结

该论文提出ILGNet，一种轻量级深度卷积神经网络，通过源自GoogLeNet的改进Inception模块，整合局部与全局特征，实现高效且精确的图像美学分类。其在AVA基准测试中达到最先进性能，计算成本显著降低——在训练和推理时间接近一半的情况下，达到GoogLeNet约2/3的准确率。

ABSTRACT

In this paper we investigate the aesthetic image classification problem, also known as automatically classifying an image into low or high aesthetic quality, which is quite a challenging problem. Considering both the local and global information of images is quite important for image aesthetic quality assessment. Currently, a powerful inception module is proposed which shows very high performance in object classification. We have the observation that the inception module has the ability of considering both the local and global features in nature. Thus, in this paper, we propose a novel DCNN structure codenamed ILGNet for image aesthetics classification, which introduces the Inception module and connects intermediate Local layers to the Global layer for the output. In addition, the ILGNet is derived from part of the GoogLeNet. Thus, we can easily use a pre-trained image classification GoogleLeNet model on the ImageNet dataset and fine tune our connected local and global layer on the large scale aesthetics assessment AVA dataset. The experimental results show that the proposed ILGNet outperforms the state of the art results in image aesthetics assessment in the AVA benchmark. The time cost of both training and test of the ILGNet are significantly less than those of full GoogLeNet with only a little reduction of the classification accuracy. Our ILGNet can achieve similar classification accuracy as that of 2/3 GoogLeNet, whose computational cost is nearly twice of ours. This makes the aesthetic assessment model more easily to be integrated into mobile and embedded systems.

研究动机与目标

• 解决基于深度学习的图像美学质量评估中高效且精确的挑战。
• 利用图像的局部与全局特征，提升美学分类性能。
• 相比完整版GoogLeNet，降低计算成本，同时保持高准确率。
• 实现美学评估模型在移动与嵌入式系统中的部署。

提出的方法

• 提出ILGNet，一种新型DCNN架构，利用Inception模块固有的多尺度特征提取能力，将中间局部层与全局分类层连接。
• 在ImageNet上预训练GoogLeNet模型，并在大规模AVA数据集上微调，用于美学分类。
• 通过融合多个网络阶段的特征，将局部表征与最终全局层结合，增强判别能力。
• 利用Inception模块的设计，自然捕捉图像中的局部纹理与全局结构信息。
• 采用迁移学习加速训练过程，并提升美学分类任务的泛化能力。
• 通过减少参数数量与FLOPs，优化推理效率，相比完整版GoogLeNet更具优势。

实验结果

研究问题

• RQ1 一种结合局部与全局特征的轻量级CNN架构，是否能在图像美学分类中超越现有模型？
• RQ2 将中间局部特征与全局分类器结合，对性能与效率有何影响？
• RQ3 微调后的基于GoogLeNet的模型在计算成本降低的情况下，能达到多大程度的高准确率？
• RQ4 由于其高效性，此类模型是否可有效部署于移动与嵌入式系统？

主要发现

• ILGNet在AVA基准测试中实现了图像美学分类的最先进性能。
• 与完整版GoogLeNet相比，ILGNet显著降低了训练与推理时间，准确率仅轻微下降。
• ILGNet在计算成本接近一半的情况下，达到约2/3 GoogLeNet模型的分类准确率。
• 连接局部与全局特征的融合增强了特征表示，带来更优的分类结果。
• 由于计算资源占用低，该模型非常适合在移动与嵌入式系统中部署。