[论文解读] A Geometric Approach to Active Learning for Convolutional Neural Networks

Ozan Şener, Silvio Savarese|arXiv (Cornell University)|Aug 1, 2017

Machine Learning and Algorithms参考文献 29被引用 27

一句话总结

本文提出了一种用于卷积神经网络（CNNs）的几何主动学习方法，将主动学习建模为核心集选择问题，利用数据几何特性识别最具信息量的样本。通过选择最能代表数据流形的图像子集，该方法在图像分类任务中的性能显著优于以往的启发式方法。

ABSTRACT

Convolutional neural networks (CNNs) have been successfully applied to many recognition and learning tasks using a universal recipe; training a deep model on a very large dataset of supervised examples. However, this approach is rather restrictive in practice since collecting a large set of labeled images is very expensive. One way to ease this problem is coming up with smart ways for choosing images to be labelled from a very large collection (ie. active learning). Our empirical study suggests that many of the active learning heuristics in the literature are not effective when applied to CNNs in batch setting. Inspired by these limitations, we define the problem of active learning as core-set selection, ie. choosing set of points such that a model learned over the selected subset is competitive for the remaining data points. We further present a theoretical result characterizing the performance of any selected subset using the geometry of the datapoints. As an active learning algorithm, we choose the subset which is expected to yield best result according to our characterization. Our experiments show that the proposed method significantly outperforms existing approaches in image classification experiments by a large margin.

研究动机与目标

• 为解决深度学习中大规模数据集标注成本过高的问题，通过改进卷积神经网络（CNNs）的主动学习方法。
• 克服现有主动学习启发式方法在批量训练设置下应用于卷积神经网络（CNNs）时的局限性。
• 将主动学习建模为核心集选择，目标是选择一个具有代表性的子集，以维持模型性能。

提出的方法

• 该方法将主动学习形式化为核心集选择，旨在选择一组数据点，使其在完整数据集上能取得具有竞争力的性能。
• 基于数据点的几何结构，提出了一种子集性能的理论表征。
• 该算法根据所提出的几何表征，选择预期性能最优的子集。
• 利用数据流形的内在几何特性，优先选择能最大化代表性和多样性的样本。

实验结果

研究问题

• RQ1 数据点的几何特性能否用于提升卷积神经网络（CNNs）中的主动学习性能？
• RQ2 为何标准主动学习启发式方法在卷积神经网络（CNNs）的批量训练设置下会失效？
• RQ3 如何利用数据几何特性对核心集选择进行形式化，以确保模型性能优异？

主要发现

• 所提出的几何主动学习方法在图像分类基准测试中显著优于现有方法。
• 与传统主动学习启发式方法相比，该方法在使用更少标注样本的情况下实现了更优性能。
• 理论分析表明，子集性能与数据点的几何分布密切相关。
• 实证结果证实，基于几何感知的采样能够生成更具代表性和多样性的训练集。