[论文解读] An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution

Rosanne Liu, Joel Lehman+5 more|arXiv (Cornell University)|Jul 9, 2018

Domain Adaptation and Few-Shot Learning参考文献 26被引用 169

一句话总结

这篇论文显示卷积神经网络在笛卡尔空间与像素空间之间的坐标变换方面存在困难，提出 CoordConv 以提供坐标通道，并在多项任务中展示了显著的性能提升（学习更快、泛化更好、生成模型中的模态崩溃减少）。

ABSTRACT

Few ideas have enjoyed as large an impact on deep learning as convolution. For any problem involving pixels or spatial representations, common intuition holds that convolutional neural networks may be appropriate. In this paper we show a striking counterexample to this intuition via the seemingly trivial coordinate transform problem, which simply requires learning a mapping between coordinates in (x,y) Cartesian space and one-hot pixel space. Although convolutional networks would seem appropriate for this task, we show that they fail spectacularly. We demonstrate and carefully analyze the failure first on a toy problem, at which point a simple fix becomes obvious. We call this solution CoordConv, which works by giving convolution access to its own input coordinates through the use of extra coordinate channels. Without sacrificing the computational and parametric efficiency of ordinary convolution, CoordConv allows networks to learn either complete translation invariance or varying degrees of translation dependence, as required by the end task. CoordConv solves the coordinate transform problem with perfect generalization and 150 times faster with 10--100 times fewer parameters than convolution. This stark contrast raises the question: to what extent has this inability of convolution persisted insidiously inside other tasks, subtly hampering performance from within? A complete answer to this question will require further investigation, but we show preliminary evidence that swapping convolution for CoordConv can improve models on a diverse set of tasks. Using CoordConv in a GAN produced less mode collapse as the transform between high-level spatial latents and pixels becomes easier to learn. A Faster R-CNN detection model trained on MNIST showed 24% better IOU when using CoordConv, and in the RL domain agents playing Atari games benefit significantly from the use of CoordConv layers.

研究动机与目标

• 定义笛卡尔空间与像素空间之间的坐标变换问题。
• 提出 CoordConv 作为一种简单的层增广，用于注入坐标信息。
• 在 toy（演示）和真实世界任务上评估 CoordConv，以评估学习、泛化和效率。
• 展示 CoordConv 在图像生成、目标检测和强化学习中的影响。

提出的方法

• 将 CoordConv 作为额外的坐标通道附加到卷积前的输入。
• 在监督设置中使用 Not-so-Clevr 数据集研究坐标变换。
• 在分类、回归和渲染任务中比较标准卷积层与 CoordConv。
• 在 GANs、VAE、Faster R-CNN 和 Atari 强化学习中评估 CoordConv，以衡量更广泛的影响。

实验结果

研究问题

• RQ1 在监督下，标准 CNN 是否能有效学习笛卡尔坐标到像素坐标的变换？
• RQ2 通过 CoordConv 增加显式坐标信息是否能够实现完美泛化和更快的学习？
• RQ3 与普通卷积层相比，CoordConv 层是否在生成模型、目标检测和强化学习任务中提升性能？

主要发现

• 卷积网络未能完全学习坐标变换，对象限分割的泛化能力较差。
• CoordConv 在坐标任务上以远少于参数量的方式实现了训练和测试的完美准确度（≈7.5k 参数），并且训练速度更快（几秒对比数小时）。
• 用 CoordConv 取代卷积可减少 GANs 的模态崩溃，并使 2D 潜在空间覆盖更完整。
• 在 Faster R-CNN 中，CoordConv 在类似 MNIST 的检测上得到 24% 更高的 IOU；在 Atari 强化学习中，CoordConv 在若干游戏上提升了性能。
• ImageNet 分类显示来自单个 CoordConv 层的收益微弱或不显著，表明 CoordConv 的收益与任务相关。