[论文解读] Few-shot Visual Reasoning with Meta-Analogical Contrastive Learning

Youngsung Kim, Jinwoo Shin+2 more|arXiv (Cornell University)|Jan 1, 2020

Multimodal Machine Learning Applications被引用 6

一句话总结

本文提出了一种基于元类比对比学习的 few-shot 视觉推理方法，以最少的数据模仿人类的关联推理。通过对比学习对 few-shot 训练样本与测试样本之间的结构关系进行对齐，该模型在 RAVEN 数据集上实现了最先进性能，尤其在低数据设置下表现优异，并通过元学习实现了对未见属性的泛化。

ABSTRACT

While humans can solve a visual puzzle that requires logical reasoning by observing only few samples, it would require training over large amount of data for state-of-the-art deep reasoning models to obtain similar performance on the same task. In this work, we propose to solve such a few-shot (or low-shot) visual reasoning problem, by resorting to analogical reasoning, which is a unique human ability to identify structural or relational similarity between two sets. Specifically, given training and test sets that contain the same type of visual reasoning problems, we extract the structural relationships between elements in both domains, and enforce them to be as similar as possible with analogical learning. We repeatedly apply this process with slightly modified queries of the same problem under the assumption that it does not affect the relationship between a training and a test sample. This allows to learn the relational similarity between the two samples in an effective manner even with a single pair of samples. We validate our method on RAVEN dataset, on which it outperforms state-of-the-art method, with larger gains when the training data is scarce. We further meta-learn our analogical contrastive learning model over the same tasks with diverse attributes, and show that it generalizes to the same visual reasoning problem with unseen attributes.

研究动机与目标

• 为解决 few-shot 视觉推理的挑战，即深度学习模型通常需要大规模数据集才能实现泛化。
• 利用类比推理——人类检测关系相似性的能力——作为 few-shot 学习在视觉推理任务中的机制。
• 开发一种对比学习框架，即使仅有一个标注样本对，也能对齐训练样本与测试样本之间的结构关系。
• 通过在多样化属性上进行元学习，提升模型在视觉推理任务中对未见属性组合的泛化能力。

提出的方法

• 该方法提取视觉推理问题中训练样本与测试样本内元素之间的结构关系。
• 通过对比学习目标强制这些关系保持相似性，最小化类比关系结构之间的距离。
• 模型对同一问题应用轻微修改的查询，以稳定并优化关系不变性的学习。
• 采用元学习策略在多样化属性上训练模型，从而实现对未见属性组合的泛化。
• 该框架假设：查询的微小变化不会改变训练样本与测试样本之间潜在的关系结构。

实验结果

研究问题

• RQ1 类比推理能否被有效利用以提升 few-shot 视觉推理性能？
• RQ2 结构关系的对比学习能否在视觉推理任务的不同属性配置间实现泛化？
• RQ3 所提出的方法是否在训练数据稀缺时优于现有最先进模型？
• RQ4 在多样化属性上进行元学习能否提升对未见属性类型的零样本泛化能力？

主要发现

• 所提方法在 RAVEN 基准上实现了最先进性能，尤其在低数据设置下表现突出。
• 当训练数据稀缺时，性能提升更为显著，展现出强大的 few-shot 泛化能力。
• 经过元学习后，模型能有效泛化到包含未见属性的视觉推理问题。
• 结构关系的对比对齐显著提升了关系推理能力，即使仅使用一个训练样本亦能实现。