[论文解读] SoftTriple Loss: Deep Metric Learning Without Triplet Sampling

Qi Qian, Lei Shang+4 more|arXiv (Cornell University)|Sep 11, 2019

Face recognition and analysis参考文献 28被引用 93

一句话总结

SoftTriple 损失通过为每个类别使用多个中心并采用平滑的相似度来实现无需三元组采样的深度度量学习，从而在细粒度任务中提升嵌入质量。

ABSTRACT

Distance metric learning (DML) is to learn the embeddings where examples from the same class are closer than examples from different classes. It can be cast as an optimization problem with triplet constraints. Due to the vast number of triplet constraints, a sampling strategy is essential for DML. With the tremendous success of deep learning in classifications, it has been applied for DML. When learning embeddings with deep neural networks (DNNs), only a mini-batch of data is available at each iteration. The set of triplet constraints has to be sampled within the mini-batch. Since a mini-batch cannot capture the neighbors in the original set well, it makes the learned embeddings sub-optimal. On the contrary, optimizing SoftMax loss, which is a classification loss, with DNN shows a superior performance in certain DML tasks. It inspires us to investigate the formulation of SoftMax. Our analysis shows that SoftMax loss is equivalent to a smoothed triplet loss where each class has a single center. In real-world data, one class can contain several local clusters rather than a single one, e.g., birds of different poses. Therefore, we propose the SoftTriple loss to extend the SoftMax loss with multiple centers for each class. Compared with conventional deep metric learning algorithms, optimizing SoftTriple loss can learn the embeddings without the sampling phase by mildly increasing the size of the last fully connected layer. Experiments on the benchmark fine-grained data sets demonstrate the effectiveness of the proposed loss function. Code is available at https://github.com/idstcv/SoftTriple

研究动机与目标

• 鉴于小批量的局限性，推动无需三元组采样的深度度量学习。
• 将 SoftMax 损失分析为具有单一中心、平滑化的三元组损失。
• 提出使用多个中心的 SoftTriple 损失以捕捉类内方差。
• 发展自适应中心策略以保持紧凑的中心集。
• 展示在细粒度数据集（CUB-2011、Cars196、SOP）上的有效性。

提出的方法

• 将 SoftMax 损失重新表述为带有类别中心的平滑化三元组损失。
• 扩展至每个类别的多个中心，并使用带平滑的最大化求和（max-over-centers）定义类别级别的相似性。
• 引入对中心最大值的熵正则化平滑，以获得类似 SoftHardTriple 的软损失。
• 给出一种使用 L2,1 正则化来合并相似中心并保持紧凑中心集的自适应方案。
• 端到端优化 SoftTriple 目标函数，无需进行三元组采样。
• 在跨数据集的检索和聚类指标上评估嵌入表示。

实验结果

研究问题

• RQ1 Can SoftMax loss be interpreted as a smoothed triplet loss with centers?
• RQ2 Does introducing multiple centers per class better model intra-class variance and improve retrieval/clustering without triplet sampling?
• RQ3 How should the number of centers be chosen and regularized to remain compact while preserving performance?
• RQ4 Is SoftTriple scalable and effective on large fine-grained datasets compared to traditional triplet-based methods?

主要发现

Method	R@1	R@2	R@4	R@8	NMI
SemiHard	42.6	55.0	66.4	77.2	55.4
LiftedStruct	43.6	56.6	68.6	79.6	56.5
Clustering	48.2	61.4	71.8	81.9	59.2
Npairs ∗	51.0	63.3	74.3	83.2	60.4
ProxyNCA	49.2	61.9	67.9	72.4	59.5
SoftMax norm	57.8	70.0	80.1	87.9	65.3
SoftTriple	60.1	71.9	81.2	88.5	66.2

• SoftMax norm already outperforms many metric learning methods, indicating its smoothed triplet interpretation.
• SoftTriple consistently improves over SoftMax norm by leveraging multiple centers.
• SoftTriple achieves state-of-the-art or competitive results on CUB-2011, Cars196, and SOP, often surpassing SoftMax norm by several points in R@1.
• Using a large initial number of centers with L2,1 regularization yields a compact center set and stable performance.
• SoftTriple with large embeddings generally outperforms comparable methods, e.g., on CUB-2011 and Cars196 tables.
• The approach eliminates the sampling phase, while maintaining or improving retrieval/clustering performance.