[论文解读] AdaNPC: Exploring Non-Parametric Classifier for Test-Time Adaptation

Yifan Zhang, Xue Wang+6 more|arXiv (Cornell University)|Apr 25, 2023

Domain Adaptation and Few-Shot Learning被引用 14

一句话总结

AdaNPC 引入一种非参数、基于记忆的测试时自适应分类器，能够在连续目标域上取得强劲的域泛化（DG）表现和鲁棒性，而无需进行大量梯度更新。

ABSTRACT

Many recent machine learning tasks focus to develop models that can generalize to unseen distributions. Domain generalization (DG) has become one of the key topics in various fields. Several literatures show that DG can be arbitrarily hard without exploiting target domain information. To address this issue, test-time adaptive (TTA) methods are proposed. Existing TTA methods require offline target data or extra sophisticated optimization procedures during the inference stage. In this work, we adopt Non-Parametric Classifier to perform the test-time Adaptation (AdaNPC). In particular, we construct a memory that contains the feature and label pairs from training domains. During inference, given a test instance, AdaNPC first recalls K closed samples from the memory to vote for the prediction, and then the test feature and predicted label are added to the memory. In this way, the sample distribution in the memory can be gradually changed from the training distribution towards the test distribution with very little extra computation cost. We theoretically justify the rationality behind the proposed method. Besides, we test our model on extensive numerical experiments. AdaNPC significantly outperforms competitive baselines on various DG benchmarks. In particular, when the adaptation target is a series of domains, the adaptation accuracy of AdaNPC is 50% higher than advanced TTA methods. The code is available at https://github.com/yfzhang114/AdaNPC.

研究动机与目标

• 激发域泛化与对未见分布的鲁棒性。
• 提出一种记忆增强的非参数方法用于测试时自适应（AdaNPC）。
• 从理论上证明非参数分类器如何降低域偏差与目标风险。
• 展示在多个 DG 基准测试上，结合不同骨干网络时的显著实证收益。

提出的方法

• 用基于 KNN 的损失进行训练，以学习将同标签特征拉近、将不同标签特征推远的表征（式 (Eq. 4)）。
• 构造源特征及标签的记忆库 M；在测试时使用它对最近的 K 个邻居进行投票（式 (Eq. 5)）。
• 在置信度较高时，通过用测试特征及预测标签更新记忆库来执行基于记忆的测试时自适应。
• 可选地重新训练一个BN层，以在自适应期间提升表征质量。
• 实现简单的先进先出（FIFO）记忆管理和直接的 KNN 搜索以提高效率。

实验结果

研究问题

• RQ1 在没有基于梯度的更新的情况下，非参数、基于记忆的分类器是否能改善域泛化的测试时自适应？
• RQ2 将在线目标样本纳入记忆对目标风险和域偏差有何影响？
• RQ3 在协变量偏移和后验偏移下，基于 KNN 的自适应有哪些理论保证？
• RQ4 AdaNPC 在多个 DG 基准与不同骨干网络上的表现如何，包括连续域自适应场景？

主要发现

Method	RMNIST	PACS	VLCS	TerraIncognita	DomainNet	Avg
ERM	97.8 ± 0.1	77.6 ± 0.3	86.7 ± 0.3	41.3 ± 0.1	53.0 ± 0.3	71.3
IRM	97.5 ± 0.2	76.9 ± 0.6	84.5 ± 1.1	28.0 ± 5.1	50.5 ± 0.7	67.5
GDRO	97.9 ± 0.1	77.4 ± 0.5	87.1 ± 0.1	33.4 ± 0.3	52.4 ± 0.1	69.6
CORAL	98.0 ± 0.0	77.7 ± 0.2	87.1 ± 0.5	41.8 ± 0.1	52.8 ± 0.2	71.5
DANN	97.9 ± 0.1	79.7 ± 0.5	85.2 ± 0.2	38.3 ± 0.1	50.6 ± 0.4	70.3
MTL	97.9 ± 0.1	77.7 ± 0.5	86.7 ± 0.2	40.8 ± 0.1	52.2 ± 0.4	71.1
SagNet	97.9 ± 0.0	77.6 ± 0.1	86.4 ± 0.4	40.8 ± 0.2	52.5 ± 0.4	71.1
ARM	98.1 ± 0.1	77.8 ± 0.3	85.8 ± 0.2	36.0 ± 0.2	51.2 ± 0.5	69.8
VREx	97.9 ± 0.1	78.1 ± 0.2	87.2 ± 0.6	30.1 ± 3.7	51.4 ± 0.5	68.9
Fish	97.9 ± 0.1	77.8 ± 0.6	85.8 ± 0.6	43.4 ± 0.3	50.8 ± 0.4	71.1
Fishr	97.8 ± 0.1	78.2 ± 0.2	86.9 ± 0.2	41.8 ± 0.2	53.6 ± 0.4	71.7
AdaNPC	98.5 ± 0.1	79.5 ± 2.4	88.8 ± 0.1	42.9 ± 0.5	53.9 ± 0.3	72.7
AdaNPC +BN	98.4 ± 0.1	80.2 ± 0.2	88.9 ± 0.1	43.1 ± 0.8	54.0 ± 0.1	72.9

• AdaNPC 在五个 OOD 基准上实现了与最先进的 DG 性能相竞争的表现。
• AdaNPC 与 AdaNPC +BN 超越了许多基线，在平均准确度上取得显著提升。
• BN 重新训练在最小化参数更新的同时带来额外收益。
• 即使不进行微调，AdaNPC 也表现出强劲的性能，且随着源数据集规模的扩大而表现良好。
• 在连续自适应中，AdaNPC 显著减轻遗忘并维持源域准确率。