[论文解读] TALON: Test-time Adaptive Learning for On-the-Fly Category Discovery

Yanan Wu, Yuhan Yan+6 more|arXiv (Cornell University)|Mar 9, 2026

Domain Adaptation and Few-Shot Learning被引用 0

一句话总结

TALON 引入一个测试时自适应框架，用于在线更新编码器与类别原型，实现对新类别的发现，避免基于哈希的量化并提升对新类的发现能力。

ABSTRACT

On-the-fly category discovery (OCD) aims to recognize known categories while simultaneously discovering novel ones from an unlabeled online stream, using a model trained only on labeled data. Existing approaches freeze the feature extractor trained offline and employ a hash-based framework that quantizes features into binary codes as class prototypes. However, discovering novel categories with a fixed knowledge base is counterintuitive, as the learning potential of incoming data is entirely neglected. In addition, feature quantization introduces information loss, diminishes representational expressiveness, and amplifies intra-class variance. It often results in category explosion, where a single class is fragmented into multiple pseudo-classes. To overcome these limitations, we propose a test-time adaptation framework that enables learning through discovery. It incorporates two complementary strategies: a semantic-aware prototype update and a stable test-time encoder update. The former dynamically refines class prototypes to enhance classification, whereas the latter integrates new information directly into the parameter space. Together, these components allow the model to continuously expand its knowledge base with newly encountered samples. Furthermore, we introduce a margin-aware logit calibration in the offline stage to enlarge inter-class margins and improve intra-class compactness, thereby reserving embedding space for future class discovery. Experiments on standard OCD benchmarks demonstrate that our method substantially outperforms existing hash-based state-of-the-art approaches, yielding notable improvements in novel-class accuracy and effectively mitigating category explosion. The code is publicly available at extcolor{blue}{https://github.com/ynanwu/TALON}.

研究动机与目标

• 在开放世界识别场景中，模型必须识别已知类别并从未标注的流数据中发现新类别的动机。
• 消除对固定离线特征提取器与哈希-based 原型的依赖，这些往往导致信息丢失和类别爆炸。
• 开发一个在测试时即可联合更新编码器和类别原型的自适应框架，以吸收来自演化数据流的新知识。
• 在离线训练阶段引入边际感知的对数校准，以增大类间边距并为未来的新类别留出空间。

提出的方法

• 使用无哈希、连续特征空间来提升表征表达力与发现稳定性。
• 离线阶段：应用边际感知的对数校准以增大类间边距并收紧类内紧凑性。
• 在线阶段：实现一个在线决策规则以区分已知类别与新类别，并维持一个动态原型记忆。
• 语义感知的原型更新通过置信度控制的指数滑动平均来细化已知类别原型。
• 通过熵基目标函数和原型级正则化实现稳定的测试时编码器自适应，以对齐特征与原型并保持类别边界。
• 通过为新类别增加新原型实现原型记忆的增长，并定期更新编码器以防止漂移。

Figure 1 : At test time , existing methods ( left ) rely on static inference and fail to adapt to label space shifts, leading to inaccurate discovery and recognition. In contrast, our TALON ( right ) continually accumulates knowledge from unlabeled test data to overcome this challenge. Moreover, our

实验结果

研究问题

• RQ1 测试时自适应（TTA）框架是否能有效支持来自未标注流数据的在线类别发现？
• RQ2 去除基于哈希的量化并同时更新编码器与原型是否能改善新类别发现并降低类别爆炸？
• RQ3 离线的边际感知对数校准是否有助于在嵌入空间中为未来的新类别发现保留空间？
• RQ4 测试时的熵基目标与正则化是否能在保持语义一致性的同时稳定自适应？
• RQ5 与基于哈希的 OCD 最新方法相比，TALON 在标准基准上的表现如何？

主要发现

• TALON 在七个基准上始终优于先前的基于哈希的 OCD 方法。
• 所提出的无哈希框架提高了表征表达力和发现稳定性。
• 边际感知对数校准提升了类内紧凑性和类间分离度，有助于未来新类别的发现。
• 语义感知的原型更新结合在线编码器自适应，减缓类别爆炸并提升新类别的准确性。
• 实验表明在新类别上取得显著提升，并在粗粒度与细粒度数据集上表现稳定。

Figure 2 : Overview of the proposed TALON framework. (a) During the offline stage, we introduce margin-aware logit calibration to enlarge inter-class margins and enhance intra-class compactness, reserving embedding space for future category discovery. (b) At test-time, we jointly update the encoder