[论文解读] Adaptive Debiasing Tsallis Entropy for Test-Time Adaptation

Xiangyu Wu, Dongming Jiang+6 more|arXiv (Cornell University)|Feb 12, 2026

Domain Adaptation and Few-Shot Learning被引用 0

一句话总结

论文提出Adaptive Debiasing Tsallis Entropy (ADTE) 通过为 Tsallis 熵定制一个类别特定的 q 值来改善视觉-语言模型的测试时间自适应，缓解来自不均衡预训练的数据带来的偏差，并在 ImageNet 变体及 10 个跨域基准测试中取得了最先进的结果。

ABSTRACT

Mainstream Test-Time Adaptation (TTA) methods for adapting vision-language models, e.g., CLIP, typically rely on Shannon Entropy (SE) at test time to measure prediction uncertainty and inconsistency. However, since CLIP has a built-in bias from pretraining on highly imbalanced web-crawled data, SE inevitably results in producing biased estimates of uncertainty entropy. To address this issue, we notably find and demonstrate that Tsallis Entropy (TE), a generalized form of SE, is naturally suited for characterizing biased distributions by introducing a non-extensive parameter q, with the performance of SE serving as a lower bound for TE. Building upon this, we generalize TE into Adaptive Debiasing Tsallis Entropy (ADTE) for TTA, customizing a class-specific parameter q^l derived by normalizing the estimated label bias from continuously incoming test instances, for each category. This adaptive approach allows ADTE to accurately select high-confidence views and seamlessly integrate with a label adjustment strategy to enhance adaptation, without introducing distribution-specific hyperparameter tuning. Besides, our investigation reveals that both TE and ADTE can serve as direct, advanced alternatives to SE in TTA, without any other modifications. Experimental results show that ADTE outperforms state-of-the-art methods on ImageNet and its five variants, and achieves the highest average performance on 10 cross-domain benchmarks, regardless of the model architecture or text prompts used. Our code is available at https://github.com/Jinx630/ADTE.

研究动机与目标

• Motivate and address bias in entropy-based test-time adaptation of vision-language models due to imbalanced pretraining data.
• Propose Tsallis Entropy as a debiased alternative to Shannon Entropy for selecting high-confidence views in TTA.
• Introduce Adaptive Debiasing Tsallis Entropy (ADTE) with class-specific q^l derived from estimated label bias.
• Integrate ADTE with a logit-adjustment-inspired bias correction and a memory-based bias estimation strategy.
• Show empirical improvements across ImageNet variants and 10 cross-domain benchmarks with architecture- and prompt-agnostic performance.

提出的方法

• Generalize Shannon Entropy to Tsallis Entropy with a non-extensive parameter q.
• Prove TE reduces to SE as q approaches 1, and that smaller q can yield higher Top-K cumulative reliability (Tcr_K) for biased distributions.
• Introduce ADTE by assigning a per-class q^l derived from normalized estimated label bias, enabling adaptive bias correction across head and tail classes.
• Use a memory bank and pseudo-labels to estimate class biases and solve for prior probabilities via Jacobi iteration (for bias estimation).
• Compute H_ADTE as sum_l P_l^{q^l}/(1−q^l) to score augmented views, select top confident views, and aggregate their predictions for final decision.
• Provide an algorithmic pipeline that combines bias estimation, ADTE computation, confident view selection, and prediction aggregation.

Figure 1: (a) VLM bias, showing higher confidence and accuracy for head classes and lower confidence and accuracy for tail classes. (b) The standard Shannon Entropy ( SE )-based method is widely used in TTA. (c) and (d) Our proposed method, which uses Tsallis Entropy ( TE ) and Adaptive Debiasing Ts

实验结果

研究问题

• RQ1 Can Tsallis Entropy (with a tunable q) better capture uncertainty under biased, imbalanced class distributions in test-time adaptation than Shannon entropy?
• RQ2 Does per-class (head/tail) adaptation of the Tsallis parameter q^l improve high-confidence view selection and final predictions in TTA for vision-language models?
• RQ3 Can an unsupervised, streaming bias estimation and normalization scheme reliably derive q^l without distribution-specific hyperparameter tuning?
• RQ4 Do ADTE-based TTA methods outperform state-of-the-art approaches across ImageNet variants and diverse cross-domain benchmarks independent of model architecture or prompts?

主要发现

• ADTE consistently outperforms state-of-the-art methods on ImageNet and five variants for both ViT-B/16 and ViT-L/14 backbones using template and text-description prompts.
• ADTE achieves the highest average performance across 10 cross-domain benchmarks, demonstrating strong cross-domain generalization.
• TE reduces to SE as q→1, establishing TE as a generalization of SE; smaller q improves high-confidence view selection under bias.
• Class-specific q^l allows adaptive debiasing aligned with the predicted bias per category, improving robustness to head/tail class biases.
• In ablations, removing ADTE or LA degrades performance, highlighting the contribution of per-class entropy debiasing and logit adjustment.

Figure 2: Comparison between SE and TE .