[Paper Review] Understanding Uncertainty Sampling

Shang Liu, Xiaocheng Li|arXiv (Cornell University)|Jul 6, 2023

Machine Learning and Algorithms4 citations

TL;DR

This paper introduces a general theoretical framework for uncertainty sampling in active learning by defining an 'equivalent loss' that unifies various uncertainty measures under a partial differential equation. It establishes the first generalization bounds for both stream-based and pool-based active learning, proves the optimality of loss-as-uncertainty, and connects uncertainty sampling to risk-sensitive and distributionally robust objectives, offering theoretical grounding for a widely used but heuristic method.

ABSTRACT

Uncertainty sampling is a prevalent active learning algorithm that queries sequentially the annotations of data samples which the current prediction model is uncertain about. However, the usage of uncertainty sampling has been largely heuristic: (i) There is no consensus on the proper definition of "uncertainty" for a specific task under a specific loss; (ii) There is no theoretical guarantee that prescribes a standard protocol to implement the algorithm, for example, how to handle the sequentially arrived annotated data under the framework of optimization algorithms such as stochastic gradient descent. In this work, we systematically examine uncertainty sampling algorithms under both stream-based and pool-based active learning. We propose a notion of equivalent loss which depends on the used uncertainty measure and the original loss function and establish that an uncertainty sampling algorithm essentially optimizes against such an equivalent loss. The perspective verifies the properness of existing uncertainty measures from two aspects: surrogate property and loss convexity. Furthermore, we propose a new notion for designing uncertainty measures called extit{loss as uncertainty}. The idea is to use the conditional expected loss given the features as the uncertainty measure. Such an uncertainty measure has nice analytical properties and generality to cover both classification and regression problems, which enable us to provide the first generalization bound for uncertainty sampling algorithms under both stream-based and pool-based settings, in the full generality of the underlying model and problem. Lastly, we establish connections between certain variants of the uncertainty sampling algorithms with risk-sensitive objectives and distributional robustness, which can partly explain the advantage of uncertainty sampling algorithms when the sample size is small.

Motivation & Objective

• To address the lack of theoretical understanding behind uncertainty sampling, a widely used but heuristic active learning strategy.
• To unify diverse uncertainty measures (e.g., entropy, margin, least confidence) under a single theoretical lens using the concept of equivalent loss.
• To develop a general, problem-agnostic uncertainty measure—'loss as uncertainty'—that applies to both classification and regression.
• To derive the first generalization bounds for uncertainty sampling in both stream-based and pool-based active learning settings.
• To connect uncertainty sampling to risk-sensitive optimization and distributional robustness, explaining its empirical success with small data.

Proposed method

• Proposes the notion of 'equivalent loss' via a partial differential equation linking the original loss and uncertainty measure, showing that uncertainty sampling optimizes this derived objective.
• Introduces 'loss as uncertainty'—using the conditional expected loss given features as the uncertainty metric, which is analytically tractable and generalizable across tasks.
• Establishes that uncertainty sampling algorithms are equivalent to stochastic gradient descent (SGD) updates on the expected equivalent loss under the empirical data distribution.
• Derives generalization bounds for both stream-based and pool-based active learning by analyzing the convergence of SGD on the equivalent loss objective.
• Uses KKT conditions and divergence constraints to prove optimality of sampling policies that prioritize high-loss, high-uncertainty samples.
• Connects uncertainty sampling to risk-sensitive objectives and distributionally robust optimization by showing that the method implicitly minimizes worst-case risk under perturbations.

Experimental results

Research questions

• RQ1 How can we formally define and unify different uncertainty measures used in active learning under a single theoretical framework?
• RQ2 What is the underlying optimization objective that uncertainty sampling actually minimizes?
• RQ3 Can we derive generalization bounds for uncertainty sampling in both stream-based and pool-based active learning?
• RQ4 Why does uncertainty sampling perform well with small data, despite lacking theoretical guarantees?
• RQ5 Can a single uncertainty measure be designed to work uniformly across classification and regression tasks?

Key findings

• The paper establishes that uncertainty sampling optimizes an 'equivalent loss' derived from the original loss and uncertainty measure via a partial differential equation, providing a unifying theoretical foundation.
• The proposed 'loss as uncertainty' measure is analytically well-behaved and generalizes across classification and regression tasks, unlike prior case-specific designs.
• Generalization bounds are derived for both stream-based and pool-based active learning, proving convergence and data efficiency under minimal assumptions.
• The framework explains the empirical success of uncertainty sampling in small-data regimes by linking it to risk-sensitive and distributionally robust optimization objectives.
• Theoretical analysis confirms that uncertainty sampling via SGD on the equivalent loss achieves an $O(1/√{T})$ convergence rate under standard assumptions.
• The optimal sampling policy, which prioritizes samples with the highest conditional expected loss, is proven to be the solution to a constrained optimization problem with KKT conditions.