[Paper Review] Convergence Rates of Active Learning for Maximum Likelihood Estimation

Kamalika Chaudhuri, Sham M. Kakade+2 more|arXiv (Cornell University)|Jun 8, 2015

Machine Learning and Algorithms16 references4 citations

TL;DR

This paper proposes a two-stage active learning algorithm for maximum likelihood estimation (MLE) in Generalized Linear Models (GLMs), achieving optimal convergence rates with only one additional round of interaction beyond random sampling. The method selects informative samples using a Fisher information-based criterion tailored to minimize expected log-likelihood error, proving tight upper and lower bounds that show near-optimal label efficiency.

ABSTRACT

An active learner is given a class of models, a large set of unlabeled examples, and the ability to interactively query labels of a subset of these examples; the goal of the learner is to learn a model in the class that fits the data well. Previous theoretical work has rigorously characterized label complexity of active learning, but most of this work has focused on the PAC or the agnostic PAC model. In this paper, we shift our attention to a more general setting -- maximum likelihood estimation. Provided certain conditions hold on the model class, we provide a two-stage active learning algorithm for this problem. The conditions we require are fairly general, and cover the widely popular class of Generalized Linear Models, which in turn, include models for binary and multi-class classification, regression, and conditional random fields. We provide an upper bound on the label requirement of our algorithm, and a lower bound that matches it up to lower order terms. Our analysis shows that unlike binary classification in the realizable case, just a single extra round of interaction is sufficient to achieve near-optimal performance in maximum likelihood estimation. On the empirical side, the recent work in ~\cite{Zhang12} and~\cite{Zhang14} (on active linear and logistic regression) shows the promise of this approach.

Motivation & Objective

• To develop a theoretically grounded active learning algorithm for maximum likelihood estimation in a general model class.
• To analyze the finite-sample label complexity of active learning in MLE settings, particularly for GLMs.
• To establish both upper and lower bounds on label requirements to prove optimality of the proposed method.
• To show that only a single round of active sampling beyond initial random labeling is sufficient for near-optimal performance in MLE, unlike in binary classification.
• To provide a principled criterion for sample selection that directly optimizes the goal of minimizing expected log-likelihood error.

Proposed method

• The algorithm uses a two-stage approach: first, it randomly samples a small number of labels to form an initial estimate $\theta_1$ of the true parameter $\theta^*$.
• In the second stage, it selects additional samples by optimizing a summary statistic of the Fisher information matrix at $\theta_1$, specifically targeting the minimization of expected log-likelihood error.
• The selection criterion is derived from the asymptotic covariance of the MLE, ensuring alignment with the learning objective.
• The method assumes regularity conditions on the model class, including convexity of the negative log-likelihood and boundedness of Fisher information and its derivatives.
• It leverages the structure of GLMs, where the Fisher information depends only on $x$ and $\theta$, enabling broad applicability to regression, binary/multi-class classification, and CRFs.
• Theoretical analysis involves bounding the expected log-likelihood error using trace of inverse Fisher information matrices, leading to convergence rate expressions in terms of sample size.

Experimental results

Research questions

• RQ1 Can active learning achieve optimal convergence rates for maximum likelihood estimation in general model classes?
• RQ2 Is a single round of active sampling sufficient to achieve near-optimal label efficiency in MLE, compared to binary classification settings?
• RQ3 What sample selection criterion optimally minimizes expected log-likelihood error in finite-sample MLE?
• RQ4 How do the label requirements of active MLE compare to theoretical lower bounds?
• RQ5 Can the proposed method be applied to widely used models like linear regression, logistic regression, and multinomial logistic regression?

Key findings

• The proposed two-stage active learning algorithm achieves a convergence rate of $ (1+o(1)) \frac{\mathrm{Tr}(I_{\Gamma^*}(\theta^*)^{-1} I_U(\theta^*))}{m_2} $, matching the theoretical lower bound up to lower-order terms.
• For GLMs including logistic and multinomial logistic regression, the method achieves optimal convergence with $ m_1 = \omega(d) $ samples in the first stage, where $ d $ is the input dimension.
• The label complexity is optimal: no algorithm using sample selection and MLE can converge faster than this rate, up to lower-order terms.
• Unlike in binary classification under the PAC model, only one additional round of interaction is needed to achieve near-optimal performance in MLE, indicating a fundamental difference in label efficiency.
• The method applies to a broad class of models, including linear regression, logistic regression, and conditional random fields, due to the general condition on Fisher information dependence.
• The analysis shows that the convergence rate does not grow with dimension $ d $ under standard assumptions, such as bounded $ \|\theta^*\|_2 $, $ \|x\|_2 $, and $ \Sigma \preceq \frac{C}{d}I $, ensuring scalability.