[Paper Review] Learning Algorithms for Active Learning

Philip Bachman, Alessandro Sordoni+1 more|arXiv (Cornell University)|Jul 31, 2017

Machine Learning and AlgorithmsComputer Science38 references53 citations

TL;DR

A meta-learning model trains end-to-end to learn active learning strategies, data representations, and prediction constructors across related tasks, demonstrated on Omniglot one-shot classification and MovieLens cold-start recommendation.

ABSTRACT

We introduce a model that learns active learning algorithms via metalearning. For a distribution of related tasks, our model jointly learns: a data representation, an item selection heuristic, and a method for constructing prediction functions from labeled training sets. Our model uses the item selection heuristic to gather labeled training sets from which to construct prediction functions. Using the Omniglot and MovieLens datasets, we test our model in synthetic and practical settings.

Motivation & Objective

• Motivate reducing labeling cost by learning active learning policies from related tasks.
• Propose an end-to-end model that jointly learns data representations, selection strategy, and prediction construction.
• Extend Matching Networks to settings without pre-existing labels and optimize via backpropagation and reinforcement learning.
• Demonstrate the approach on one-shot Omniglot tasks and cold-start MovieLens recommendations.

Proposed method

• Cast active learning as a sequential decision problem with a labeled support set and an unlabeled pool.
• Extend Matching Networks with a learned item-selection policy and a context-sensitive encoding to build predictions.
• Train the architecture end-to-end using a combination of backpropagation and policy gradients (GAE) with an oracle policy for efficiency.
• Use fast (within-support) and slow (held-out evaluation) prediction modules to guide training.
• Incorporate context-free and context-sensitive encoders, a controller LSTM, a selection module, and an attention-based Matching Network predictor.
• Optimize via an approximation of the objective that balances prediction reward and labeling cost.

Experimental results

Research questions

• RQ1 Can a learned active learning policy outperform engineered heuristics across related tasks?
• RQ2 To what extent can end-to-end meta-learning jointly optimize representations, selection strategy, and prediction construction for active learning?
• RQ3 Does the model generalize to more classes or shots than seen during training (e.g., 20-way Omniglot)?
• RQ4 Is the approach effective in practical settings like cold-start recommender systems (MovieLens)?

Key findings

• On Omniglot, the active learner approaches an optimistic balanced baseline and often nearly matches it, with small degradation in challenging settings (e.g., 1-shot, 10-way).
• The active policy demonstrates good anytime performance and generalizes to problems with more classes than during training (e.g., 20-way classification).
• A 2.2 percentage-point drop is observed for 1-shot 10-way compared with the optimistic baseline, illustrating challenge as tasks grow harder.
• In MovieLens, the learned policy outperforms baselines in RMSE, achieving about 2.5% improvement after 10 ratings.
• The model’s active learning components (selection and fast/slow prediction) are essential; removing key features (e.g., attention temperature) degrades performance.
• Overall, end-to-end learning of representations, selection, and prediction can outperform task-agnostic heuristics in both one-shot classification and collaborative filtering.