[Paper Review] Bayesian active learning for choice models with deep Gaussian processes

Jie Yang, Diego Klabjan|arXiv (Cornell University)|May 4, 2018

Gaussian Processes and Bayesian Inference16 references3 citations

TL;DR

This paper proposes a Bayesian active learning framework for choice models using deep Gaussian processes (DGP) with Gumbel-distributed noise to model individual preferences from pairwise comparisons. By introducing a novel acquisition function, the method efficiently identifies the most preferred choice with minimal queries, outperforming baseline models in both synthetic functions and a real airline itinerary dataset, particularly when computational cost is not a constraint.

ABSTRACT

In this paper, we propose an active learning algorithm and models which can gradually learn individual's preference through pairwise comparisons. The active learning scheme aims at finding individual's most preferred choice with minimized number of pairwise comparisons. The pairwise comparisons are encoded into probabilistic models based on assumptions of choice models and deep Gaussian processes. The next-to-compare decision is determined by a novel acquisition function. We benchmark the proposed algorithm and models using functions with multiple local optima and one public airline itinerary dataset. The experiments indicate the effectiveness of our active learning algorithm and models.

Motivation & Objective

• To develop an active learning algorithm that minimizes the number of pairwise comparisons needed to identify an individual’s most preferred choice.
• To model individual preferences using deep Gaussian processes with Gumbel-distributed noise, aligning with discrete choice model assumptions such as nested logit.
• To design a novel acquisition function that guides the selection of the next pairwise comparison to maximize information gain.
• To evaluate the method on both synthetic functions with multiple local optima and a real-world airline itinerary dataset.
• To compare the performance of deep Gaussian process models (DC-DGP) against standard Gaussian process models (DC-GP) and random search under varying query and computational constraints.

Proposed method

• The method models latent utility as a deep Gaussian process (DGP) prior over choice features, enabling non-linear preference modeling beyond linear utility functions.
• Pairwise comparisons are encoded using a probabilistic model with Gumbel-distributed noise, which supports nested logit and generalized extreme value (GEV) choice model assumptions.
• The joint distribution over pairwise comparisons is modeled without assuming independence, allowing for correlated preferences across choices.
• A new acquisition function is proposed to select the next pair of choices to compare, balancing exploration and exploitation in preference space.
• The algorithm sequentially queries the most informative pairwise comparisons and updates the posterior distribution over latent utilities using Bayesian inference.
• Adaptive UCB and other acquisition strategies are evaluated as alternatives to the proposed function for robustness across different query budgets.

Experimental results

Research questions

• RQ1 Can deep Gaussian processes with Gumbel noise effectively model complex, non-linear individual preference structures in choice experiments?
• RQ2 Does the proposed acquisition function outperform standard strategies like UCB or random selection in minimizing the number of queries to find the most preferred choice?
• RQ3 How do the performance and computational cost of DC-DGP models compare to DC-GP models across varying feature dimensions and query budgets?
• RQ4 In what scenarios does the use of deeper DGP structures (e.g., DC-DGP5) provide significant gains over simpler models (e.g., DC-DGP1)?
• RQ5 How does the method scale on real-world data, such as an airline itinerary dataset with 543 instances and 17-dimensional features?

Key findings

• The DC-DGP5 model achieved the best performance in finding the optimal itinerary within 20 queries on the airline dataset, outperforming DC-DGP1 and DC-GP.
• For high-dimensional problems (17D), the proposed acquisition function outperformed adaptive UCB in the first 15 queries, though performance declined beyond that point.
• When computational time is a concern, the DC-GP model with adaptive UCB is recommended, especially in high-dimensional or time-sensitive settings.
• In low-dimensional or time-constrained scenarios (e.g., 2D), the DC-GP model with adaptive UCB provides a strong trade-off between speed and accuracy.
• The DC-DGP1 model with adaptive UCB is preferred when a larger number of queries is tolerable and high accuracy is needed.
• The DC-DGP5 model with the proposed acquisition function is optimal when computational cost is not a constraint and only a few queries are allowed.