[Paper Review] Adaptive Model Selection Framework: An Application to Airline Pricing

Naman Shukla, Arinbjörn Kolbeinsson+2 more|arXiv (Cornell University)|May 21, 2019

Advanced Bandit Algorithms Research11 references3 citations

TL;DR

This paper proposes an adaptive meta-decision framework using Thompson sampling in a multi-armed bandit setting to dynamically route airline customers to the best-performing ancillary pricing models in real time. By continuously learning from online responses, the framework improves revenue per offer by 43% and conversion by 58% compared to random model selection, outperforming any single model in diverse customer contexts.

ABSTRACT

Multiple machine learning and prediction models are often used for the same prediction or recommendation task. In our recent work, where we develop and deploy airline ancillary pricing models in an online setting, we found that among multiple pricing models developed, no one model clearly dominates other models for all incoming customer requests. Thus, as algorithm designers, we face an exploration - exploitation dilemma. In this work, we introduce an adaptive meta-decision framework that uses Thompson sampling, a popular multi-armed bandit solution method, to route customer requests to various pricing models based on their online performance. We show that this adaptive approach outperform a uniformly random selection policy by improving the expected revenue per offer by 43% and conversion score by 58% in an offline simulation.

Motivation & Objective

• To address the exploration-exploitation dilemma in online airline ancillary pricing, where no single model consistently outperforms others across all customer contexts.
• To improve revenue and conversion metrics by adaptively routing customer requests to the most effective pricing models in real time.
• To develop a scalable meta-decision framework that enables dynamic, data-driven model selection without relying on static offline metrics.
• To lay the foundation for future deployment of contextual multi-armed bandit methods in online recommendation and pricing systems.

Proposed method

• Models are treated as 'arms' in a multi-armed bandit framework, with each arm corresponding to a distinct machine learning pricing model.
• Thompson sampling is used to balance exploration and exploitation by stochastically selecting models based on posterior distributions of their performance.
• The system maintains posterior beliefs over each model’s conversion probability using a Beta-Binomial conjugate prior, updated with real-time customer responses (purchase/no purchase).
• The framework supports active learning by automatically reducing traffic to underperforming models, minimizing revenue loss during exploration.
• The approach is extended to contextual bandits by incorporating customer-specific features (e.g., travel type, route) to improve routing precision.
• A simulation environment is constructed to evaluate performance using real-world airline data, measuring revenue per offer and conversion rate.

Experimental results

Research questions

• RQ1 Can an adaptive routing mechanism outperform random or fixed model selection in dynamic airline ancillary pricing?
• RQ2 How does Thompson sampling-based model selection improve business metrics like revenue per offer and conversion rate in an online setting?
• RQ3 To what extent can a meta-decision framework reduce revenue loss during model exploration compared to static or rule-based routing?
• RQ4 Can the framework be extended to handle non-stationary environments and contextual customer features?
• RQ5 How does the inclusion of caution-based sampling and concurrence improve robustness and performance in live deployment?

Key findings

• The adaptive model selection framework improved expected revenue per offer by 43% compared to a uniformly random model selection policy in offline simulations.
• The framework increased the conversion score by 58% over the random baseline, demonstrating significant gains in customer response rates.
• The approach achieved better overall performance than any single model in isolation, due to dynamic routing that adapts to context-specific model strengths.
• Thompson sampling effectively balanced exploration and exploitation, reducing long-term regret and minimizing revenue loss from suboptimal model usage.
• The system automatically down-weighted underperforming models over time, protecting revenue during the exploration phase.
• The framework is extensible to contextual bandits and supports future integration of caution-based sampling and concurrence for enhanced robustness.