[Paper Review] Practical Bayesian Optimization for Transportation Simulators

Laura Luise Schultz, Vadim Sokolov|arXiv (Cornell University)|Oct 8, 2018

Simulation Techniques and ApplicationsDecision Sciences30 references3 citations

TL;DR

This paper proposes a fully Bayesian optimization framework with dimensionality reduction for calibrating complex, high-dimensional transportation simulators. By combining active learning, Gaussian process regression, and linear/nonlinear dimensionality reduction (active subspaces and neural networks), the method enables efficient, parallelized optimization that reduces computational cost and improves sample efficiency, achieving accurate calibration of a real-world multi-modal network in Bloomington, Illinois, within 6.4 minutes of true demand on average.

ABSTRACT

We provide a method to solve optimization problem when objective function is a complex stochastic simulator of an urban transportation system. To reach this goal, a Bayesian optimization framework is introduced. We show how the choice of prior and inference algorithm effect the outcome of our optimization procedure. We develop dimensionality reduction techniques that allow for our optimization techniques to be applicable for real-life problems. We develop a distributed, Gaussian Process Bayesian regression and active learning models that allow parallel execution of our algorithms and enable usage of high performance computing. We present a fully Bayesian approach that is more sample efficient and reduces computational budget. Our framework is supported by theoretical analysis and an empirical study. We demonstrate our framework on the problem of calibrating a multi-modal transportation network of city of Bloomington, Illinois. Finally, we discuss directions for further research.

Motivation & Objective

• To address the challenge of calibrating high-dimensional, stochastic transportation simulators that lack analytical gradients and are computationally expensive to evaluate.
• To develop a fully Bayesian approach that enables uncertainty quantification and sensitivity analysis to prior misspecification.
• To overcome the curse of dimensionality in simulator calibration using active subspace and deep learning-based dimensionality reduction techniques.
• To enable scalable, distributed optimization via parallel batching and high-performance computing for practical deployment in transportation research and planning.
• To provide a theoretically grounded, empirically validated framework that outperforms traditional meta-heuristic methods in sample efficiency and accuracy.

Proposed method

• The framework employs Gaussian process regression with fully Bayesian hyperparameter inference to model the simulator’s black-box objective function.
• It uses expected improvement (EI) as the acquisition function for active learning, sequentially selecting the most informative parameter sets to evaluate.
• Linear dimensionality reduction via active subspaces identifies a low-dimensional subspace where the simulator’s output varies most, reducing the optimization dimensionality.
• Nonlinear dimensionality reduction via neural networks maps high-dimensional parameters to a lower-dimensional latent space while preserving predictive fidelity.
• A distributed, batched optimization strategy enables parallel execution of simulator runs, accelerating convergence and supporting high-performance computing environments.
• The method integrates prior knowledge through constrained parameter bounds and uses Latin Hypercube Sampling (LHS) for initial design of experiments.

Experimental results

Research questions

• RQ1 How can Bayesian optimization be effectively applied to high-dimensional, stochastic transportation simulators with no analytical gradients?
• RQ2 To what extent does a fully Bayesian approach improve sample efficiency and uncertainty quantification compared to frequentist or heuristic methods?
• RQ3 Can active subspaces and neural networks effectively reduce the dimensionality of complex transportation model parameters while preserving calibration accuracy?
• RQ4 How does the proposed framework compare in performance and scalability to traditional meta-heuristic optimization techniques like SPSA or genetic algorithms?
• RQ5 What is the computational overhead and convergence speed of the distributed, batched Bayesian optimization framework in real-world urban transportation calibration?

Key findings

• The framework achieved calibration accuracy within 6.4 minutes of the true demand across a 24-hour period, with a standard deviation of 5 minutes, demonstrating high fidelity to real-world data.
• The use of active subspaces reduced the effective dimensionality of the parameter space, enabling efficient optimization in high-dimensional settings.
• The fully Bayesian approach provided robust uncertainty quantification and enabled sensitivity analysis to prior distributions, improving model reliability.
• The parallel batching strategy significantly reduced wall-clock time, making the method practical for large-scale simulators on high-performance computing clusters.
• The integration of neural networks for nonlinear dimensionality reduction further improved calibration accuracy compared to linear methods in complex, nonlinear simulator landscapes.
• Empirical results showed that the proposed method outperformed traditional meta-heuristic approaches in sample efficiency, requiring fewer simulator evaluations to reach optimal calibration.