[Paper Review] Monotone Function Estimation for Computer Experiments

Shirin Golchi, Derek Bingham+2 more|arXiv (Cornell University)|Sep 15, 2013

Gaussian Processes and Bayesian Inference21 references3 citations

TL;DR

This paper proposes a fully Bayesian Gaussian process emulator for deterministic computer experiments that incorporates monotonicity constraints using virtual derivatives and Markov chain Monte Carlo (MCMC) sampling. By enforcing monotonicity through sign constraints on derivatives at specified input points, the method improves predictive accuracy and uncertainty quantification, especially in data-sparse or flat regions, outperforming unconstrained GPs in both simulated and real-world queuing system examples.

ABSTRACT

In statistical modeling of computer experiments sometimes prior information is available about the underlying function. For example, the physical system simulated by the computer code may be known to be monotone with respect to some or all inputs. We develop a Bayesian approach to Gaussian process modelling capable of incorporating monotonicity information for computer model emulation. Markov chain Monte Carlo methods are used to sample from the posterior distribution of the process given the simulator output and monotonicity information. The performance of the proposed approach in terms of predictive accuracy and uncertainty quantification is demonstrated in a number of simulated examples as well as a real queueing system application.

Motivation & Objective

• To develop a Bayesian Gaussian process model that incorporates monotonicity information from computer simulations.
• To address the challenge of interpolating monotone functions in deterministic computer experiments where standard GPs may fail to respect monotonicity.
• To provide full Bayesian inference by sampling from the exact posterior distribution of hyperparameters and predictions, avoiding plug-in estimates.
• To improve predictive performance and uncertainty quantification in scenarios with sparse data or nearly flat response surfaces.
• To enable flexible modeling of monotonicity over subsets of input space through user-specified derivative sign constraints.

Proposed method

• The method encodes monotonicity constraints using binary indicators (virtual derivatives) at selected input locations, specifying that the derivative sign is positive or negative.
• A link function transforms the derivative sign constraints into a probabilistic framework, allowing the use of MCMC to sample from the joint posterior distribution.
• The approach uses a constrained sequential Monte Carlo (SMC) algorithm to efficiently sample from the posterior, ensuring interpolated paths satisfy monotonicity and data fidelity.
• The method jointly samples hyperparameters (length-scale l and variance σ²), function values, and derivative signs, providing full uncertainty quantification.
• The covariance structure uses the Matérn kernel with λ = 5/2, and derivatives of the covariance function are analytically derived for use in the likelihood evaluation.
• Monotonicity is enforced by restricting the derivative process to remain in R⁺ (positive reals), with the posterior mass concentrated in the positive domain when constraints are strict.

Experimental results

Research questions

• RQ1 Can monotonicity constraints be effectively incorporated into a Gaussian process emulator for deterministic computer experiments?
• RQ2 How does enforcing monotonicity improve predictive accuracy and uncertainty quantification compared to unconstrained GP models?
• RQ3 What is the impact of monotonicity constraints on posterior uncertainty, especially in regions with sparse data or near-flat response surfaces?
• RQ4 How does the proposed MCMC-based approach compare to plug-in or approximate inference methods in terms of uncertainty calibration?
• RQ5 Can the method flexibly model monotonicity over partial input ranges by specifying derivative sign constraints at selected points?

Key findings

• The monotone GP emulator significantly reduces predictive mean squared error (MSE) compared to the unconstrained GP in both simulated examples and the real queuing system application.
• In the queuing system example, the unconstrained GP produced wide, uninformative credible intervals due to high posterior uncertainty, while the monotone model provided tighter, more reliable uncertainty estimates.
• The method successfully captured the true monotone behavior in regions where the unconstrained GP inferred non-monotone relationships due to data gaps.
• When the true function is nearly constant but monotone, the constrained model avoided spurious non-monotonicity that the unconstrained GP sometimes produced.
• The posterior distribution of derivatives was concentrated in R⁺ when monotonicity constraints were strong, confirming effective enforcement of the prior information.
• The SMC algorithm demonstrated stability and scalability, enabling efficient sampling even in higher-dimensional input spaces with monotonicity constraints.