[Paper Review] Scalable Uncertainty Quantification via GenerativeBootstrap Sampler

Minsuk Shin, Lu Wang+1 more|arXiv (Cornell University)|Jun 1, 2020

Gaussian Processes and Bayesian Inference4 citations

TL;DR

This paper proposes the Generative Bootstrap Sampler (GBS), a scalable method that replaces iterative bootstrap resampling with a single optimization to generate bootstrap distributions. By learning a generator function mapping data weights to bootstrap statistics, GBS accelerates computation by hundreds of folds while maintaining asymptotic equivalence and high accuracy across diverse models like linear regression, logistic regression, and Gaussian processes.

ABSTRACT

It has been believed that the virtue of using statistical procedures is on uncertainty quantification in statistical decisions, and the bootstrap method has been commonly used for this purpose. However, nowadays as the size of data massively increases and statistical models become more complicated, the implementation of bootstrapping turns out to be practically challenging due to its repetitive nature in computation. To overcome this issue, we propose a novel computational procedure called {\it Generative Bootstrap Sampler} (GBS), which constructs a generator function of bootstrap evaluations, and this function transforms the weights on the observed data points to the bootstrap distribution. The GBS is implemented by one single optimization, without repeatedly evaluating the optimizer of bootstrapped loss function as in standard bootstrapping procedures. As a result, the GBS is capable of reducing computational time of bootstrapping by hundreds of folds when the data size is massive. We show that the bootstrapped distribution evaluated by the GBS is asymptotically equivalent to the conventional counterpart and empirically they are indistinguishable. We examine the proposed idea to bootstrap various models such as linear regression, logistic regression, Cox proportional hazard model, and Gaussian process regression model, quantile regression, etc. The results show that the GBS procedure is not only accelerating the computational speed, but it also attains a high level of accuracy to the target bootstrap distribution. Additionally, we apply this idea to accelerate the computation of other repetitive procedures such as bootstrapped cross-validation, tuning parameter selection, and permutation test.

Motivation & Objective

• To address the computational infeasibility of traditional bootstrapping in large-scale data settings due to its repetitive optimization nature.
• To develop a scalable alternative to standard bootstrapping that maintains statistical fidelity while drastically reducing computational cost.
• To enable efficient uncertainty quantification in complex models such as Cox proportional hazards and Gaussian process regression.
• To extend the method to accelerate other repetitive statistical procedures, including bootstrapped cross-validation and permutation tests.

Proposed method

• GBS introduces a generator function that maps data point weights to bootstrap statistics, bypassing repeated optimization.
• The generator is trained via a single optimization procedure, learning the functional relationship between weights and bootstrap outcomes.
• The method leverages deep learning to model the non-linear mapping from data weights to bootstrap estimates, enabling fast inference.
• The generator is trained to approximate the empirical bootstrap distribution, ensuring asymptotic equivalence to conventional bootstrapping.
• The approach is applied to various models by training the generator on the loss function of interest, then sampling from the learned distribution.
• The framework generalizes to other repetitive procedures like bootstrapped cross-validation and tuning parameter selection by reusing the learned generator.

Experimental results

Research questions

• RQ1 Can a single optimization procedure replace iterative bootstrapping while preserving statistical accuracy?
• RQ2 How does the computational efficiency of GBS compare to standard bootstrapping across large datasets?
• RQ3 To what extent is the GBS-generated distribution asymptotically equivalent to the conventional bootstrap distribution?
• RQ4 Can GBS be effectively applied to complex models such as Cox proportional hazards and Gaussian process regression?
• RQ5 Can the GBS framework be extended to accelerate other repetitive statistical procedures like permutation tests and cross-validation?

Key findings

• GBS reduces computational time for bootstrapping by up to hundreds of folds on massive datasets, significantly improving scalability.
• The bootstrap distribution generated by GBS is asymptotically equivalent to the conventional bootstrap, ensuring statistical validity.
• Empirical results show that GBS-generated distributions are indistinguishable from standard bootstrap distributions in practice.
• The method achieves high accuracy across diverse models, including linear regression, logistic regression, Cox models, and Gaussian process regression.
• GBS successfully accelerates other repetitive procedures such as bootstrapped cross-validation, tuning parameter selection, and permutation tests.
• The generator function effectively captures the complex mapping from data weights to bootstrap statistics, enabling fast and reliable uncertainty quantification.