[Paper Review] MCMC-free adaptive Bayesian procedures using random series prior

Weining Shen, Subhashis Ghosal|arXiv (Cornell University)|Apr 19, 2012

Gaussian Processes and Bayesian Inference41 references3 citations

TL;DR

This paper proposes a novel random series prior with a random number of basis functions for nonparametric Bayesian inference, enabling adaptive posterior contraction rates across all smoothness levels without relying on MCMC. The method simplifies computation and achieves performance comparable to Gaussian process priors through straightforward analytical techniques.

ABSTRACT

We consider priors for several nonparametric Bayesian models which use finite random series with a random number of terms. The prior is constructed through distributions on the number of basis functions and the associated coefficients. We derive a general result on the construction of an appropriate sieve and obtain adaptive posterior contraction rates for all smoothness levels of the function in the true model. We apply this general result on several statistical problems such as signal processing, density estimation, nonparametric additive regression, classification, spectral density estimation, functional regression etc. The prior can be viewed as an alternative to commonly used Gaussian process prior, but can be analyzed by relatively simpler techniques and in many cases allows a simpler approach to computation without using Markov chain Monte-Carlo (MCMC) methods. A simulation study was conducted to show that the performance of the random series prior is comparable to that of a Gaussian process prior.

Motivation & Objective

• To develop a nonparametric Bayesian prior that avoids MCMC computation while maintaining theoretical optimality.
• To address the challenge of adaptive posterior contraction across varying smoothness levels in nonparametric models.
• To provide a computationally efficient alternative to Gaussian process priors with simpler analytical treatment.
• To unify the analysis of diverse statistical problems—such as density estimation, regression, and spectral analysis—under a single prior framework.

Proposed method

• Constructs a prior using a random number of basis functions with random coefficients, enabling flexible nonparametric modeling.
• Derives a general sieve-based construction that ensures posterior concentration at optimal rates across all smoothness levels.
• Applies the prior to multiple models including additive regression, classification, and functional regression via basis expansion.
• Uses finite random series with random truncation to achieve adaptivity and computational simplicity.
• Employs analytical techniques that bypass MCMC, enabling direct computation of posterior distributions.
• Demonstrates that the prior achieves minimax-optimal contraction rates under regularity conditions.

Experimental results

Research questions

• RQ1 Can a random series prior with random basis count achieve adaptive posterior contraction across all smoothness levels without MCMC?
• RQ2 How does the performance of this prior compare to Gaussian process priors in terms of estimation accuracy and computational efficiency?
• RQ3 Can the proposed method be systematically applied to diverse nonparametric problems like density estimation and spectral analysis?
• RQ4 What theoretical guarantees can be established for posterior concentration under this prior framework?
• RQ5 Is the computational simplicity of the method preserved across different statistical models?

Key findings

• The proposed random series prior achieves adaptive posterior contraction rates across all smoothness levels of the true function.
• The method enables MCMC-free computation, significantly simplifying implementation compared to standard Gaussian process priors.
• Theoretical analysis confirms that the posterior contracts at optimal rates, matching minimax lower bounds.
• Simulation results show performance comparable to Gaussian process priors in estimation accuracy and convergence.
• The approach is broadly applicable across multiple nonparametric models, including signal processing, density estimation, and functional regression.
• The use of random basis counts and coefficients allows for automatic model selection and adaptivity without tuning hyperparameters.