[Paper Review] Adaptive Posterior Convergence Rates in Bayesian Density Deconvolution with Supersmooth Errors

Abhra Sarkar, Debdeep Pati+2 more|arXiv (Cornell University)|Aug 25, 2013

Statistical Methods and Inference23 references3 citations

TL;DR

This paper establishes adaptive posterior convergence rates in Bayesian density deconvolution under supersmooth errors using a Dirichlet process mixture of normals prior. It proves the posterior contracts at the minimax rate $(\log n)^{-\eta/\beta}$ in $L_p$ norms for $2 \leq p \leq \infty$, adaptively over the smoothness $\eta$ of the true density, under mild tail conditions, marking the first theoretical confirmation of adaptive minimax posterior concentration in deconvolution with nonparametric priors.

ABSTRACT

Bayesian density deconvolution using nonparametric prior distributions is a useful alternative to the frequentist kernel based deconvolution estimators due to its potentially wide range of applicability, straightforward uncertainty quantification and generalizability to more sophisticated models. This article is the first substantive effort to theoretically quantify the behavior of the posterior in this recent line of research. In particular, assuming a known supersmooth error density, a Dirichlet process mixture of Normals on the true density leads to a posterior convergence rate same as the minimax rate $(\log n)^{-η/β}$ adaptively over the smoothness $η$ of an appropriate Hölder space of densities, where $β$ is the degree of smoothness of the error distribution. Our main contribution is achieving adaptive minimax rates with respect to the $L_p$ norm for $2 \leq p \leq \infty$ under mild regularity conditions on the true density. En route, we develop tight concentration bounds for a class of kernel based deconvolution estimators which might be of independent interest.

Motivation & Objective

• To establish theoretical posterior concentration rates in Bayesian density deconvolution with supersmooth measurement errors.
• To investigate whether Dirichlet process mixture of normals (DPMM) priors can achieve adaptive minimax posterior convergence rates in $L_p$ norms.
• To determine the minimal regularity conditions on the true density and error density for posterior consistency and optimal rates.
• To extend existing posterior consistency results from standard density estimation to the deconvolution setting with measurement error.
• To provide sufficient conditions under which the posterior contracts at the minimax rate without requiring prior knowledge of the smoothness $\eta$.

Proposed method

• Uses a Dirichlet process mixture of normals prior on the unknown density $f_X$, with a conjugate Normal base measure and inverse-gamma prior on the bandwidth.
• Employs kernel-based deconvolution estimators as a benchmark and derives tight concentration bounds for them, which are used in posterior contraction analysis.
• Applies a general posterior contraction framework based on Kullback-Leibler (KL) divergence and entropy conditions, adapted to the deconvolution setting.
• Establishes posterior contraction in $L_p$ norms via verification of sufficient conditions involving KL divergence, entropy, and prior concentration bounds.
• Uses Talagrand’s inequality and moment bounds to control the empirical process term in the posterior contraction analysis.
• Derives the optimal bandwidth $h_n \asymp (\log n)^{-1/\beta}$ to balance bias and variance in the deconvolution estimator, linking it to the posterior concentration rate.

Experimental results

Research questions

• RQ1 Can a Dirichlet process mixture of normals prior achieve adaptive minimax posterior convergence rates in Bayesian density deconvolution with supersmooth errors?
• RQ2 What are the minimal regularity conditions on the true density and error density for posterior contraction at the minimax rate?
• RQ3 Does the posterior concentration rate in $L_p$ norms ($2 \leq p \leq \infty$) match the frequentist minimax rate $(\log n)^{-\eta/\beta}$?
• RQ4 Can the posterior achieve this rate adaptively, without requiring knowledge of the smoothness $\eta$ of the true density?
• RQ5 How do tail conditions on the true density and error density affect posterior concentration in deconvolution?

Key findings

• The posterior distribution contracts at the minimax rate $(\log n)^{-\eta/\beta}$ in $L_p$ norms for $2 \leq p \leq \infty$, under mild polynomial tail conditions on the true density and error density.
• Adaptive minimax posterior concentration is achieved without prior knowledge of the smoothness $\eta$, making the method fully adaptive.
• The rate $(\log n)^{-\eta/\beta}$ is optimal and matches the frequentist minimax rate for deconvolution with supersmooth errors.
• Tight concentration bounds for kernel-based deconvolution estimators are derived, which are of independent interest and used in the posterior contraction proof.
• The proof relies on verifying KL divergence and entropy conditions using Talagrand’s inequality and moment bounds on the empirical process.
• For accelerated rates, when the error variance $\sigma_n \asymp n^{-1/(2\eta+1)}(\log n)^{t/\eta}$ with $t > (2 + 1/\eta + 1/c_3)/(2 + 1/\eta)$, the posterior contracts at rate $n^{-\eta/(2\eta+1)}(\log n)^t$, matching standard density estimation rates.