[Paper Review] Monte Carlo and Quasi-Monte Carlo Density Estimation via Conditioning

Pierre L’Ecuyer, Florian Puchhammer+1 more|arXiv (Cornell University)|Jun 11, 2019

Statistical Methods and Inference2 references4 citations

TL;DR

This paper proposes a novel Monte Carlo and quasi-Monte Carlo density estimation method using conditional Monte Carlo to produce unbiased density estimators that converge faster than standard nonparametric methods. By leveraging conditional independence and randomized quasi-Monte Carlo, the approach achieves improved convergence rates and reduced mean integrated square error, particularly effective in simulation settings with controlled sampling.

ABSTRACT

Estimating the unknown density from which a given independent sample originates is more difficult than estimating the mean, in the sense that for the best popular non-parametric density estimators, the mean integrated square error converges more slowly than at the canonical rate of $\mathcal{O}(1/n)$. When the sample is generated from a simulation model and we have control over how this is done, we can do better. We examine an approach in which conditional Monte Carlo yields, under certain conditions, a random conditional density which is an unbiased estimator of the true density at any point. By averaging independent replications, we obtain a density estimator that converges at a faster rate than the usual ones. Moreover, combining this new type of estimator with randomized quasi-Monte Carlo to generate the samples typically brings a larger improvement on the error and convergence rate than for the usual estimators, because the new estimator is smoother as a function of the underlying uniform random numbers.

Motivation & Objective

• Address the challenge of slow convergence in standard nonparametric density estimation, which typically converges at O(1/n) rate.
• Improve density estimation accuracy in simulation experiments where the user controls the sampling mechanism.
• Develop a method that produces unbiased density estimators through conditional Monte Carlo, enabling faster convergence.
• Combine conditional Monte Carlo with randomized quasi-Monte Carlo (RQMC) to further enhance convergence speed and reduce error.
• Demonstrate the method’s superiority over classical estimators in terms of mean integrated square error and convergence rate on real-world simulation examples.

Proposed method

• Use conditional Monte Carlo to construct a random conditional density estimator that is unbiased for the true density at any point.
• Generate independent replications of the conditional density estimator and average them to form a final density estimator.
• Apply randomized quasi-Monte Carlo (RQMC) to generate the underlying uniform random numbers, exploiting the smoothness of the conditional density estimator.
• Leverage the smoothness of the conditional density as a function of uniform random numbers to achieve faster convergence with RQMC.
• Use lattice and Sobol’ point sets with low-discrepancy properties and order-dependent weights to improve RQMC performance.
• Estimate the density using kernel density estimation or similar nonparametric techniques on the conditional realizations, with variance reduction via conditioning.

Experimental results

Research questions

• RQ1 Can conditional Monte Carlo be used to construct an unbiased density estimator that converges faster than classical nonparametric methods?
• RQ2 How does the combination of conditional Monte Carlo and RQMC improve convergence rates and reduce mean integrated square error in density estimation?
• RQ3 To what extent does the smoothness of the conditional density estimator as a function of uniform random numbers enhance the performance of RQMC?
• RQ4 How does the proposed method compare to standard Monte Carlo and quasi-Monte Carlo density estimators in terms of error and convergence rate on practical simulation examples?
• RQ5 Can the method be effectively applied to estimate quantiles and expected shortfall with improved confidence interval quality?

Key findings

• The proposed conditional density estimator (CDE) achieves a faster convergence rate than standard nonparametric estimators, which typically converge at O(1/n).
• With RQMC, the method reduces the integrated variance (IV) by a factor of approximately 16 for n = 2^19 in the network reliability example, demonstrating significant error reduction.
• The empirical convergence rate ˆν for lattice and Sobol’ points improved to about -1.2, indicating faster decay of error with sample size.
• In the Asian option example, the RQMC-based CDE method achieved a substantial reduction in IV compared to standard MC, with log-IV decreasing more steeply with log n.
• The method improves quantile and expected shortfall estimation by providing a more accurate density estimate at the quantile point, leading to better confidence interval quality.
• The conditional density estimator is smoother as a function of the uniform random numbers, which amplifies the benefits of RQMC, especially when the conditional density is sufficiently smooth.