[Paper Review] Optimal unbiased estimators via convex hulls

Nabil Kahalé|arXiv (Cornell University)|Sep 6, 2019

Statistical Methods and Inference17 references4 citations

TL;DR

This paper presents a geometric algorithm using convex hulls to compute optimal unbiased estimators for Monte Carlo methods in O(m) time, significantly improving upon prior O(m³) methods. It establishes necessary and sufficient conditions for square-integrability of coupled and independent sum estimators and provides a convex hull-based characterization of distributions that maximize estimator efficiency.

ABSTRACT

Necessary and sufficient conditions for the square-integrability of recently proposed unbiased estimators are established. A geometric characterization of a distribution that optimizes the performance of these estimators is given. An algorithm based on convex hulls that finds the optimal distribution truncated to its first m terms in time linear in m is described. The algorithm exploits a connection with a recent randomized dimension reduction method and is illustrated via a numerical example.

Motivation & Objective

• To establish necessary and sufficient conditions for the square-integrability of coupled sum and independent sum estimators in Monte Carlo methods.
• To provide a geometric characterization of infinite-support distributions that optimize the performance of these estimators using convex hulls.
• To develop a computationally efficient algorithm that computes an optimal m-truncated distribution for these estimators in O(m) time.
• To improve upon existing O(m³) dynamic programming approaches by leveraging a connection to randomized dimension reduction and convex hull geometry.
• To demonstrate the practicality and efficiency of the proposed method through a numerical example and theoretical guarantees.

Proposed method

• Uses convex hulls to identify the optimal distribution for unbiased estimators by solving a geometric optimization problem.
• Applies a lower hull construction on the sequence (ϑ_i, γ_i) to derive the optimal q* via affine segments with negative slopes.
• Employs a dual formulation of the optimization problem to achieve linear-time complexity O(m) by exploiting properties of the lower hull.
• Derives the second moment of the estimators under the new square-integrability conditions using series expressions involving differences of γ_i.
• Leverages the Cauchy-Schwarz inequality to bound the performance and prove optimality of the derived estimator.
• Uses dynamic programming insights from prior work but reformulates the problem to avoid cubic complexity via geometric insights.

Experimental results

Research questions

• RQ1 What are the necessary and sufficient conditions for the square-integrability of the coupled sum and independent sum estimators?
• RQ2 How can the optimal infinite-support distribution be characterized geometrically to maximize estimator efficiency?
• RQ3 Can the O(m³) dynamic programming algorithm for finding optimal m-truncated distributions be accelerated using geometric techniques?
• RQ4 What is the relationship between the randomized dimension reduction method and the optimization of unbiased estimators?
• RQ5 How does the convex hull-based algorithm achieve O(m) time complexity while preserving optimality?

Key findings

• The paper establishes a necessary and sufficient condition for square-integrability of the coupled sum and independent sum estimators, which is strictly weaker than the sufficient condition in Glynn & Rhee (2015).
• An optimal distribution for the estimator is characterized as the one whose dual sequence lies on the lower hull of the set {(ϑ_i, γ_i)}, ensuring minimal variance.
• The proposed algorithm computes the optimal m-truncated distribution in O(m) time by exploiting the convex hull structure of the parameter space.
• The method achieves the same optimal performance as the O(m³) dynamic programming approach but with a significant speedup, making it practical for high-dimensional problems.
• The algorithm is derived from a connection to randomized dimension reduction and is shown to be equivalent to solving the same optimization problem in linear time.
• Numerical results confirm that the convex hull-based method produces estimators with minimal variance and is robust across different parameter settings.