[Paper Review] Maximum likelihood estimation and confidence bands for a discrete log-concave distribution

Fadoua Balabdaoui, Hanna Jankowski+1 more|arXiv (Cornell University)|Jul 20, 2011

Statistical Methods and Inference4 citations

TL;DR

This paper proposes a maximum likelihood estimation (MLE) framework for discrete log-concave probability mass functions, establishing strong consistency and asymptotic normality under both well- and misspecified models. It enables construction of pointwise confidence bands for the true pmf using the R package logcondiscr, validated on H1N1 pandemic data from Ontario, Canada.

ABSTRACT

The assumption of log-concavity is a flexible and appealing nonparametric shape constraint in distribution modelling. In this work, we study the log-concave maximum likelihood estimator (MLE) of a probability mass function (pmf). We show that the MLE is strongly consistent and derive its pointwise asymptotic theory under both the well- and misspecified setting. Our asymptotic results are used to calculate confidence intervals for the true log-concave pmf. Both the MLE and the associated confidence intervals may be easily computed using the R package logcondiscr. We illustrate our theoretical results using recent data from the H1N1 pandemic in Ontario, Canada.

Motivation & Objective

• To develop a nonparametric maximum likelihood estimator (MLE) for discrete probability mass functions under the log-concavity shape constraint.
• To establish theoretical properties of the MLE, including strong consistency and asymptotic normality, under both well-specified and misspecified models.
• To construct valid pointwise confidence intervals for the true log-concave pmf using asymptotic theory.
• To provide a practical computational framework using the R package logcondiscr for estimating and inferring log-concave distributions.
• To demonstrate the method’s utility through an empirical analysis of real-world H1N1 pandemic data from Ontario, Canada.

Proposed method

• Utilizes the log-concave maximum likelihood estimator (MLE) to estimate a discrete probability mass function under the shape constraint of log-concavity.
• Applies asymptotic theory to derive the limiting distribution of the MLE under both well- and misspecified models, enabling inference.
• Derives pointwise confidence intervals for the true pmf based on the asymptotic normality of the MLE.
• Employs convex optimization techniques to compute the MLE, leveraging the log-concave structure for computational tractability.
• Implements the method in the R package logcondiscr for accessible and efficient estimation and inference.
• Validates the method on real data from the H1N1 pandemic in Ontario, Canada, to demonstrate practical applicability.

Experimental results

Research questions

• RQ1 What are the theoretical properties of the log-concave MLE for discrete distributions, particularly its consistency and asymptotic distribution?
• RQ2 How does the MLE perform under model misspecification, and what are the implications for inference?
• RQ3 Can valid confidence bands be constructed for the true log-concave pmf using asymptotic theory?
• RQ4 How can the MLE and associated confidence intervals be efficiently computed in practice?
• RQ5 What insights does the method yield when applied to real-world epidemiological data such as H1N1 case counts?

Key findings

• The log-concave MLE for discrete distributions is strongly consistent, ensuring convergence to the true distribution under mild regularity conditions.
• The MLE exhibits asymptotic normality under both well-specified and misspecified models, enabling valid inference.
• Pointwise confidence intervals for the true log-concave pmf can be reliably constructed using the asymptotic distribution of the MLE.
• The method is computationally feasible and efficiently implemented in the R package logcondiscr for practical use.
• Empirical application to H1N1 data from Ontario, Canada, demonstrates the method’s ability to provide smooth, shape-constrained estimates with reliable uncertainty quantification.
• The log-concavity constraint enhances estimation accuracy and interpretability compared to unconstrained nonparametric methods.