[Paper Review] Nonparametric Estimation of Band-limited Probability Density Functions

Rahul Agarwal, Zhe Chen+1 more|arXiv (Cornell University)|Mar 20, 2015

Neural dynamics and brain function17 references3 citations

TL;DR

This paper proposes a nonparametric maximum likelihood estimator for band-limited probability density functions (BLML), which is consistent, computationally efficient, and outperforms kernel density estimation (KDE) in both accuracy and speed. The method leverages the band-limited assumption and Nyquist sampling to enable fast computation via BLMLQuick, achieving a convergence rate of O(n⁻¹) and superior performance on synthetic and neuronal spike train data.

ABSTRACT

In this paper, a nonparametric maximum likelihood (ML) estimator for band-limited (BL) probability density functions (pdfs) is proposed. The BLML estimator is consistent and computationally efficient. To compute the BLML estimator, three approximate algorithms are presented: a binary quadratic programming (BQP) algorithm for medium scale problems, a Trivial algorithm for large-scale problems that yields a consistent estimate if the underlying pdf is strictly positive and BL, and a fast implementation of the Trivial algorithm that exploits the band-limited assumption and the Nyquist sampling theorem ("BLMLQuick"). All three BLML estimators outperform kernel density estimation (KDE) algorithms (adaptive and higher order KDEs) with respect to the mean integrated squared error for data generated from both BL and infinite-band pdfs. Further, the BLMLQuick estimate is remarkably faster than the KD algorithms. Finally, the BLML method is applied to estimate the conditional intensity function of a neuronal spike train (point process) recorded from a rat's entorhinal cortex grid cell, for which it outperforms state-of-the-art estimators used in neuroscience.

Motivation & Objective

• To develop a nonparametric maximum likelihood estimator for probability density functions that is consistent and efficiently computable.
• To address the limitations of kernel density estimation (KDE), including slow convergence rates and arbitrary kernel selection.
• To exploit the band-limited assumption as a smoothness constraint that enables existence of a global likelihood maximum without penalization.
• To design computationally efficient algorithms—BLMLQuick, BLML-BQP, and BLMLTrivial—for large-scale and medium-scale data.
• To demonstrate superior performance on both synthetic data and real neuronal spike train data from rat grid cells.

Proposed method

• Proposes the BLML estimator as the solution to a constrained likelihood maximization problem under the band-limited assumption, where the pdf has finite support in the Fourier domain.
• Derives the BLML estimator as a squared sinc-weighted sum of empirical data points, with coefficients solved via a system of nonlinear equations: ρₙ(c) = 0.
• Introduces three algorithms: BLML-BQP (binary quadratic programming) for medium-scale problems, BLMLTrivial for large-scale data under strict positivity, and BLMLQuick for optimal speed.
• Employs the Nyquist sampling theorem in BLMLQuick by binning data into discrete bins at a sampling rate fₛ > 2f_c, enabling fast computation via block-Toeplitz structure and low-rank approximation.
• Uses convex optimization (e.g., Newton’s method) with self-concordant cost functions to solve the coefficient estimation problem efficiently.
• Applies the Akaike algorithm to invert structured matrices in O(B²) time, with further acceleration via low-rank approximation (R ≈ 20 for B = 1000).

Experimental results

Research questions

• RQ1 Can a nonparametric maximum likelihood estimator exist for band-limited probability density functions without requiring likelihood penalization?
• RQ2 Does the BLML estimator achieve faster convergence rates than existing KDE methods, particularly in terms of mean integrated squared error (MISE)?
• RQ3 Can the band-limited assumption be exploited to design a computationally efficient estimator that maintains consistency and smoothness?
• RQ4 How does BLML perform on real neuronal spike train data compared to state-of-the-art estimators in neuroscience?
• RQ5 What is the computational complexity of the BLML estimator, and can it scale efficiently to large datasets while preserving accuracy?

Key findings

• The BLML estimator achieves a convergence rate of O(n⁻¹) in mean integrated squared error (MISE), matching parametric maximum likelihood estimators, and outperforms all tested KDEs (including 2nd and 6th-order Gaussian and sinc kernels).
• BLMLQuick is significantly faster than all KDE algorithms, with computational complexity O(n + f_c²n^(2/(r−1)+0.5) + f_cln^(1/(r−1)+0.25)), and achieves O(n⁻¹) convergence under appropriate sampling conditions.
• On synthetic data from both band-limited and infinite-band pdfs, BLML consistently outperforms adaptive and higher-order KDEs in MISE and computational time.
• On rat entorhinal cortex grid cell spike train data, BLML outperforms state-of-the-art estimators used in neuroscience, demonstrating practical superiority in real-world applications.
• The BLML estimator remains consistent even when the underlying pdf is strictly positive and band-limited, and the BLMLQuick variant maintains consistency under the same conditions.
• The BLML estimator does not require likelihood penalization, preserving the asymptotic properties of maximum likelihood estimates such as consistency, asymptotic normality, and efficiency.