[Paper Review] Penalized maximum likelihood estimation for generalized linear point processes

Niels Richard Hansen|arXiv (Cornell University)|Mar 3, 2010

Point processes and geometric inequalities34 references3 citations

TL;DR

This paper develops a penalized maximum likelihood estimation framework for generalized linear point processes with functional predictors in Sobolev spaces, leveraging Gâteaux differentiability and reproducing kernel Hilbert space theory. It establishes that the penalized estimator lies in a finite-dimensional spline-like space in special cases and proposes a descent algorithm via gradient flow in infinite-dimensional Sobolev spaces, with convergence guarantees and applications to neural spike trains and genomics.

ABSTRACT

A generalized linear point process is specified in terms of an intensity that depends upon a linear predictor process through a fixed non-linear function. We present a framework where the linear predictor is parametrized by a Banach space and give results on Gateaux differentiability of the log-likelihood. Of particular interest is when the intensity is expressed in terms of a linear filter parametrized by a Sobolev space. Using that the Sobolev spaces are reproducing kernel Hilbert spaces we derive results on the representation of the penalized maximum likelihood estimator in a special case and the gradient of the negative log-likelihood in general. The latter is used to develop a descent algorithm in the Sobolev space. We conclude the paper by extensions to multivariate and additive model specifications. The methods are implemented in the R-package ppstat.

Motivation & Objective

• To develop a theoretical framework for penalized maximum likelihood estimation in generalized linear point processes with functional predictors in Banach spaces.
• To establish Gâteaux differentiability of the log-likelihood for intensity functions parameterized by Sobolev spaces.
• To derive a finite-dimensional representation of the penalized MLE in a special case, analogous to smoothing splines.
• To propose a descent algorithm in infinite-dimensional Sobolev spaces using the gradient of the negative log-likelihood.
• To extend the framework to multivariate and additive point process models for applications in neuroscience and genomics.

Proposed method

• Parametrize the linear predictor of the point process intensity using elements from a Sobolev space, a reproducing kernel Hilbert space.
• Use Gâteaux differentiation to derive the gradient of the negative log-likelihood functional in the Sobolev space.
• Express the log-likelihood and its gradient as continuous linear functionals via stochastic integrals of the predictor with respect to counting processes.
• Establish convergence of a descent algorithm by proving the gradient norm tends to zero under weak compactness and weak continuity assumptions.
• Leverage the reproducing kernel property of Sobolev spaces to represent the penalized MLE in a finite-dimensional subspace spanned by explicit basis functions in the special case.
• Implement the algorithm as a sequence of finite-dimensional approximations, with convergence to the true solution in the weak topology.

Experimental results

Research questions

• RQ1 Can penalized maximum likelihood estimation be rigorously formulated for generalized linear point processes with functional predictors in Sobolev spaces?
• RQ2 Under what conditions does the penalized MLE lie in a finite-dimensional subspace, and what is the structure of the basis functions in such cases?
• RQ3 How can the gradient of the negative log-likelihood be represented and computed in infinite-dimensional Sobolev spaces?
• RQ4 Can a descent algorithm be constructed and proven to converge for this class of nonparametric point process models?
• RQ5 What theoretical guarantees (e.g., convergence, boundedness) can be established for the penalized MLE in this framework?

Key findings

• The penalized maximum likelihood estimator lies in a finite-dimensional subspace spanned by explicitly defined basis functions in a special case, analogous to smoothing splines.
• The negative log-likelihood is bounded below on bounded sets in the Sobolev space, ensuring the existence of minimizers.
• The gradient of the negative log-likelihood is sequentially weak-weak continuous, which supports convergence analysis of iterative algorithms.
• The descent algorithm based on gradient steps converges weakly to the true minimizer due to the boundedness and weak compactness of the parameter space.
• Weak convergence in the reproducing kernel Hilbert space implies pointwise convergence of the estimated intensity function.
• The framework supports extensions to multivariate and additive models, enabling applications to complex point process data such as neuronal spike trains and genomic regulatory elements.