[Paper Review] Neural Bayes estimators for censored inference with peaks-over-threshold models

J. Ian Richards, Matthew Sainsbury-Dale+2 more|arXiv (Cornell University)|Jun 27, 2023

Statistical Methods and Bayesian Inference4 citations

TL;DR

This paper proposes neural Bayes estimators (NBEs) that enable fast, likelihood-free inference for censored peaks-over-threshold models in spatial extremes by embedding censoring information directly into convolutional neural network architectures. The method achieves substantial gains in computational and statistical efficiency over traditional likelihood-based approaches, enabling real-time fitting of high-dimensional spatial extremal models—demonstrated on PM₂.₅ data across Saudi Arabia with inference in milliseconds per model post-training.

ABSTRACT

Making inference with spatial extremal dependence models can be computationally burdensome since they involve intractable and/or censored likelihoods. Building on recent advances in likelihood-free inference with neural Bayes estimators, that is, neural networks that approximate Bayes estimators, we develop highly efficient estimators for censored peaks-over-threshold models that {use data augmentation techniques} to encode censoring information in the neural network {input}. Our new method provides a paradigm shift that challenges traditional censored likelihood-based inference methods for spatial extremal dependence models. Our simulation studies highlight significant gains in both computational and statistical efficiency, relative to competing likelihood-based approaches, when applying our novel estimators to make inference with popular extremal dependence models, such as max-stable, $r$-Pareto, and random scale mixture process models. We also illustrate that it is possible to train a single neural Bayes estimator for a general censoring level, precluding the need to retrain the network when the censoring level is changed. We illustrate the efficacy of our estimators by making fast inference on hundreds-of-thousands of high-dimensional spatial extremal dependence models to assess extreme particulate matter 2.5 microns or less in diameter (${ m PM}_{2.5}$) concentration over the whole of Saudi Arabia.

Motivation & Objective

• To address the computational burden of likelihood-based inference in spatial extremal dependence models with censored data.
• To develop a neural network-based approach that approximates Bayes estimators without requiring explicit likelihood evaluation.
• To enable amortized, reusable inference for high-dimensional spatial extremes under various censoring levels.
• To demonstrate scalability and statistical efficiency on real-world environmental data, such as PM₂.₅ concentrations.
• To explore the feasibility of training a single NBE for arbitrary censoring thresholds, reducing retraining needs.

Proposed method

• Design a convolutional neural network (CNN) architecture that explicitly encodes censoring thresholds as part of the input representation.
• Train the network to map spatial extremal data (with censored values) to posterior mean estimates of model parameters, approximating Bayes estimators.
• Use a simulation-based training framework to generate synthetic data under known parameters and censoring levels, enabling end-to-end learning.
• Incorporate a user-defined censoring probability τ to generalize the estimator across different censoring levels without retraining.
• Leverage the amortized nature of neural networks to enable rapid inference—single forward pass per model—post-training.
• Adapt the network to handle both stationary and anisotropic spatial dependence structures in a locally stationary framework.

Experimental results

Research questions

• RQ1 Can neural Bayes estimators outperform traditional likelihood-based inference in terms of computational speed and statistical efficiency for censored spatial extremes?
• RQ2 Can a single neural estimator be trained to handle arbitrary censoring levels without retraining?
• RQ3 How well do NBEs perform on high-dimensional spatial extremal models with complex dependence structures like max-stable, r-Pareto, and random scale mixture processes?
• RQ4 Can NBEs enable large-scale, real-time inference on massive spatial datasets, such as nationwide PM₂.₅ concentration modeling?
• RQ5 What are the limitations of the current framework in handling irregular domains, non-stationary processes, or high-dimensional parameter spaces?

Key findings

• The proposed neural Bayes estimators achieve significant computational speedups, enabling inference on hundreds of thousands of high-dimensional spatial models in a matter of milliseconds per model post-training.
• The method demonstrates superior statistical efficiency compared to likelihood-based approaches, particularly in settings with high censoring and complex extremal dependence.
• A single NBE can be trained for an arbitrary censoring threshold τ, eliminating the need for retraining when the censoring level changes.
• The framework successfully enables large-scale bootstrap studies with complex extremal models that were previously computationally infeasible.
• Application to PM₂.₅ data across Saudi Arabia reveals new insights into spatial extremal dependence at an unprecedented scale and resolution.
• The approach is extensible to other censored data problems, including survival analysis, by adapting the network architecture and input representation.