[Paper Review] CUQIpy: I. Computational uncertainty quantification for inverse problems in Python

Nicolai A. B. Riis, Amal Alghamdi+5 more|arXiv (Cornell University)|May 26, 2023

Gaussian Processes and Bayesian Inference53 references4 citations

TL;DR

CUQIpy is an open-source Python package for computational uncertainty quantification (UQ) in inverse problems using a Bayesian framework. It enables users to model inverse problems with intuitive syntax, automatically selects optimal sampling strategies, and supports large-scale imaging applications like CT and image deblurring through efficient, structure-aware sampling and integration with libraries like PyTorch and FEniCS.

ABSTRACT

This paper introduces CUQIpy, a versatile open-source Python package for computational uncertainty quantification (UQ) in inverse problems, presented as Part I of a two-part series. CUQIpy employs a Bayesian framework, integrating prior knowledge with observed data to produce posterior probability distributions that characterize the uncertainty in computed solutions to inverse problems. The package offers a high-level modeling framework with concise syntax, allowing users to easily specify their inverse problems, prior information, and statistical assumptions. CUQIpy supports a range of efficient sampling strategies and is designed to handle large-scale problems. Notably, the automatic sampler selection feature analyzes the problem structure and chooses a suitable sampler without user intervention, streamlining the process. With a selection of probability distributions, test problems, computational methods, and visualization tools, CUQIpy serves as a powerful, flexible, and adaptable tool for UQ in a wide selection of inverse problems. Part II of the series focuses on the use of CUQIpy for UQ in inverse problems with partial differential equations (PDEs).

Motivation & Objective

• Address the growing need for scalable and accessible uncertainty quantification in large-scale inverse problems across scientific and engineering domains.
• Provide a user-friendly yet extensible framework that allows non-experts to model Bayesian inverse problems while giving experts full control over sampling and computational methods.
• Bridge the gap between theoretical UQ and practical implementation by supporting advanced sampling strategies tailored to problem structure, such as linearity and distributional relationships.
• Enable interoperability with modern array libraries (e.g., PyTorch) and third-party tools via a plugin architecture for enhanced performance and flexibility.
• Facilitate the adoption of UQ in real-world applications such as medical imaging and geoscience by offering built-in test problems, visualization tools, and efficient computational pipelines.

Proposed method

• Formulates inverse problems within a Bayesian framework, combining prior knowledge with noisy data to produce posterior distributions.
• Employs a high-level modeling interface with syntax closely aligned to mathematical notation, allowing concise specification of random variables and hierarchical models.
• Integrates automatic sampler selection via the `BayesianProblem` class, which analyzes problem structure and selects optimal sampling methods (e.g., NUTS, Hamiltonian Monte Carlo) without user intervention.
• Supports array-agnostic computation, enabling seamless use of NumPy, PyTorch, or other backends for GPU acceleration and automatic differentiation.
• Provides a plugin system to extend functionality with external libraries such as CIL (for CT), FEniCS (for finite elements), and PyTorch (for automatic differentiation).
• Utilizes advanced sampling techniques like the No-U-Turn Sampler (NUTS) with gradient information to efficiently explore complex posterior distributions in high-dimensional spaces.

Figure 1 : Figures generated by CUQIpy ’s automatic UQ analysis of the built-in 2D deconvolution test problem Deconvolution2D with a blurred cookie. Far left: True sharp image. Middle left: Observed blurred and noisy image. Middle right: Deblurred image (posterior mean). Far right: Quantification of

Experimental results

Research questions

• RQ1 How can a unified, accessible, and extensible software framework be designed to support uncertainty quantification in large-scale inverse problems?
• RQ2 What role does automated sampler selection play in improving usability and performance for non-expert users in Bayesian inverse problems?
• RQ3 How can a probabilistic programming interface be designed to balance high-level abstraction with low-level control for expert users?
• RQ4 To what extent can integration with modern computational backends (e.g., PyTorch) enhance performance and scalability in UQ workflows?
• RQ5 Can a modular plugin architecture effectively extend the reach of a core UQ library to diverse application domains like medical imaging and PDE-constrained problems?

Key findings

• CUQIpy successfully enables users to model and solve Bayesian inverse problems with minimal code, using intuitive syntax that mirrors mathematical notation.
• The automatic sampler selection feature effectively identifies and applies appropriate sampling strategies—such as NUTS—based on problem structure, reducing user burden and improving reliability.
• Using the Pyro-powered `cuqipypyTorch` plugin, the package achieves efficient joint sampling of complex hierarchical models, such as the Eight Schools problem, with 1000 posterior samples and 500 warm-up samples.
• The average treatment effect in the Eight Schools problem was estimated at 5.6 with a standard deviation of 12.7, consistent with results from established software, validating the framework’s accuracy.
• The integration with PyTorch enables automatic differentiation and GPU acceleration, significantly improving performance for large-scale problems.
• The framework supports a wide range of probability distributions, computational methods, and visualization tools, including violin plots for posterior analysis, enhancing interpretability and reproducibility.