[Paper Review] Physics-Informed Machine Learning with Conditional Karhunen-Loève Expansions

Alexandre M. Tartakovsky, David A. Barajas‐Solano+1 more|arXiv (Cornell University)|Dec 4, 2019

Model Reduction and Neural Networks45 references4 citations

TL;DR

This paper introduces PICKLE, a physics-informed machine learning method that uses conditional Karhunen-Loève expansions (cKLEs) to reconstruct spatially heterogeneous parameters and states from sparse measurements. By conditioning KLEs on observations and minimizing PDE residual norms, PICKLE achieves higher accuracy than MAP and PINN methods, especially for discontinuous and rough fields.

ABSTRACT

We present a new physics-informed machine learning approach for the inversion of PDE models with heterogeneous parameters. In our approach, the space-dependent partially-observed parameters and states are approximated via Karhunen-Loève expansions (KLEs). Each of these KLEs is then conditioned on their corresponding measurements, resulting in low-dimensional models of the parameters and states that resolve observed data. Finally, the coefficients of the KLEs are estimated by minimizing the norm of the residual of the PDE model evaluated at a finite set of points in the computational domain, ensuring that the reconstructed parameters and states are consistent with both the observations and the PDE model to an arbitrary level of accuracy. In our approach, KLEs are constructed using the eigendecomposition of covariance models of spatial variability. For the model parameters, we employ a parameterized covariance model calibrated on parameter observations; for the model states, the covariance is estimated from a number of forward simulations of the PDE model corresponding to realizations of the parameters drawn from their KLE. We apply the proposed approach to identifying heterogeneous log-diffusion coefficients in diffusion equations from spatially sparse measurements of the log-diffusion coefficient and the solution of the diffusion equation. We find that the proposed approach compares favorably against state-of-the-art point estimates such as maximum a posteriori estimation and physics-informed neural networks.

Motivation & Objective

• Address the inverse problem of estimating space-dependent parameters and states in PDE models from sparse, noisy measurements.
• Overcome the ill-posedness of PDE-constrained inverse problems through low-dimensional, data-consistent parameterizations.
• Develop a method that ensures consistency with both observed data and the underlying PDE model to arbitrary accuracy.
• Improve upon existing methods like MAP and physics-informed neural networks (PINNs) in reconstructing fields with complex spatial variability, including discontinuities.
• Enable robust estimation of parameters and states regardless of their underlying statistical distribution, including non-Gaussian fields.

Proposed method

• Approximate spatially heterogeneous parameters and states using Karhunen-Loève expansions (KLEs), which represent random fields as weighted sums of eigenfunctions derived from covariance operators.
• Condition the KLEs on sparse measurements via conditional Gaussian processes, resulting in cKLEs that exactly honor observed data.
• Construct parameter covariance models using a parameterized kernel (e.g., Matérn) calibrated on observed parameter data.
• Estimate state covariance from forward PDE simulations using realizations of parameters sampled from their cKLE representation.
• Optimize cKLE coefficients by minimizing the L2 norm of the PDE residual evaluated at a finite set of collocation points, enforcing PDE consistency.
• Use a regularized optimization framework where the objective combines data misfit (weighted by observation error covariance) and PDE residual minimization.

Experimental results

Research questions

• RQ1 Can cKLE-based parameterization improve reconstruction accuracy for PDE inverse problems with sparse, noisy measurements compared to standard MAP estimation?
• RQ2 How does the PICKLE method perform in reconstructing discontinuous or rough spatial fields that violate Gaussian assumptions?
• RQ3 To what extent does the cKLE representation outperform deep neural networks (e.g., PINNs) in capturing spatially correlated fields with limited data?
• RQ4 How sensitive is the accuracy of PICKLE to the quality and number of parameter and state measurements?
• RQ5 Can cKLEs effectively model non-Gaussian fields, such as discontinuous diffusion coefficients or bounded-domain solutions, when the underlying field is not Gaussian?

Key findings

• PICKLE reduces the relative ℓ₁ error in estimating a discontinuous log-diffusion coefficient to 0.179, which is more than twice lower than the MAP estimate’s error of 0.380.
• For continuous, rough log-diffusion fields, PICKLE achieves higher accuracy than both maximum a posteriori (MAP) estimation and physics-informed neural networks (PINNs).
• The method successfully reconstructs fields with non-Gaussian statistics, including discontinuous conductivity fields and bounded-domain solutions, without requiring Gaussian assumptions.
• The accuracy of PICKLE depends on the fidelity of the estimated parameter covariance, which improves with more measurements.
• Transfer learning can be used to estimate parameter covariance from similar systems, enabling generalization across domains with similar spatial variability.
• cKLEs provide a more effective low-dimensional representation of spatial fields than neural networks, especially under data scarcity, due to their intrinsic physical consistency and structure.