[Paper Review] Living in the Physics and Machine Learning Interplay for Earth Observation

Gustau Camps‐Valls, Daniel Heestermans Svendsen+8 more|arXiv (Cornell University)|Oct 18, 2020

Scientific Computing and Data Management44 references4 citations

TL;DR

This paper proposes a hybrid framework integrating physics-based models with machine learning to improve the consistency, credibility, and generalization of Earth observation systems. By embedding physical laws—such as conservation of mass and energy—into data-driven models through constrained learning, physics-aware neural networks achieve better generalization, reduced overfitting, and improved interpretability, with empirical results showing a 1000-fold reduction in parameter estimation error compared to standard methods.

ABSTRACT

Most problems in Earth sciences aim to do inferences about the system, where accurate predictions are just a tiny part of the whole problem. Inferences mean understanding variables relations, deriving models that are physically interpretable, that are simple parsimonious, and mathematically tractable. Machine learning models alone are excellent approximators, but very often do not respect the most elementary laws of physics, like mass or energy conservation, so consistency and confidence are compromised. In this paper, we describe the main challenges ahead in the field, and introduce several ways to live in the Physics and machine learning interplay: to encode differential equations from data, constrain data-driven models with physics-priors and dependence constraints, improve parameterizations, emulate physical models, and blend data-driven and process-based models. This is a collective long-term AI agenda towards developing and applying algorithms capable of discovering knowledge in the Earth system.

Motivation & Objective

• To address the lack of physical consistency in purely data-driven machine learning models used in Earth sciences.
• To improve model credibility and generalization by embedding fundamental physical laws—such as energy and mass conservation—into machine learning architectures.
• To reduce overfitting and data requirements by constraining the parameter space using physical priors.
• To enable robust parameter estimation and uncertainty quantification in complex Earth system models through physics-aware Bayesian inference.
• To develop a hybrid AI agenda that unifies data-driven and process-based modeling for scientific discovery in Earth observation.

Proposed method

• Uses physics-informed neural networks (PINNs) to embed differential equations and conservation laws directly into the loss function of machine learning models.
• Applies variational inference and Gibbs sampling with the FUSS algorithm to efficiently sample from complex, high-dimensional posterior distributions in inverse modeling.
• Employs kernel-based distribution matching and dependence constraints to enforce statistical consistency between data-driven and physical models.
• Utilizes model emulation to replace computationally expensive physical simulators with fast, differentiable surrogate models.
• Integrates domain knowledge via prior constraints in Bayesian frameworks, limiting the search space to physically plausible solutions.
• Combines forward and inverse modeling using variational approaches to infer latent functions, driving forces, and parameters simultaneously.

Experimental results

Research questions

• RQ1 How can machine learning models be made physically consistent while maintaining high predictive accuracy in Earth observation?
• RQ2 What are effective methods to embed conservation laws and differential equations into data-driven models?
• RQ3 Can physics-aware machine learning reduce overfitting and improve generalization with less training data?
• RQ4 How can Bayesian inference be efficiently performed in high-dimensional, complex posterior distributions arising in Earth system modeling?
• RQ5 To what extent can hybrid models improve uncertainty quantification and parameter estimation in physical systems?

Key findings

• Physics-aware machine learning models achieve significantly better generalization and reduced overfitting by constraining the parameter space to physically plausible regions.
• The FUSS-within-Gibbs sampler reduces mean squared error (MSE) in parameter estimation by over 1000-fold compared to standard Metropolis-Hastings within Gibbs sampling for chaotic systems.
• Empirical results show an MSE of order 10^-4 in estimating parameters R and Ω in a noisy logistic map, compared to an MSE of ~0.65 with standard MH-within-Gibbs.
• The integration of physics into machine learning leads to sparser, more interpretable models that require less training data for similar performance.
• Physics-informed constraints improve model credibility and enable extrapolation beyond observed data, addressing a key limitation of pure data-driven models.
• The hybrid modeling framework enhances computational efficiency and supports uncertainty quantification, making it suitable for complex Earth system applications.