[Paper Review] Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles

Xiaowei Jia, Jared Willard+5 more|arXiv (Cornell University)|Jan 28, 2020

Model Reduction and Neural Networks39 references41 citations

TL;DR

The paper proposes Physics-Guided Recurrent Neural Networks (PGRNN) that integrate energy conservation and physics-based models with LSTM to predict lake temperature, achieving higher accuracy with limited data and ensuring physical consistency.

ABSTRACT

Physics-based models of dynamical systems are often used to study engineering and environmental systems. Despite their extensive use, these models have several well-known limitations due to simplified representations of the physical processes being modeled or challenges in selecting appropriate parameters. While-state-of-the-art machine learning models can sometimes outperform physics-based models given ample amount of training data, they can produce results that are physically inconsistent. This paper proposes a physics-guided recurrent neural network model (PGRNN) that combines RNNs and physics-based models to leverage their complementary strengths and improves the modeling of physical processes. Specifically, we show that a PGRNN can improve prediction accuracy over that of physics-based models, while generating outputs consistent with physical laws. An important aspect of our PGRNN approach lies in its ability to incorporate the knowledge encoded in physics-based models. This allows training the PGRNN model using very few true observed data while also ensuring high prediction accuracy. Although we present and evaluate this methodology in the context of modeling the dynamics of temperature in lakes, it is applicable more widely to a range of scientific and engineering disciplines where physics-based (also known as mechanistic) models are used, e.g., climate science, materials science, computational chemistry, and biomedicine.

Motivation & Objective

• Motivate improved modeling of engineering/environmental systems by combining physics-based models with machine learning.
• Address limitations of pure physics-based models (bias, parameter calibration) and pure black-box ML (physical inconsistency, data hunger).
• Develop a framework (PGRNN) that preserves physical laws while leveraging data-driven learning to predict lake temperatures across depths and time.
• Demonstrate data-efficient training via physics-based pre-training using synthetic GLM outputs.
• Validate the approach on Lake Mendota, assessing both predictive accuracy and physical consistency.

Proposed method

• Use LSTM-based RNNs to model daily lake temperature dynamics across depths with a global model that takes depth as an input feature.
• Incorporate energy conservation by introducing an energy flow constraint, linking predicted temperatures to incoming/outgoing heat fluxes and lake energy, and add a physics-based penalty to the loss: L = L_RNN + lambda_EC * L_EC.
• Compute the lake thermal energy U_t from temperatures, layer areas, densities, and thickness, and enforce Delta U_t ≈ F_in − F_out during ice-free periods via a ReLU-based penalty with threshold tau_EC.
• Pre-train the PGRNN on synthetic data generated by a generic GLM to inject physical realism and improve initialization, then fine-tune with limited observations.
• Evaluate under varying training data fractions to test data efficiency, and compare to GLM and standard RNN baselines.
• Implement and test on Lake Mendota with 21 hidden units, using a 0.5 m depth discretization and specific hyperparameters (tau_EC = 24 W/m^2, lambda_EC = 0.01, learning rate 0.005).

Experimental results

Research questions

• RQ1 Can a physics-guided RNN improve lake temperature prediction over traditional physics-based models and pure ML when data are scarce?
• RQ2 Does enforcing energy conservation within an RNN improve physical consistency without sacrificing accuracy?
• RQ3 Does pre-training with synthetic physics-based simulations reduce data requirements for accurate predictions?
• RQ4 How well does PGRNN generalize across depths and seasons for lake systems?
• RQ5 Is the approach applicable beyond lake temperature to other physics-driven domains?

Key findings

• PGRNN (RNN with energy conservation) consistently outperforms a plain RNN, especially with limited observed data.
• With as little as 20% of observed data, RGRNN EC achieves RMSE comparable to or better than GLM calibrated with 100% data, and 20% RNN EC outperforms fully calibrated GLM.
• Energy-consistency training reduces the mismatch between energy flux balance and lake energy change relative to GLM and plain RNN.
• Pre-training with synthetic GLM data enables high accuracy with very few real observations, and helps initialize weights closer to optimal solutions.
• The energy conservation term improves physical plausibility and generalization, while the approach remains trainable with semi-supervised data (no labels required for flux calculations).
• Experiments on Lake Mendota show the method captures depth-dependent temperature dynamics and seasonal patterns, demonstrating practical feasibility.