[Paper Review] Deep Learning and Data Assimilation for Real-Time Production Prediction in Natural Gas Wells

Kelvin Loh, Pejman Shoeibi Omrani+1 more|arXiv (Cornell University)|Feb 14, 2018

Reservoir Engineering and Simulation Methods18 citations

TL;DR

This paper proposes a hybrid deep learning and data assimilation framework using a modified Long Short-Term Memory (LSTM) network enhanced with an Ensemble Kalman Filter (EnKF) for real-time prediction of gas flow rates in mature North Sea gas wells. The approach improves prediction accuracy and robustness by continuously updating model parameters using real-time observations, reducing prediction error by up to 50% in challenging operational conditions with salt deposition and dynamic flow behavior.

ABSTRACT

The prediction of the gas production from mature gas wells, due to their complex end-of-life behavior, is challenging and crucial for operational decision making. In this paper, we apply a modified deep LSTM model for prediction of the gas flow rates in mature gas wells, including the uncertainties in input parameters. Additionally, due to changes in the system in time and in order to increase the accuracy and robustness of the prediction, the Ensemble Kalman Filter (EnKF) is used to update the flow rate predictions based on new observations. The developed approach was tested on the data from two mature gas production wells in which their production is highly dynamic and suffering from salt deposition. The results show that the flow predictions using the EnKF updated model leads to better Jeffreys' J-divergences than the predictions without the EnKF model updating scheme.

Motivation & Objective

• To address the challenge of predicting declining gas flow rates in mature, end-of-life gas wells suffering from salt deposition and complex dynamics.
• To develop a real-time, deployable prediction system that is robust to unknown field conditions and requires minimal retraining.
• To integrate deep learning with data assimilation techniques to enhance model accuracy and adaptability over time.
• To evaluate the performance of an EnKF-updated LSTM model against a baseline LSTM model on real-world field data from two mature gas wells.

Proposed method

• A modified deep LSTM model is trained on historical production data from Well A (Jan 2009–Dec 2011) to learn temporal dependencies and production decline trends.
• Input variables—flow rate, pressure, temperature, and choke valve settings—are normalized using min-max scaling, with measurement uncertainties preserved via sensor accuracy-based standard deviation scaling.
• The Ensemble Kalman Filter (EnKF) is applied online to update the LSTM model’s bias parameters using new real-time observations, enabling continuous model correction.
• The EnKF framework treats the LSTM model as a state-space system, where model parameters are updated based on the innovation (difference between observed and predicted values) using ensemble-based error covariance estimation.
• The approach is tested on Well B (2012–2013), where the trained model is applied without retraining, demonstrating generalization and adaptability to unseen field conditions.
• Model performance is quantitatively evaluated using Jeffreys’ J-divergence, with lower values indicating better predictive accuracy and distributional alignment.

Experimental results

Research questions

• RQ1 Can a deep LSTM model trained on one mature gas well generalize to predict production in a second, unseen well with different physical characteristics?
• RQ2 How does online model updating via EnKF improve prediction accuracy and robustness in the presence of unmodeled system changes and measurement uncertainty?
• RQ3 To what extent does the EnKF-based parameter update reduce prediction bias and improve temporal tracking of actual flow rates in dynamic, salt-prone well conditions?
• RQ4 Does the EnKF-updated model maintain superior performance during initial spin-up and under high uncertainty, compared to a static LSTM baseline?

Key findings

• The EnKF-updated LSTM model achieved significantly lower median J-divergence (283.1) than the baseline model during the most challenging period (March 10–24, 2013), indicating superior predictive accuracy.
• The EnKF model reduced prediction bias and improved tracking of actual flow rates, especially during periods of high dynamic change and salt deposition-induced decline.
• Even during the initial EnKF spin-up phase, the updated model outperformed the baseline, demonstrating robustness and minimal delay in convergence.
• The framework successfully adapted to Well B’s unknown physical characteristics without retraining, proving generalization capability across different field conditions.
• The median J-divergence was consistently lower across all tested periods (July 13–27, 2012; March 10–24, 2013; Jan 1–March 31, 2013), confirming sustained performance improvement.
• The approach enables a 'deploy and forget' operational model, as the EnKF continuously corrects model drift and uncertainty, reducing reliance on manual retraining.