[Paper Review] Towards Global Remote Discharge Estimation: Using the Few to Estimate The Many

Yotam Gigi, Gal Elidan+6 more|arXiv (Cornell University)|Jan 3, 2019

Flood Risk Assessment and Management19 references3 citations

TL;DR

This paper proposes a Common Mechanism Regression (CMR) model to enable global remote river discharge estimation using sparse in-situ measurements and satellite data. By combining a shared global mechanism with local site-specific components, and using spectral initialization to achieve global optimization of a non-convex objective, the method significantly improves discharge prediction accuracy—demonstrated by a 13% reduction in test MSE compared to NDWI-based per-site regression.

ABSTRACT

Learning hydrologic models for accurate riverine flood prediction at scale is a challenge of great importance. One of the key difficulties is the need to rely on in-situ river discharge measurements, which can be quite scarce and unreliable, particularly in regions where floods cause the most damage every year. Accordingly, in this work we tackle the problem of river discharge estimation at different river locations. A core characteristic of the data at hand (e.g. satellite measurements) is that we have few measurements for many locations, all sharing the same physics that underlie the water discharge. We capture this scenario in a simple but powerful common mechanism regression (CMR) model with a local component as well as a shared one which captures the global discharge mechanism. The resulting learning objective is non-convex, but we show that we can find its global optimum by leveraging the power of joining local measurements across sites. In particular, using a spectral initialization with provable near-optimal accuracy, we can find the optimum using standard descent methods. We demonstrate the efficacy of our approach for the problem of discharge estimation using simulations.

Motivation & Objective

• Address the critical challenge of sparse in-situ river discharge measurements in flood-prone developing regions.
• Enable scalable, automated, and globally applicable river discharge estimation to support early flood warning systems.
• Overcome the limitations of per-site models by leveraging shared physical mechanisms across river locations.
• Develop a machine learning framework that generalizes from few measurements at many sites to estimate discharge at scale.
• Achieve global optimization of a non-convex learning objective through spectral initialization and descent methods.

Proposed method

• Formulate a regression model with two components: a shared global weight vector (w̄) capturing the universal discharge mechanism and local weight vectors (vi) for site-specific adaptation.
• Model the learning objective as a non-convex optimization problem that jointly estimates w̄ and vi across I sites with T samples per site.
• Use spectral initialization to approximate the true shared mechanism w̄ with high probability, leveraging the concentration of the sample covariance matrix.
• Apply gradient descent or alternating least squares to converge to the global optimum after spectral initialization.
• Ensure theoretical convergence guarantees by proving that the principal eigenvector of the sample matrix Q approximates w̄ under i.i.d. local regressors and sufficient data (IT ≥ CP²B²/ε²).
• Integrate satellite imagery (e.g., LANDSAT8) and USGS discharge data to train and validate the CMR model in real-world settings.

Experimental results

Research questions

• RQ1 Can a shared physical mechanism across river sites be effectively learned from few measurements at many locations?
• RQ2 Does spectral initialization enable global optimization of the non-convex CMR objective in remote discharge estimation?
• RQ3 To what extent does the CMR model improve discharge prediction accuracy compared to per-site models like NDWI?
• RQ4 Can the CMR model generalize effectively when T < P (samples per site less than feature dimension), where local models fail?
• RQ5 How does the product IT (number of sites × samples per site) affect the recovery accuracy of the shared mechanism w̄?

Key findings

• The CMR model with spectral initialization recovers the true shared mechanism w̄ with high accuracy when √(IT) ≥ CP²B²/ε², as proven by the theoretical concentration bound.
• Spectral initialization significantly improves recovery success rates, especially in low-data regimes (e.g., T < P), where standard methods fail.
• In synthetic simulations, CMR achieves >90% success in recovering w̄ when I > B and T > P, and even succeeds when T < P due to shared mechanism learning.
• In real-world experiments using 23 USGS gauge sites and LANDSAT8 data, CMR reduces normalized test MSE from 0.70 (NDWI) to 0.65, a 7.1% relative improvement.
• The CMR model achieves a 13% relative reduction in test MSE compared to NDWI (0.47 vs. 0.54 for training, 0.65 vs. 0.70 for test), demonstrating strong generalization.
• The shared component of CMR provides substantial predictive gains even when the model includes expressive local components, indicating the value of transfer learning across sites.