[Paper Review] Continuous Field Reconstruction from Sparse Observations with Implicit Neural Networks

Xihaier Luo, Wei Xu+3 more|arXiv (Cornell University)|Jan 21, 2024

Meteorological Phenomena and Simulations6 citations

TL;DR

The paper introduces MMGN, an implicit neural representation model that reconstructs continuous spatiotemporal fields from sparse observations using a context-aware latent code and a multiplicative Gabor-based decoder, outperforming other INR baselines on climate and sea surface temperature data.

ABSTRACT

Reliably reconstructing physical fields from sparse sensor data is a challenge that frequently arises in many scientific domains. In practice, the process generating the data often is not understood to sufficient accuracy. Therefore, there is a growing interest in using the deep neural network route to address the problem. This work presents a novel approach that learns a continuous representation of the physical field using implicit neural representations (INRs). Specifically, after factorizing spatiotemporal variability into spatial and temporal components using the separation of variables technique, the method learns relevant basis functions from sparsely sampled irregular data points to develop a continuous representation of the data. In experimental evaluations, the proposed model outperforms recent INR methods, offering superior reconstruction quality on simulation data from a state-of-the-art climate model and a second dataset that comprises ultra-high resolution satellite-based sea surface temperature fields.

Motivation & Objective

• Motivate reliable reconstruction of complex physical fields from sparse sensor data.
• Develop a continuous field representation via implicit neural representations (INRs) tailored for sparsity and irregular sampling.
• Leverage a context-aware latent code to guide a decoder in reconstructing spatiotemporal fields.
• Demonstrate improved reconstruction quality over state-of-the-art INR baselines on climate and satellite datasets.

Proposed method

• Propose MMGN with an encoder–decoder architecture where the encoder produces a time-specific latent code z_t from measurements U_t, and the decoder D_phi uses z_t and spatial coordinates x to predict u(x,t).
• Use an auto-decoder scheme that optimizes latent codes z_t per time step via gradient descent to fit sparse observations.
• In the decoder, apply a Gabor-based coordinate transform g(x) and a multiplicative modulation similar to a multiplicative filter network to fuse x and z.
• Incorporate a context-aware indexing mechanism that replaces plain time t with a latent code guided by available measurements, enabling continuous spatial representation.
• Employ spectral-feature inspired Gabor filters to mitigate spectral bias and achieve a continuous field representation.
• Train both encoder and decoder jointly, with z_t optimized per sample during training as described in the auto-decoder formulation.

Figure 1: Field reconstruction from sparse observations: The Prior approach uses the time index ( $t$ ) as a reference to indicate a specific time instance. Our approach is context-aware, leveraging available context information by incorporating measurements at time $t$ .

Experimental results

Research questions

• RQ1 Can a context-aware latent code improve continuous field reconstruction from sparse, irregular observations compared to time-indexed INRs?
• RQ2 Do multiplicative Gabor-based decoders better capture high-frequency and complex structures in physical fields than standard INR architectures?
• RQ3 How does MMGN perform relative to state-of-the-art INR baselines on climate-model temperature fields and high-resolution satellite sea surface temperature data?
• RQ4 What is the robustness of MMGN to varying data sparsity and noise levels in reconstruction tasks?

Key findings

• MMGN consistently outperforms baseline INR models across tasks and datasets, with relative error reductions up to 78.94% on simulation data and 37.51% on satellite data at low sampling ratios.
• The Gabor-based decoder with multiplicative modulation substantially improves high-frequency detail capture over Fourier-based or purely MLP decoders.
• Context-aware latent codes provide semantic guidance that enhances reconstruction quality, especially when data is sparse or irregular.
• MMGN maintains favorable efficiency–accuracy trade-offs, offering faster inference than several baselines while achieving higher accuracy.
• Ablation studies show removing or replacing the Gabor filters degrades performance, confirming the importance of the chosen decoder design.

Figure 2: Architecture of the MMGN Model. The MMGN model employs auto-decoding to infer the latent variable ${\bm{z}}$ . Consequently, only the decoder is explicitly defined, and encoding takes place through stochastic optimization. More precisely, the latent code ${\bm{z}}=\arg\min_{{\bm{z}}}\mathc