[Paper Review] Incorporating Symmetry into Deep Dynamics Models for Improved Generalization

Rui Wang, Robin Walters+1 more|arXiv (Cornell University)|Feb 8, 2020

Model Reduction and Neural Networks51 references82 citations

TL;DR

The paper introduces equivariant neural networks that enforce physical symmetries (translation, rotation, uniform motion, scaling) in deep dynamics models to improve generalization and physical consistency in forecasting turbulent flows and ocean data.

ABSTRACT

Recent work has shown deep learning can accelerate the prediction of physical dynamics relative to numerical solvers. However, limited physical accuracy and an inability to generalize under distributional shift limit its applicability to the real world. We propose to improve accuracy and generalization by incorporating symmetries into convolutional neural networks. Specifically, we employ a variety of methods each tailored to enforce a different symmetry. Our models are both theoretically and experimentally robust to distributional shift by symmetry group transformations and enjoy favorable sample complexity. We demonstrate the advantage of our approach on a variety of physical dynamics including Rayleigh Bénard convection and real-world ocean currents and temperatures. Compared with image or text applications, our work is a significant step towards applying equivariant neural networks to high-dimensional systems with complex dynamics. We open-source our simulation, data, and code at \url{https://github.com/Rose-STL-Lab/Equivariant-Net}.

Motivation & Objective

• Motivate the need for better generalization in deep dynamics prediction for physical systems.
• Develop neural architectures that are equivariant to key physical symmetries.
• Demonstrate improved generalization and physical consistency on turbulence and ocean datasets.

Proposed method

• Formulate f as G-equivariant so f(gx)=g f(x) for symmetry group G.
• Build equivariant convolutions in ResNet and U-net using group representations (translation, rotation via E(2)-CNN, uniform motion via input-centering/shift tricks, scale via group correlation over scaling group).
• Implement scale-equivariant methods including magnitude (resolution-independent) scaling and resolution-dependent group correlations.
• Provide theoretical guarantees on equivariance and discuss when data augmentation is redundant.
• Evaluate on Rayleigh–Bénard convection and real ocean current/temperature data, measuring RMSE and Energy Spectrum Error (ESE).

Experimental results

Research questions

• RQ1 Can embedding symmetries into CNN architectures improve forecasting accuracy for high-dimensional physical dynamics?
• RQ2 Do equivariant models generalize under distributional shifts induced by transformations (translation, rotation, uniform motion, scaling) better than non-equivariant baselines?
• RQ3 How do scale-related symmetries affect predictions when physical laws couple space, time, and magnitude?
• RQ4 What is the impact of equivariance on physical consistency metrics such as energy spectra in turbulent flows and oceans?

Key findings

• Equivariant models show reduced equivariance errors and improved physical consistency compared to non-equivariant baselines.
• On simulated Rayleigh–Bénard convection, Equ-ResNet/Unet models maintain accuracy under transformed test sets and outperform data-augmented baselines in RMSE and ESE.
• Equivariant scaling (including up/down scaling) and rotation-based equivariance yield notable improvements in generalization under distributional shifts, with reductions in RMSE and ESE across tests.
• Equivariant models trained without augmented data outperform or match augmented baselines in both RMSE and ESE on ocean current datasets.