[Paper Review] Learning to Control PDEs with Differentiable Physics

Philipp Holl, Vladlen Koltun+1 more|arXiv (Cornell University)|Jan 21, 2020

Model Reduction and Neural Networks57 references44 citations

TL;DR

The paper introduces a hierarchical predictor-corrector framework with a differentiable PDE solver to learn long-horizon control of PDE-governed systems, demonstrated on Burgers and incompressible Navier–Stokes flows with partial observability.

ABSTRACT

Predicting outcomes and planning interactions with the physical world are long-standing goals for machine learning. A variety of such tasks involves continuous physical systems, which can be described by partial differential equations (PDEs) with many degrees of freedom. Existing methods that aim to control the dynamics of such systems are typically limited to relatively short time frames or a small number of interaction parameters. We present a novel hierarchical predictor-corrector scheme which enables neural networks to learn to understand and control complex nonlinear physical systems over long time frames. We propose to split the problem into two distinct tasks: planning and control. To this end, we introduce a predictor network that plans optimal trajectories and a control network that infers the corresponding control parameters. Both stages are trained end-to-end using a differentiable PDE solver. We demonstrate that our method successfully develops an understanding of complex physical systems and learns to control them for tasks involving PDEs such as the incompressible Navier-Stokes equations.

Motivation & Objective

• Motivate learning-based control for complex PDEs with many degrees of freedom.
• Split control into planning (prediction) and action (correction) modules.
• Leverage a differentiable PDE solver to backpropagate through time for end-to-end training.
• Enable long-horizon control by hierarchical temporal modeling across multiple time scales.
• Handle partial observability by conditioning the agent on observable states rather than full states.

Proposed method

• Propose a predictor-corrector architecture with an Observation Predictor (OP) and a Control Force Estimator (CFE).
• Model prediction of intermediate observable states at center points between time steps, recursively, to create a temporal hierarchy across multiple time scales.
• Instantiate a unique OP model for each time scale and train via a prediction-refinement execution scheme for stability and accuracy.
• Use a differentiable PDE solver (PhiFlow) to backpropagate through the full trajectory and optimize the control forces with differentiable physics losses.
• Combine differentiable physics losses with optional supervised losses to train agents that minimize the integral of squared control forces while matching target observables.
• Discuss execution orders (prediction refinement, staggered execution) to balance planning accuracy and computational efficiency.

Experimental results

Research questions

• RQ1 Can a neural agent learn to control PDE-governed systems over long time horizons?
• RQ2 Does a hierarchical predictor-corrector architecture with a differentiable solver enable stable long-horizon control under partial observability?
• RQ3 How does temporal hierarchy and prediction refinement compare to single-shot or purely supervised approaches in PDE control?
• RQ4 What are the effects of observable-state conditioning on control performance?

Key findings

• Differentiable physics losses substantially improve trajectory reconstruction quality compared to purely supervised training.
• The refined prediction-refinement scheme achieves the lowest control effort among tested methods on Burgers’ equation (14.2 ±0.7 vs 83.4 ±2.0 with CFE chain supervised).
• For Burgers’ equation, the differentiable physics setup with refinement attains similar trajectory quality with much shorter inference times (3.05 ±0.37 ms) relative to iterative optimization.
• In 2D incompressible flow, the staggered and refined schemes with differentiable physics require far less force than supervised baselines (e.g., 22.6 ±1.1 vs 243 ±11 for a natural flow task).
• Refined, diff-physics predictions in the shape/flow tasks achieve about half the final loss of the staggered variant and converge to target states with smoother transitions.
• The learned solution manifolds enable near-instantaneous inference and can serve as good initial guesses for traditional solvers, improving convergence.