[Paper Review] Optimal control of nonequilibrium systems through automatic differentiation

Megan C. Engel, Jamie Smith+1 more|arXiv (Cornell University)|Jan 1, 2022

Advanced Thermodynamics and Statistical Mechanics4 citations

TL;DR

This paper introduces a novel optimal control framework for nonequilibrium systems using automatic differentiation (AD) to compute protocols that minimize dissipated work, valid arbitrarily far from equilibrium. The method enables accurate optimization in complex systems like magnetization reversal and barrier crossing, outperforming near-equilibrium theories and revealing nontrivial protocol shapes—such as accelerating toward and decelerating from energy barriers—especially in high-barrier regimes.

ABSTRACT

Controlling the evolution of nonequilibrium systems to minimize dissipated heat or work is a key goal for designing nanodevices, both in nanotechnology and biology. Progress in computing optimal protocols has thus far been limited to either simple systems or near-equilibrium evolution. Here, we present an approach for computing optimal protocols based on automatic differentiation. Our methodology is applicable to complex systems and multidimensional protocols and is valid arbitrarily far from equilibrium. We validate our method by reproducing theoretical optimal protocols for a Brownian particle in a time-varying harmonic trap. We also compute departures from near-equilibrium behaviour for magnetization reversal on an Ising lattice and for barrier crossing driven by a harmonic trap, which has been used to represent a range of biological processes including biomolecular unfolding reactions. Algorithms based on automatic differentiation outperform the near-equilibrium theory for far-from-equilibrium magnetization reversal and driven barrier crossing. The optimal protocol for crossing an energy landscape barrier of 10kT is found to hasten the approach to, and slow the departure from, the barrier region compared to the near-equilibrium theoretical protocol.

Motivation & Objective

• To develop a general-purpose method for computing optimal control protocols in complex, far-from-equilibrium systems where traditional approaches fail.
• To overcome the limitations of near-equilibrium approximations in optimal control, which restrict applicability to low-energy-barrier systems.
• To enable high-precision optimization of thermodynamic work and heat dissipation in systems such as nanomagnets and biomolecules.
• To demonstrate the method’s efficacy on canonical nonequilibrium problems, including Ising model magnetization reversal and barrier crossing in Brownian dynamics.
• To reveal nontrivial protocol structures that emerge in far-from-equilibrium regimes, deviating significantly from near-equilibrium theoretical predictions.

Proposed method

• Leverages automatic differentiation (AD) to compute gradients of the average dissipated work with respect to protocol parameters by backpropagating through entire stochastic simulations.
• Applies AD to Monte Carlo (MC) simulations of the 2D Ising model to optimize magnetization reversal protocols.
• Uses AD in molecular dynamics (MD) simulations of a Brownian particle in a time-varying harmonic trap to reproduce analytical optimal protocols.
• Employs a reparametrization trick to enable efficient gradient computation in over-damped Langevin dynamics by treating noise as a deterministic function of random variables.
• Optimizes protocols using the Adam optimizer with batched trajectory sampling and piecewise polynomial parametrization (e.g., Chebyshev polynomials) for smooth, adaptive control profiles.
• For high-barrier systems, applies the REINFORCE method to improve gradient estimation when direct backpropagation is unstable.

Experimental results

Research questions

• RQ1 How can optimal control protocols be computed for complex, far-from-equilibrium systems where near-equilibrium approximations break down?
• RQ2 What are the structural differences between optimal protocols in the far-from-equilibrium regime and those predicted by linear-response theory?
• RQ3 Can automatic differentiation enable accurate and efficient optimization of dissipated work in systems like the Ising model and biomolecular barrier crossing?
• RQ4 How does the optimal protocol shape evolve as the energy barrier height increases beyond the near-equilibrium limit?
• RQ5 To what extent do AD-based protocols outperform near-equilibrium theoretical protocols in high-dissipation, high-barrier scenarios?

Key findings

• For the 2D Ising model, the AD-based protocol matches near-equilibrium theory in the linear regime but significantly outperforms it in the far-from-equilibrium regime, reducing dissipated work.
• In the Brownian particle in a time-varying harmonic trap, the method successfully reproduces the analytically known optimal protocol, validating its accuracy.
• For a 10 $k_{ ext{B}}T$ barrier crossing, the optimal protocol accelerates the particle toward the barrier region and slows its departure, deviating markedly from the near-equilibrium protocol.
• The AD method captures nontrivial protocol dynamics in high-barrier systems, revealing that optimal control is not simply a perturbation of equilibrium behavior.
• In the 10 $k_{ ext{B}}T$ barrier case, the REINFORCE method improved convergence over direct gradient estimation, enabling reliable optimization in high-dissipation regimes.
• The method enables efficient optimization of complex, multidimensional protocols in systems with arbitrary complexity and far-from-equilibrium dynamics, overcoming prior limitations of analytical or near-equilibrium approaches.