[Paper Review] Local turnpike analysis using local dissipativity for discrete time discounted optimal control

Lars Grüne, Lisa Krügel|arXiv (Cornell University)|Jan 27, 2021

Economic theories and models23 references8 citations

TL;DR

This paper introduces a local dissipativity framework for discrete-time discounted optimal control problems to analyze local turnpike behavior around locally optimal equilibria. By defining local strict dissipativity and deriving thresholds β₁ and β₂ for the discount factor, the authors show that optimal trajectories converge to a local equilibrium if β ∈ [β₁, β₂], with convergence guaranteed when the discount factor lies within this interval, even when multiple equilibria exist due to discounting effects.

ABSTRACT

Recent results in the literature have provided connections between the so-called turnpike property, near optimality of closed-loop solutions, and strict dissipativity. Motivated by applications in economics, optimal control problems with discounted stage cost are of great interest. In contrast to non-discounted optimal control problems, it is more likely that several asymptotically stable optimal equilibria coexist. Due to the discounting and transition cost from a local to the global equilibrium, it may be more favourable staying in a local equilibrium than moving to the global - cheaper - equilibrium. In the literature, strict dissipativity was shown to provide criteria for global asymptotic stability of optimal equilibria and turnpike behavior. In this paper, we propose a local notion of discounted strict dissipativity and a local turnpike property, both depending on the discount factor. Using these concepts, we investigate the local behaviour of (near-)optimal trajectories and develop conditions on the discount factor to ensure convergence to a local asymptotically stable optimal equilibrium.

Motivation & Objective

• . The paper aims to address the challenge of local convergence in discounted optimal control, where multiple locally stable equilibria may coexist due to discounting.
• It investigates conditions under which optimal trajectories converge to a locally asymptotically stable optimal equilibrium, rather than being drawn to a globally cheaper but harder-to-reach equilibrium.
• The objective is to establish a local turnpike property using a localized version of strict dissipativity, tailored to the discount factor.
• The study seeks to provide a framework that links local dissipativity to local turnpike behavior, extending global results to local settings.

Proposed method

• . The authors define a local notion of discounted strict dissipativity, requiring the existence of a local storage function λ(x) such that ℓ(x,u) + λ(x) − βλ(f(x,u)) ≥ 0 in a neighborhood of the equilibrium.
• They introduce two thresholds β₁ and β₂, derived from local dynamics and value function growth, such that for β ∈ [β₁, β₂], trajectories near a local equilibrium remain near it and converge to it.
• The dynamic programming principle is used to relate the optimal value function V∞(x) to the stage cost and discounted future cost, enabling recursive analysis of trajectories.
• An invariance condition is introduced to ensure trajectories stay within the local region of interest, and it is shown to be satisfied under suitable growth conditions on the value function.
• The analysis is conducted in discrete time, leveraging the turnpike formalism to express convergence in terms of time spent near the equilibrium, not just asymptotic convergence.
• Numerical examples are used to validate the theoretical thresholds and illustrate the transition between local and global equilibrium behavior based on β.

Experimental results

Research questions

• RQ1 . Under what conditions does a locally optimal equilibrium attract nearby optimal trajectories in discounted optimal control problems?
• RQ2 How does the discount factor β influence the stability and convergence behavior of trajectories near a local equilibrium in the presence of multiple equilibria?
• RQ3 Can a local version of strict dissipativity be used to derive a local turnpike property, even when global dissipativity does not hold?
• RQ4 What role do the thresholds β₁ and β₂ play in determining the interval of discount factors for which local convergence occurs?
• RQ5 How do the local and global properties of the optimal control problem jointly determine whether the interval [β₁, β₂] is non-empty?

Key findings

• . The paper establishes that for a discount factor β ∈ [β₁, β₂], optimal trajectories starting near a locally asymptotically stable equilibrium converge to it, with β₁ and β₂ determined by local and global properties, respectively.
• The threshold β₁ depends only on local dynamics and the local storage function, ensuring convergence to the local equilibrium when β ≥ β₁.
• The threshold β₂ depends on the behavior of the optimal value function away from the local equilibrium, ensuring trajectories remain near the equilibrium when β ≤ β₂.
• When β₁ ≤ β₂, the interval [β₁, β₂] is non-empty, and local turnpike behavior is guaranteed: trajectories spend a quantifiable amount of time near the local equilibrium.
• Numerical results confirm that for β < β₂, trajectories may diverge from the local equilibrium and converge to a global one, especially when transition costs are high.
• The approach is robust to non-convex stage costs, as demonstrated in Example 7.3, where strict dissipativity holds even though the stage cost is strictly concave in x.