[Paper Review] Lack of Equality between Abel and Cesaro Limits in Discrete Optimal Control and the Implied Duality Gap

Ilya Shvartsman|arXiv (Cornell University)|Oct 27, 2020

Optimization and Variational Analysis15 references4 citations

TL;DR

This paper constructs a discrete-time optimal control example with a compact state space where Cesàro and Abel limits for long-run average and discounted cost problems differ, demonstrating a duality gap in infinite-dimensional linear programming (IDLP) formulations. The result confirms that strong duality does not hold in general for IDLP problems derived from optimal control, even under standard compactness and continuity assumptions.

ABSTRACT

In a recent paper it has been shown that if Cesaro and Abel limits for a certain discrete time optimal control problem are not equal, then there is a duality gap between a certain infinite-dimensional linear programming problem and its dual. In this paper we construct an example of a problem satisfying the assumptions of the aforementioned paper, where Cesaro and Abel limits are different.

Motivation & Objective

• To demonstrate that Cesàro and Abel limits can differ in discrete-time optimal control problems under standard assumptions.
• To show that such inequality implies a duality gap between an infinite-dimensional linear program (IDLP) and its dual.
• To construct a concrete example satisfying the conditions of prior theoretical results where the limits fail to be equal.
• To extend known counterexamples from continuous time to discrete time with compact state spaces.
• To validate the theoretical link between limit inequality and duality gap in IDLP formulations of optimal control.

Proposed method

• Adapts a continuous-time uncontrolled system with piecewise constant cost function g(x) to a discrete-time controlled system.
• Modifies the dynamics to ensure compact invariant state space Y, using a perturbation term q(x) that vanishes for x ∈ [0,2] and becomes negative for x ≥ 3.
• Imposes control constraints so that y(t) remains constant over intervals, enabling trajectory approximation via piecewise-constant control policies.
• Uses a smoothed version of the discontinuous cost function g to ensure continuity of f and g, satisfying required regularity conditions.
• Applies Tauberian-type arguments to analyze the asymptotic behavior of average and discounted cost values.
• Establishes continuity of V_T(y₀) and h_α(y₀) via uniform continuity and Hausdorff continuity of f, g, and U(·), ensuring applicability of IDLP duality theory.

Experimental results

Research questions

• RQ1 Can Cesàro and Abel limits fail to be equal in discrete-time optimal control problems with compact state spaces?
• RQ2 Does the inequality between Cesàro and Abel limits imply a duality gap in the associated infinite-dimensional linear programming (IDLP) formulation?
• RQ3 Is it possible to construct a concrete example satisfying the assumptions of prior IDLP duality results where the limits differ?
• RQ4 How do the dynamics and cost function structure affect the convergence of average and discounted cost values?
• RQ5 What role does compactness and continuity of the state and control sets play in the existence and equality of these limits?

Key findings

• A discrete-time optimal control system is constructed where the Cesàro limit lim_{T→∞} V_T(y₀) and Abel limit lim_{α↑1} h_α(y₀) exist but are unequal for certain initial states.
• The example satisfies all assumptions of Theorem 4.1 in [4], including compactness of Y, continuity of f and g, and uniform Hausdorff continuity of U(·).
• The inequality between Cesàro and Abel limits implies a duality gap between the primal and dual infinite-dimensional linear programs (IDLPs).
• For initial states with x₀ ∈ (1+ε, 2−ε) and y₀ = 0, both the average-cost and discounted-cost optimal values converge to zero, preserving equality in this case.
• The construction relies on a compact invariant set and a modified dynamics with q(x) to ensure trajectories remain bounded and the system evolves on a compact set.
• The proof establishes that the optimal value of the IDLP is strictly less than that of its dual, confirming the presence of a duality gap.