[Paper Review] An Optimal Control Approach to the Persistent Monitoring Problem

Christos G. Cassandras, Xuchao Lin+1 more|arXiv (Cornell University)|Feb 28, 2012

Distributed Control Multi-Agent Systems20 references3 citations

TL;DR

This paper proposes an optimal control framework for persistent monitoring using mobile agents in a 1D mission space, modeling uncertainty as a dynamic queueing system. It reduces the problem to parametric optimization over switching points and waiting times, using Infinitesimal Perturbation Analysis (IPA) for gradient-based optimization, and demonstrates that agents should avoid end points and instead switch at interior locations to minimize total uncertainty over time.

ABSTRACT

We propose an optimal control framework for persistent monitoring problems where the objective is to control the movement of mobile nodes to minimize an uncertainty metric in a given mission space. For multi agent in a one-dimensional mission space, we show that the optimal solution is obtained in terms of a sequence of switching locations and waiting time on these switching points, thus reducing it to a parametric optimization problem. Using Infinitesimal Perturbation Analysis (IPA) we obtain a complete solution through a gradient-based algorithm. We also discuss a receding horizon controller which is capable of obtaining a near-optimal solution on-the-fly.

Motivation & Objective

• To develop a control strategy that minimizes accumulated uncertainty in a dynamically changing environment monitored by mobile agents.
• To address the challenge of balancing agent movement to ensure all areas are visited infinitely often while respecting sensing and motion constraints.
• To reduce the complex optimal control problem to a parametric optimization over switching locations and dwell times.
• To enable real-time control through a receding horizon controller that achieves near-optimal performance.
• To establish theoretical conditions under which optimal trajectories consist of full-speed motion, switching at interior points, and non-zero waiting times.

Proposed method

• Formulates the persistent monitoring problem as minimizing a time-integrated uncertainty metric over a 1D mission space.
• Models uncertainty growth and reduction as a queueing process, where agents act as servers visiting sampling points.
• Reduces the optimal control problem to a parametric optimization over switching points $\theta_k$ and waiting times $w_k$.
• Applies generalized Infinitesimal Perturbation Analysis (IPA) to compute gradients of the cost function with respect to the parameters.
• Uses a gradient-based algorithm with Armijo line search to iteratively update switching locations and waiting times.
• Develops a receding horizon controller that enables on-the-fly, near-optimal control by solving a sequence of finite-horizon problems.

Experimental results

Research questions

• RQ1 What is the optimal trajectory structure for a single agent monitoring a 1D environment with dynamic uncertainty?
• RQ2 How can the optimal control problem be reduced to a parametric optimization over switching points and waiting times?
• RQ3 Can IPA be effectively used to compute gradients for a hybrid system with discontinuous dynamics in the context of persistent monitoring?
• RQ4 Under what conditions do agents avoid the boundary points of the mission space in the optimal solution?
• RQ5 How robust is the IPA-based optimization to stochastic variations in sensing effectiveness over time?

Key findings

• The optimal trajectory for each agent consists of full-speed motion to switching points, followed by non-negative waiting times, and then switching direction—never reaching the boundary points of the mission space.
• The total uncertainty cost is significantly reduced through optimal selection of switching points and waiting times, with reductions observed in both one- and two-agent simulations.
• For a one-agent case with 20 sampling points, the optimal cost $J^* = 17.77$ when agents can access the full interval $[0,20]$, compared to $J^* = 39.14$ when restricted to $[4,16]$.
• In a stochastic environment where sensing effectiveness $A_i(t)$ varies randomly over time, the IPA-based algorithm still converges to a near-optimal solution with $J^* = 17.54$, demonstrating robustness.
• The algorithm converges in approximately 10–15 seconds using Armijo step-sizes, indicating computational feasibility despite non-smooth cost functions in stochastic cases.
• The presence of strictly positive waiting times at switching points is confirmed in simulations, especially when sensing range is limited near boundaries.