[Paper Review] Feasibility-Guided Learning for Robust Control in Constrained Optimal Control Problems

Wei Xiao, Călin Belta+1 more|arXiv (Cornell University)|Dec 6, 2019

Advanced Control Systems Optimization19 references4 citations

TL;DR

This paper proposes a feasibility-guided learning approach to enhance robustness in constrained optimal control by optimizing parameters of High-Order Control Barrier Functions (HOCBFs) for arbitrary relative degree systems. By using machine learning to classify feasible/infeasible parameter regions and integrating a differentiable classifier into the learning process, the method improves QP feasibility under time-varying and unknown unsafe sets, outperforming gradient descent in simulation-based robot navigation tasks with safety guarantees.

ABSTRACT

Optimal control problems with constraints ensuring safety and convergence to desired states can be mapped onto a sequence of real time optimization problems through the use of Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs). One of the main challenges in these approaches is ensuring the feasibility of the resulting quadratic programs (QPs) if the system is affine in controls. The recently proposed penalty method has the potential to improve the existence of feasible solutions to such problems. In this paper, we further improve the feasibility robustness (i.e., feasibility maintenance in the presence of time-varying and unknown unsafe sets) through the definition of a High Order CBF (HOCBF) that works for arbitrary relative degree constraints; this is achieved by a proposed feasibility-guided learning approach. Specifically, we apply machine learning techniques to classify the parameter space of a HOCBF into feasible and infeasible sets, and get a differentiable classifier that is then added to the learning process. The proposed feasibility-guided learning approach is compared with the gradient-descent method on a robot control problem. The simulation results show an improved ability of the feasibility-guided learning approach over the gradient-decent method to determine the optimal parameters in the definition of a HOCBF for the feasibility robustness, as well as show the potential of the CBF method for robot safe navigation in an unknown environment.

Motivation & Objective

• To address the challenge of infeasible quadratic programs (QPs) in CBF- and CLF-based optimal control for systems with stringent safety constraints and tight control limits.
• To improve feasibility robustness in the presence of time-varying and unknown unsafe sets, especially for high relative degree systems.
• To develop a learning-based method that optimizes HOCBF parameters to delay constraint activation and maximize robustness.
• To compare the feasibility-guided learning approach with gradient descent in a robot control setting, demonstrating improved performance and safety.
• To enable adaptive, real-time safe navigation in unknown or dynamic environments using a differentiable, learnable barrier function framework.

Proposed method

• Parameterize a High-Order Control Barrier Function (HOCBF) with tunable penalty and power parameters to handle arbitrary relative degree constraints.
• Use machine learning to classify the HOCBF parameter space into feasible and infeasible regions based on QP feasibility.
• Train a differentiable classifier to map HOCBF parameters to feasibility outcomes, enabling end-to-end optimization.
• Integrate the learned classifier into a feasibility-guided optimization (FGO) framework to maximize the minimum distance to unsafe sets.
• Formulate a constrained optimal control problem where the HOCBF constraint activates as late as possible to enhance robustness.
• Compare the FGO approach with gradient descent (GD) in simulations, using classification accuracy and feasibility robustness as metrics.

Experimental results

Research questions

• RQ1 Can a machine learning-based classifier improve the feasibility of QPs in CBF- and CLF-based optimal control under unknown and time-varying unsafe sets?
• RQ2 How does feasibility-guided learning (FGO) compare to gradient descent in optimizing HOCBF parameters for robustness?
• RQ3 To what extent does the proposed method maintain QP feasibility when unsafe sets are unknown or dynamically changing?
• RQ4 Can the learned HOCBF parameters ensure safe robot navigation in an unknown environment with moving obstacles?
• RQ5 Does the differentiable classifier enable effective, scalable optimization of HOCBF parameters for high relative degree systems?

Key findings

• The feasibility-guided learning (FGO) approach achieved a classification accuracy of 97.0% on 4000 training samples, significantly outperforming gradient descent in identifying optimal HOCBF parameters.
• FGO demonstrated a 28.8% success rate in finding better solutions than gradient descent when using 3500 training samples, with a 5% improvement when combining both methods.
• The learned HOCBF parameters $(p_1^*, p_2^*, q_1^*, q_2^*) = (0.7426, 1.9745, 1.9148, 0.7024)$ enabled safe robot navigation across all tested obstacle configurations, including varying sizes and random movements.
• The CBF-based framework maintained safety guarantees without requiring global environment knowledge, unlike A* and RRT, which require full map information.
• Computation time for the CBF method was under 0.01 seconds per step, making it significantly more efficient than A* (1.3s) and RRT (0.3s) in dynamic environments requiring frequent replanning.
• In dynamic environments, the CBF framework showed superior adaptability and robustness compared to A* and RRT, especially when obstacle positions changed rapidly.