[Paper Review] A Differentiable Augmented Lagrangian Method for Bilevel Nonlinear Optimization

Benoit Landry, Zachary Manchester+1 more|arXiv (Cornell University)|Feb 8, 2019

Optimization and Variational Analysis4 citations

TL;DR

This paper proposes a differentiable augmented Lagrangian method for bilevel nonlinear optimization that enables automatic differentiation through the lower-level solver without relying on KKT conditions. By implementing the solver in Julia and combining it with SQP-based solvers like SNOPT, the approach achieves scalable, differentiable bilevel optimization suitable for robotics problems such as robust control and parameter estimation with contact dynamics.

ABSTRACT

Many problems in modern robotics can be addressed by modeling them as bilevel optimization problems. In this work, we leverage augmented Lagrangian methods and recent advances in automatic differentiation to develop a general-purpose nonlinear optimization solver that is well suited to bilevel optimization. We then demonstrate the validity and scalability of our algorithm with two representative robotic problems, namely robust control and parameter estimation for a system involving contact. We stress the general nature of the algorithm and its potential relevance to many other problems in robotics.

Motivation & Objective

• Address the challenge of solving bilevel optimization problems in robotics, particularly those with complex dynamics and contact forces.
• Overcome limitations of KKT-based approaches that require optimal lower-level solutions and are sensitive to non-convexity or non-smoothness.
• Develop a general-purpose, differentiable nonlinear program solver that supports both equality and inequality constraints without projection steps.
• Enable scalable parameter estimation and robust control in robotic systems by leveraging parallelization of lower-level problems.
• Demonstrate the method's viability on real-world robotic problems involving contact and friction, showing superior scalability with dataset size.

Proposed method

• Formulate bilevel optimization problems where the lower-level problem is solved using a differentiable augmented Lagrangian algorithm.
• Implement the lower-level solver in Julia to fully exploit native support for automatic differentiation and avoid non-differentiable operations.
• Use sequential quadratic programming (SQP) solvers like SNOPT for the upper-level problem, enabling constraint handling beyond unconstrained objectives.
• Avoid reliance on Karush–Kuhn–Tucker (KKT) conditions for gradient computation, instead auto-differentiating directly through the solver’s algorithmic steps.
• Handle infeasible lower-level problems by treating the residual as a differentiable loss, enabling gradient computation even when the lower problem is not solved to optimality.
• Parallelize the lower-level solves across multiple threads, with each thread handling one sample in a dataset, enabling efficient scaling with data size.

Experimental results

Research questions

• RQ1 Can a differentiable, general-purpose nonlinear solver be developed for bilevel optimization that avoids reliance on KKT conditions?
• RQ2 How does the proposed method scale with increasing numbers of samples in parameter estimation tasks involving contact dynamics?
• RQ3 Can the differentiable solver be effectively combined with state-of-the-art SQP solvers to solve constrained bilevel problems in robotics?
• RQ4 What is the trade-off between computational cost and scalability when using auto-differentiation through a nonlinear solver versus traditional KKT-based approaches?
• RQ5 To what extent can infeasible lower-level problems still yield meaningful gradients and stable optimization in bilevel formulations?

Key findings

• The proposed bilevel parameter estimation approach scales significantly better with dataset size than classical methods that introduce decision variables for unobserved contact forces.
• For 44 samples, the bilevel method took 30.7 seconds (scaled to 7.7), while the classical method required 24.55 seconds per sample (scaled to 246.0), indicating superior scalability of the bilevel approach.
• Even with infeasible lower-level problems due to noisy data, the method produces well-defined gradients, enabling stable optimization without requiring convergence to optimality.
• The method successfully estimated friction coefficients (μ) with values ranging from 0.15 to 0.23, achieving accurate results across multiple simulation runs.
• The differentiable solver enables end-to-end optimization in robotics applications such as contact-rich manipulation and robust control, demonstrating practical utility beyond theoretical formulation.
• The implementation in Julia allows for efficient automatic differentiation and is publicly available at https://github.com/blandry/Bilevel.jl, supporting reproducibility and extension.