[Paper Review] Galerkin Methods for Complementarity Problems and Variational Inequalities

Geoffrey J. Gordon|arXiv (Cornell University)|Jun 20, 2013

Advanced Optimization Algorithms Research1 references3 citations

TL;DR

This paper introduces a novel Galerkin method for solving complementarity problems and variational inequalities that improves upon Bertsekas' approach by ensuring error bounds depend directly on the distance from the true solution to the basis subspace, while requiring only projections onto the feasible set and the basis span—enabling efficient, scalable solutions for large-scale problems in machine learning, game theory, and physical simulation.

ABSTRACT

Complementarity problems and variational inequalities arise in a wide variety of areas, including machine learning, planning, game theory, and physical simulation. In all of these areas, to handle large-scale problem instances, we need fast approximate solution methods. One promising idea is Galerkin approximation, in which we search for the best answer within the span of a given set of basis functions. Bertsekas proposed one possible Galerkin method for variational inequalities. However, this method can exhibit two problems in practice: its approximation error is worse than might be expected based on the ability of the basis to represent the desired solution, and each iteration requires a projection step that is not always easy to implement efficiently. So, in this paper, we present a new Galerkin method with improved behavior: our new error bounds depend directly on the distance from the true solution to the subspace spanned by our basis, and the only projections we require are onto the feasible region or onto the span of our basis.

Motivation & Objective

• Address the limitations of existing Galerkin methods for variational inequalities, particularly poor error bounds and computationally expensive projection steps.
• Develop a method that achieves better approximation accuracy by directly relating error to the distance from the true solution to the basis subspace.
• Reduce computational overhead by restricting projections to only the feasible set and the basis span, avoiding complex intermediate projections.
• Enable scalable solution of large-scale complementarity problems in machine learning, planning, and physical simulation using efficient linear complementarity problem (LCP) solvers.
• Establish theoretical convergence guarantees and practical efficiency through a projective LCP formulation with structured matrix properties.

Proposed method

• Propose a fixed-point formulation involving two projections: one onto the feasible set $C$ and one onto the span of basis functions $\Phi$.
• Define the Galerkin approximation as the fixed point of the system: $x = \Pi_C(z)$ and $z = \Pi_\Phi((I - \alpha F)(x))$, where $\alpha$ is a step size.
• Ensure the approximate solution $\bar{x}$ is always feasible ($\bar{x} \in C$) and satisfies a residual condition: $-F(\bar{x}) + \epsilon \in N_C(\bar{x})$ for some $\epsilon \in \text{null}(\Phi^\top)$.
• Transform the Galerkin fixed-point system into a linear complementarity problem (LCP) of the form $\text{CP}(Nx + r, K)$, where $N = I - \Pi_\Phi + \alpha \Pi_\Phi M$ and $r = \alpha \Pi_\Phi q$.
• Leverage the projective structure of $N$ to accelerate the Unified Interior Point (UIP) method, reducing per-iteration cost to $O(nk^2)$ when $k \ll n$.
• Prove that if $M$ is positive definite, then $N$ is also positive definite, ensuring strong monotonicity and polynomial-time convergence of the LCP solver.

Experimental results

Research questions

• RQ1 Can a Galerkin method for variational inequalities be designed to achieve error bounds that depend directly on the distance from the true solution to the basis subspace?
• RQ2 Can the projection complexity of existing Galerkin methods be reduced by avoiding intermediate projections and restricting to only the feasible set and basis span?
• RQ3 Does transforming the Galerkin fixed-point system into a projective LCP preserve convergence and enable faster solution via specialized interior-point methods?
• RQ4 Can the resulting LCP be solved efficiently when the basis dimension $k$ is much smaller than the problem dimension $n$?
• RQ5 What conditions ensure that the transformed LCP remains well-posed and convergent, particularly in terms of matrix properties like positive definiteness?

Key findings

• The proposed Galerkin method achieves error bounds that depend directly on the distance from the true solution to the subspace spanned by the basis functions, improving upon prior methods.
• The method requires only two projections: onto the feasible set $C$ and onto the basis span $\Phi$, eliminating the need for complex intermediate projections.
• The fixed-point solution $\bar{x}$ is always feasible ($\bar{x} \in C$), and satisfies a residual condition involving $\epsilon \in \text{null}(\Phi^\top)$, ensuring approximate optimality.
• The Galerkin system is reformulated as a linear complementarity problem $\text{CP}(Nx + r, K)$, where $N$ inherits strong monotonicity from the original matrix $M$ if $M$ is positive definite.
• The matrix $N$ is projective, enabling the use of the Unified Interior Point (UIP) method with $O(nk^2)$ per-iteration cost, which is efficient when $k \ll n$.
• The number of iterations for the UIP method remains polynomial under monotonicity, ensuring scalable convergence for large-scale problems.