[Paper Review] GPCG: A Case Study in the Performance and Scalability of Optimization Algorithms

Steven J. Benson, Lois Curfman McInnes+1 more|ArXiv.org|Jan 19, 2001

Advanced Optimization Algorithms Research8 references4 citations

TL;DR

This paper presents GPCG, a parallel implementation of the gradient projection conjugate gradient algorithm for bound-constrained convex quadratic optimization, leveraging PETSc's parallel linear algebra infrastructure. It demonstrates high scalability and efficiency on up to 64 processors, solving problems with over 2.5 million variables, with block Jacobi preconditioners significantly reducing solution time compared to diagonal methods.

ABSTRACT

GPCG is an algorithm within the Toolkit for Advanced Optimization (TAO) for solving bound constrained, convex quadratic problems. Originally developed by More' and Toraldo, this algorithm was designed for large-scale problems but had been implemented only for a single processor. The TAO implementation is available for a wide range of high-performance architecture, and has been tested on up to 64 processors to solve problems with over 2.5 million variables.

Motivation & Objective

• To address the performance and scalability challenges of large-scale bound-constrained quadratic optimization on parallel architectures.
• To overcome the scalability limitations of active-set methods due to dynamic redistribution of reduced Hessian matrices across processors.
• To enable efficient reuse of high-performance linear algebra and parallel computing infrastructure in optimization software.
• To evaluate the impact of different preconditioners on convergence and solution time in a scalable parallel setting.
• To demonstrate that object-oriented design with external toolkits enables efficient, portable, and extensible large-scale optimization.

Proposed method

• Implement GPCG within the Toolkit for Advanced Optimization (TAO), using object-oriented abstractions to interface with PETSc for parallel linear algebra operations.
• Leverage MPI for interprocessor communication and exploit PETSc’s support for distributed sparse matrices and preconditioners.
• Use a data-structure-neutral interface to allow flexible experimentation with matrix storage formats like block sparse and structured sparsity.
• Apply various preconditioners—diagonal Jacobi, block Jacobi with ILU(0) and ILU(2) subdomain solvers—to improve conjugate gradient convergence.
• Dynamically redistribute matrix rows across processors when the set of free variables changes between iterations to maintain load balance.
• Measure performance in terms of GPCG iterations, conjugate gradient iterations, total solution time, and scalability across 4 to 64 processors.

Experimental results

Research questions

• RQ1 How does the performance of GPCG scale across increasing numbers of processors and problem sizes?
• RQ2 What is the impact of different preconditioners on the convergence rate and total solution time of GPCG in a parallel environment?
• RQ3 Can the dynamic redistribution of the reduced Hessian matrix be managed efficiently to maintain scalability?
• RQ4 How does the number of active variables at the solution affect the performance and scalability of GPCG?
• RQ5 To what extent can external linear algebra toolkits like PETSc enhance the development and performance of large-scale optimization algorithms?

Key findings

• The number of GPCG iterations remained nearly constant (21–27) across different preconditioners and processor counts, indicating robust convergence behavior.
• The block Jacobi preconditioner with ILU(2) subdomain solver reduced total solution time by approximately 50% compared to the diagonal Jacobi preconditioner, despite higher setup cost.
• For n = 640,000 variables, the block Jacobi with ILU(2) achieved a solution time of 128 seconds on 16 processors, compared to 364 seconds for diagonal Jacobi.
• The conjugate gradient iterations were significantly reduced with better preconditioners: 6,312 for diagonal, 6,712 for block Jacobi-ILU(0), and 2,303 for block Jacobi-ILU(2).
• The implementation achieved high efficiency and solved problems with over 2.5 million variables using only 8 processors, highlighting low memory requirements of iterative solvers.
• Scalability was limited not only by redistribution overhead but also by the size of the active set; performance improved as the number of free variables increased.