[Paper Review] Parallelizing the dual revised simplex method

Q. Huangfu, J. A. J. Hall|arXiv (Cornell University)|Mar 6, 2015

Advanced Optimization Algorithms Research14 references11 citations

TL;DR

This paper presents two novel parallel dual revised simplex solvers—PAMI and SIP—designed for large-scale sparse linear programs on standard desktop architectures. PAMI exploits suboptimization-based parallelism across iterations with a 0.95 cutoff to maintain pivot quality, achieving a mean speedup of 1.51 and outperforming Clp, the leading open-source solver; SIP enables single-iteration parallelism and complements PAMI, especially when PAMI slows down, with both approaches collectively matching CPLEX performance in key benchmarks.

ABSTRACT

This paper introduces the design and implementation of two parallel dual simplex solvers for general large scale sparse linear programming problems. One approach, called PAMI, extends a relatively unknown pivoting strategy called suboptimization and exploits parallelism across multiple iterations. The other, called SIP, exploits purely single iteration parallelism by overlapping computational components when possible. Computational results show that the performance of PAMI is superior to that of the leading open-source simplex solver, and that SIP complements PAMI in achieving speedup when PAMI results in slowdown. One of the authors has implemented the techniques underlying PAMI within the FICO Xpress simplex solver and this paper presents computational results demonstrating their value. This performance increase is sufficiently valuable for the achievement to be used as the basis of promotional material by FICO. In developing the first parallel revised simplex solver of general utility and commercial importance, this work represents a significant achievement in computational optimization.

Motivation & Objective

• To develop a general-purpose, commercially viable parallel dual revised simplex solver for large-scale sparse linear programs on standard multi-core desktops.
• To overcome the long-standing challenge of achieving scalable speedup in revised simplex methods, which are traditionally considered unsuitable for parallelization due to their sequential nature and computational bottlenecks.
• To design and implement two complementary parallelization strategies—PAMI and SIP—that exploit different forms of parallelism while maintaining numerical stability and pivot quality.
• To demonstrate that efficient parallelization of the dual revised simplex method is not only feasible but also commercially impactful, as validated by integration into FICO Xpress and use in promotional materials.

Proposed method

• PAMI (Parallelization Across Multiple Iterations) uses a suboptimization pivoting strategy to enable parallelism across multiple simplex iterations, with a dynamic cutoff factor (0.95) to maintain pivot quality and prevent degeneracy-induced performance degradation.
• SIP (Single-Iteration Parallelism) exploits fine-grained, data-parallel execution within a single simplex iteration by overlapping computational components such as sparse matrix-vector multiplication (spmv), factorization (ftran), and dual update operations.
• Both solvers are built on a high-performance sequential dual simplex solver (hsol) that incorporates sparse LU factorization, hyper-sparse linear system solution, efficient LU update techniques, and advanced pivoting rules like dual steepest-edge (DSE) and bound flipping ratio test (BFRT).
• The implementation targets standard desktop multi-core architectures, avoiding reliance on specialized high-performance hardware, to ensure broad applicability and real-world usability.
• Performance is evaluated using a diverse benchmark set of large-scale sparse LP problems, with comparisons to Clp (best open-source solver), CPLEX (industry-standard commercial solver), and the FICO Xpress solver after integration of the techniques.
• The design emphasizes memory access coalescing and load balancing to minimize overheads in parallel execution, particularly in the context of irregular sparse matrix operations.

Experimental results

Research questions

• RQ1 Can meaningful speedup be achieved in the dual revised simplex method through parallelization, despite its inherently sequential nature and reliance on sparse matrix operations?
• RQ2 Does the use of suboptimization as a pivoting strategy enable effective parallelism across multiple iterations without sacrificing pivot quality or convergence speed?
• RQ3 Can single-iteration parallelism via task-level overlap (SIP) provide a viable, complementary alternative to multi-iteration schemes like PAMI, especially in cases where PAMI incurs overhead or slowdown?
• RQ4 To what extent can a research-grade parallel simplex implementation match or exceed the performance of mature commercial solvers like CPLEX in real-world sparse LP problems?
• RQ5 Can the integration of these techniques into a commercial solver (FICO Xpress) result in measurable, commercially significant performance improvements, as evidenced by adoption in advertising and benchmarking?

Key findings

• PAMI achieves a mean speedup of 1.51 across the test set, outperforming Clp, the best available open-source simplex solver, demonstrating that parallelization of the revised simplex method can yield substantial performance gains.
• SIP achieves a mean speedup of 1.15, which is lower than PAMI’s, but it frequently complements PAMI by delivering speedup on problems where PAMI experiences slowdown, such as Linf_520c (speedup 1.31) and maros-r7 (speedup 1.12).
• The combined performance of the better of PAMI and SIP matches or exceeds that of CPLEX 12.4 on the majority of test problems, particularly when preprocessing and crash are disabled, indicating strong competitiveness with leading commercial solvers.
• The performance of PAMI is limited by the dominance of ftran-dse (dual steepest-edge factorization) in the computational pipeline—this operation accounts for over 80% of runtime—while other components (spmv, chuzc, update-dual) collectively contribute less than 5%, highlighting ftran-dse as the primary bottleneck.
• The integration of PAMI and SIP techniques into the FICO Xpress solver led to a commercially significant performance improvement, with FICO publicly announcing the achievement in advertising and blogs, confirming the real-world impact of the research.
• Independent benchmarking by Mittelmann confirms that the enhanced Xpress solver with these techniques matches or exceeds the serial performance of CPLEX, even with preprocessing and crash enabled, validating the robustness and scalability of the approach.