[Paper Review] Quadratic Decomposable Submodular Function Minimization

Pan Li, Niao He+1 more|arXiv (Cornell University)|Jun 26, 2018

Complexity and Algorithms in Graphs3 citations

TL;DR

This paper introduces a novel dual optimization framework for quadratic decomposable submodular function minimization (QDSFM), a key problem in graph and hypergraph learning. It proposes a randomized coordinate descent (RCD) algorithm with linear convergence, enabled by new conic projection techniques and weak strong convexity analysis, achieving faster convergence and higher accuracy than state-of-the-art methods in semi-supervised learning on hypergraphs.

ABSTRACT

We introduce a new convex optimization problem, termed quadratic decomposable submodular function minimization. The problem is closely related to decomposable submodular function minimization and arises in many learning on graphs and hypergraphs settings, such as graph-based semi-supervised learning and PageRank. We approach the problem via a new dual strategy and describe an objective that may be optimized via random coordinate descent (RCD) methods and projections onto cones. We also establish the linear convergence rate of the RCD algorithm and develop efficient projection algorithms with provable performance guarantees. Numerical experiments in semi-supervised learning on hypergraphs confirm the efficiency of the proposed algorithm and demonstrate the significant improvements in prediction accuracy with respect to state-of-the-art methods.

Motivation & Objective

• To address the lack of efficient, linearly convergent solvers for quadratic decomposable submodular function minimization (QDSFM), a critical problem in graph and hypergraph learning.
• To develop a new dual formulation for QDSFM that involves projection onto a product cone, differing from the polytope structure in standard DSFM.
• To establish weak strong convexity of the dual QDSFM problem to guarantee linear convergence of RCD and alternating projection methods.
• To design efficient conic projection algorithms using generalized Frank-Wolfe and min-norm-point methods with provable convergence guarantees.
• To empirically validate the proposed algorithm on hypergraph semi-supervised learning, demonstrating superior convergence speed and prediction accuracy.

Proposed method

• Derives a new dual formulation for QDSFM that requires finding the best approximation of a hyperplane via a product cone, replacing the polytope structure of standard DSFM.
• Proposes a randomized coordinate descent (RCD) algorithm for the dual QDSFM problem, with convergence rate analysis based on weak strong convexity.
• Introduces generalized Frank-Wolfe and min-norm-point methods to compute conic projections efficiently in each RCD iteration, with a 1/k convergence rate for the inner loop.
• Establishes linear convergence of the RCD algorithm by proving weak strong convexity of the dual objective, leveraging the underlying conic geometry.
• Employs projection algorithms with provable performance guarantees, including bounds on iteration complexity and convergence rates.
• Implements and evaluates the method on real and synthetic hypergraph datasets using semi-supervised learning benchmarks.

Experimental results

Research questions

• RQ1 Can a new dual formulation be derived for QDSFM that captures the conic structure of the problem, enabling efficient optimization?
• RQ2 Does the proposed RCD algorithm achieve linear convergence for QDSFM, and what conditions ensure this rate?
• RQ3 Can efficient conic projection algorithms be designed to support RCD with provable convergence and low computational cost?
• RQ4 How does the proposed method compare to existing solvers (e.g., PDHG, SGD, DRCD) in terms of convergence speed and prediction accuracy on hypergraph semi-supervised learning tasks?
• RQ5 What is the theoretical dependence of the convergence rate on problem parameters such as the number of hyperedges, node degrees, and regularization strength?

Key findings

• The proposed RCD algorithm achieves linear convergence for QDSFM, with iteration complexity bounded by $ O(R\mu(\beta^{-1}W^{-1},\beta^{-1}W^{-1})\log\frac{1}{\epsilon}) $, where $ \mu $ captures problem-specific condition numbers.
• The dual QDSFM problem requires projection onto a product cone, unlike the product polytope in standard DSFM, necessitating new analytic tools for convergence analysis.
• Numerical experiments on synthetic and real hypergraphs show that QRCD-MNP and QRCD-FW outperform PDHG and SGD in convergence speed and prediction accuracy.
• QRCD-MNP achieves higher accuracy than QRCD-FW due to more accurate inner-loop projections, though it incurs higher CPU time per iteration.
• The method demonstrates significant improvements in prediction accuracy over state-of-the-art methods in semi-supervised learning on hypergraphs, with faster convergence and better generalization.
• Theoretical analysis confirms that the convergence rate depends on $ N^2 \max(1, 9/(2\beta)) \max_{i,j} W_{ii}/W_{jj} $, showing sensitivity to regularization and degree imbalance.