[Paper Review] Low-Rank Boolean Matrix Approximation by Integer Programming

Réka Á. Kovács, Oktay Günlük+1 more|arXiv (Cornell University)|Mar 13, 2018

Sparse and Compressive Sensing Techniques13 references3 citations

TL;DR

This paper presents the first mixed-integer programming (MIP) formulation for low-rank Boolean matrix approximation with polynomially many variables and constraints, enabling exact solutions for realistic problem sizes. The method uses integer programming with preprocessing and symmetry reduction to efficiently solve Boolean matrix factorization, achieving high-quality approximations on synthetic and real-world datasets including SPECT, Primary Tumor, and U.S. Congressional Voting Records.

ABSTRACT

Low-rank approximations of data matrices are an important dimensionality reduction tool in machine learning and regression analysis. We consider the case of categorical variables, where it can be formulated as the problem of finding low-rank approximations to Boolean matrices. In this paper we give what is to the best of our knowledge the first integer programming formulation that relies on only polynomially many variables and constraints, we discuss how to solve it computationally and report numerical tests on synthetic and real-world data.

Motivation & Objective

• To develop an exact, computationally viable method for low-rank Boolean matrix approximation, which is NP-hard and typically solved via heuristics.
• To formulate the Boolean matrix approximation problem as a mixed-integer program with only polynomially many variables and constraints, enabling efficient solution via commercial solvers.
• To enable exact solutions for real-world datasets previously accessible only through heuristic approximation methods.
• To incorporate preprocessing steps—removing all-zero rows/columns and handling duplicate rows/columns—while preserving objective equivalence.
• To demonstrate the effectiveness of the approach on synthetic and real-world datasets with quantitative error and runtime evaluation.

Proposed method

• Formulate the Boolean matrix approximation problem as a mixed-integer program (MIP) using binary variables for factor matrices $ C \in \{0,1\}^{n \times k} $ and $ R \in \{0,1\}^{k \times m} $.
• Define the objective function as minimizing the Frobenius norm of the approximation error $ \|X - C \circ R\|_F^2 $, where $ \circ $ denotes Boolean matrix multiplication.
• Use a polynomial-sized MIP formulation that avoids exponential variable/constraint growth, making it scalable to realistic problem sizes.
• Apply preprocessing: remove all-zero rows and columns, and handle duplicates by scaling the objective function by $ \alpha_i \beta_j $, where $ \alpha_i $ and $ \beta_j $ are repetition counts.
• Leverage symmetry reduction and variable fixing to improve computational efficiency during MIP solution via CPLEX.
• Solve the MIP using commercial optimization software (CPLEX) on standard hardware, with a 2-hour time limit for large instances.

Experimental results

Research questions

• RQ1 Can an exact integer programming formulation for low-rank Boolean matrix approximation be constructed with only polynomially many variables and constraints?
• RQ2 How does the proposed MIP formulation compare to existing heuristic methods in terms of solution quality and computational efficiency on real-world data?
• RQ3 To what extent do preprocessing steps—removing zero rows/columns and handling duplicates—improve the scalability and performance of the MIP approach?
• RQ4 Does the MIP formulation yield better or more interpretable factorizations than existing heuristic methods, particularly in cases where linear algebraic rank differs from Boolean rank?
• RQ5 Can the MIP approach solve real-world Boolean matrix approximation problems to optimality or near-optimality within practical time limits?

Key findings

• The proposed MIP formulation is the first to rely on only polynomially many variables and constraints, enabling exact solution of low-rank Boolean matrix approximation for realistic problem sizes.
• On synthetic data with exact Boolean rank 5 and 20 features, the improved formulation with preprocessing reduced average solution time significantly compared to the full model.
• For noisy synthetic data, problem complexity increased with noise level, but preprocessing helped mitigate this by reducing dimensionality.
• On real-world datasets—including SPECT (267×22), Primary Tumor (339×24), and U.S. Congress (435×32)—rank-5 approximations achieved at least 80% data reconstruction accuracy even with suboptimal solutions.
• The approximation error curves were non-monotonic due to suboptimal solutions, but the results suggest that optimal solutions would show decreasing error with increasing rank $ k $.
• The method outperforms prior heuristics in solution quality and provides interpretable factorizations, as demonstrated on the congressional voting dataset where Boolean rank revealed distinct voting roles.