[Paper Review] Noisy and Incomplete Boolean Matrix Factorizationvia Expectation Maximization

Lifan Liang, Songjian Lu|arXiv (Cornell University)|May 29, 2019

Error Correcting Code Techniques16 references4 citations

TL;DR

This paper proposes a novel probabilistic Boolean matrix factorization algorithm using Expectation Maximization that eliminates assumptions about latent factor shapes by modeling continuous parameters via Beta distributions and enabling gradient ascent through reparameterization. It outperforms state-of-the-art methods like LoM and message passing in handling noisy and incomplete data, especially when factor sizes vary significantly, as demonstrated in synthetic and real-world applications including gene expression analysis and brain segmentation.

ABSTRACT

Probabilistic approach to Boolean matrix factorization can provide solutions robustagainst noise and missing values with linear computational complexity. However,the assumption about latent factors can be problematic in real world applications.This study proposed a new probabilistic algorithm free of assumptions of latentfactors, while retaining the advantages of previous algorithms. Real data experimentshowed that our algorithm was favourably compared with current state-of-the-artprobabilistic algorithms.

Motivation & Objective

• To address the limitations of existing Boolean matrix factorization (BMF) methods that rely on strong assumptions about latent factor shapes, which are often unrealistic in real-world data.
• To develop a robust, scalable BMF algorithm that effectively handles noisy and incomplete data without prior assumptions on factor structure.
• To improve performance in real-world applications such as gene expression analysis and brain region segmentation where data is inherently incomplete and noisy.
• To enable effective learning of diverse factor patterns by relaxing constraints on factor size and shape through continuous parameterization.

Proposed method

• The method models latent factors using continuous parameters in [0,1] drawn from Beta distributions, allowing flexible representation without assuming fixed shapes.
• It reparameterizes the parameters from [0,1] to (−∞, +∞) to enable efficient gradient ascent optimization.
• The algorithm jointly estimates latent factors and models uniform noise within an Expectation-Maximization (EM) framework.
• The generative process uses independent Bernoulli distributions for each element in the observed matrix, factor matrices, and their logical combination via OR-AND operations.
• The likelihood is maximized using EM, where the E-step computes posterior probabilities and the M-step updates parameters via gradient ascent.
• The approach is scalable and effective for large-scale Boolean matrices due to the use of continuous, differentiable parameters.

Experimental results

Research questions

• RQ1 Can a Boolean matrix factorization algorithm achieve robust performance on noisy and incomplete data without assuming uniform or fixed shapes for latent factors?
• RQ2 How does relaxing assumptions on latent factor structure affect performance in real-world biomedical datasets with missing values and noise?
• RQ3 Does the proposed EM-based method with continuous parameterization and gradient ascent outperform existing probabilistic BMF methods like LoM and message passing?
• RQ4 To what extent can the algorithm recover meaningful biological patterns in gene expression data and spatial transcriptomics without prior knowledge of factor size or shape?

Key findings

• On synthetic data with varying factor sizes and low to moderate noise, the proposed EM algorithm outperformed both message passing and LoM, especially when factor sizes differed significantly.
• In the MovieLens dataset, the EM algorithm achieved 68.2% accuracy at 95% observed data, outperforming message passing (66.4%) and OrM (64.7%) in the 100K dataset.
• For breast cancer subtype classification, the EM algorithm achieved 81.3% accuracy, significantly outperforming LoM (77.8%) and message passing (77.7%), particularly in discriminating Her2 and Normal-like subtypes.
• In hippocampal region segmentation using spatial transcriptomics, the algorithm successfully identified spatially coherent clusters with 2 to 15 factors, even without spatial priors, indicating robustness to missing data and noise.
• The EM algorithm maintained stable performance across low-observation regimes (e.g., 1% observed data), while LoM performance degraded significantly when observed fraction dropped below 30%.
• The method demonstrated superior ability to capture subtle, high-variance patterns in data by avoiding restrictive assumptions on factor size, as evidenced by improved classification of biologically heterogeneous subtypes.