[Paper Review] Fast Convolutive Nonnegative Matrix Factorization Through Coordinate and Block Coordinate Updates

Anthony Degleris, Ben Antin+2 more|arXiv (Cornell University)|Jun 29, 2019

Blind Source Separation Techniques28 references4 citations

TL;DR

This paper proposes fast, scalable algorithms—Hierarchical Alternating Least Squares (HALS) and Alternating Nonnegative Least Squares (ANLS)—for Convolutive Nonnegative Matrix Factorization (CNMF), extending efficient NMF methods to model temporal motifs in time series. The proposed methods outperform traditional multiplicative updates on large-scale synthetic and real-world data, offering faster convergence and improved accuracy.

ABSTRACT

Identifying recurring patterns in high-dimensional time series data is an important problem in many scientific domains. A popular model to achieve this is convolutive nonnegative matrix factorization (CNMF), which extends classic nonnegative matrix factorization (NMF) to extract short-lived temporal motifs from a long time series. Prior work has typically fit this model by multiplicative parameter updates---an approach widely considered to be suboptimal for NMF, especially in large-scale data applications. Here, we describe how to extend two popular and computationally scalable NMF algorithms---Hierarchical Alternating Least Squares (HALS) and Alternatining Nonnegative Least Squares (ANLS)---for the CNMF model. Both methods demonstrate performance advantages over multiplicative updates on large-scale synthetic and real world data.

Motivation & Objective

• Address the lack of efficient, scalable optimization algorithms for Convolutive Nonnegative Matrix Factorization (CNMF), which is critical for modeling short-lived temporal patterns in time series.
• Overcome the limitations of the widely used Multiplicative Update (MU) method, which is known to be suboptimal for large-scale NMF and is similarly ineffective for CNMF.
• Extend two high-performance NMF algorithms—HALS and ANLS—into the CNMF framework to enable faster and more accurate factorization of large-scale time series data.
• Provide new mathematical reformulations of the CNMF objective function to offer deeper interpretability and computational advantages.
• Demonstrate the superiority of coordinate and block coordinate descent-based methods (HALS and ANLS) over MU in both synthetic and real-world applications.

Proposed method

• Adapt the HALS algorithm to CNMF by deriving closed-form updates for the convolutional basis tensors (W) and activation matrices (H) using coordinate descent principles.
• Apply ANLS to CNMF by alternately solving nonnegative least squares subproblems for W and H, leveraging the block coordinate descent framework.
• Derive closed-form solutions for W and H updates using Lagrangian relaxation and Karush-Kuhn-Tucker (KKT) conditions, ensuring nonnegativity constraints are satisfied.
• Reformulate the CNMF objective function using tensor and matrix operations to enable efficient computation and clearer interpretation of temporal motifs.
• Implement alternating updates between W and H, where each update minimizes the reconstruction error under nonnegativity constraints, ensuring convergence and scalability.
• Use the structure of the convolutional model to isolate relevant temporal segments during optimization, reducing computational complexity per update step.

Experimental results

Research questions

• RQ1 Can established, high-performance NMF algorithms like HALS and ANLS be effectively extended to the CNMF framework to improve optimization efficiency?
• RQ2 How do HALS and ANLS compare to the standard Multiplicative Update (MU) method in terms of convergence speed and reconstruction accuracy on large-scale time series data?
• RQ3 What mathematical reformulations of the CNMF objective function enable more efficient and interpretable optimization?
• RQ4 Can coordinate and block coordinate descent methods provide a scalable and robust alternative to MU for CNMF in real-world applications?
• RQ5 What are the computational and statistical advantages of using HALS and ANLS over MU when extracting short-lived temporal motifs from high-dimensional time series?

Key findings

• HALS and ANLS demonstrate significantly faster convergence and lower reconstruction error compared to the Multiplicative Update (MU) method on large-scale synthetic and real-world time series datasets.
• The proposed HALS and ANLS algorithms achieve superior performance in identifying short-lived temporal motifs in spectrograms and neural spiking data, outperforming MU in both speed and accuracy.
• Closed-form updates derived via KKT conditions ensure nonnegativity and enable efficient optimization, with the H-update depending only on a local temporal segment of the residual.
• The reformulation of the CNMF objective function enables clearer interpretation of the convolutional basis as spatiotemporal features with duration L, improving model interpretability.
• The algorithms scale effectively to large datasets, making them suitable for real-world applications in neuroscience, audio processing, and medical data mining.
• Empirical results show that HALS and ANLS converge in fewer iterations than MU while maintaining or improving reconstruction quality, especially in high-noise or high-dimensionality regimes.