[Paper Review] On Restricted Nonnegative Matrix Factorization

Dmitry Chistikov, Stefan Kiefer+3 more|arXiv (Cornell University)|May 23, 2016

Matrix Theory and Algorithms2 references3 citations

TL;DR

This paper establishes a deep connection between restricted nonnegative matrix factorization (RNMF) and probabilistic automata, resolving an open question posed by Paz in 1971 by showing that minimal covering probabilistic automata do not always exist with rational transition probabilities. The key result is that a rational matrix may require irrational entries in its minimal RNMF factors, even when the matrix has low rational rank, thus falsifying a long-standing claim from 1974.

ABSTRACT

Nonnegative matrix factorization (NMF) is the problem of decomposing a given nonnegative $n imes m$ matrix $M$ into a product of a nonnegative $n imes d$ matrix $W$ and a nonnegative $d imes m$ matrix $H$. Restricted NMF requires in addition that the column spaces of $M$ and $W$ coincide. Finding the minimal inner dimension $d$ is known to be NP-hard, both for NMF and restricted NMF. We show that restricted NMF is closely related to a question about the nature of minimal probabilistic automata, posed by Paz in his seminal 1971 textbook. We use this connection to answer Paz's question negatively, thus falsifying a positive answer claimed in 1974. Furthermore, we investigate whether a rational matrix $M$ always has a restricted NMF of minimal inner dimension whose factors $W$ and $H$ are also rational. We show that this holds for matrices $M$ of rank at most $3$ and we exhibit a rank-$4$ matrix for which $W$ and $H$ require irrational entries.

Motivation & Objective

• To investigate whether rational nonnegative matrices always admit a restricted NMF with rational factors.
• To resolve an open question posed by Paz (1971) on the existence of minimal covering probabilistic automata with rational transition probabilities.
• To clarify the relationship between restricted NMF and the nested polytope problem (NPP).
• To determine the conditions under which the restricted nonnegative rank of a rational matrix can be achieved with rational factor matrices.
• To examine the computational complexity and algebraic structure of RNMF, particularly over the rationals.

Proposed method

• Established a formal correspondence between restricted NMF and the coverability relation in labelled Markov chains (LMCs), linking RNMF to probabilistic automata theory.
• Used geometric reasoning on the nested polytope problem (NPP) to analyze minimal vertex solutions, translating matrix factorization constraints into polyhedral geometry.
• Constructed a specific rank-4 rational matrix M for which no 5-point rational solution exists in the NPP, proving that the restricted nonnegative rank over Q is 6.
• Employed slack functions and interval analysis to rigorously prove that the minimal vertex configuration in the NPP requires irrational coordinates.
• Leveraged the duality between RNMF and minimal probabilistic automata to show that a minimal covering automaton may not exist with rational probabilities.
• Used algebraic techniques to demonstrate that the minimal RNMF of a rational matrix may require irrational entries in the factor matrices W and H.

Experimental results

Research questions

• RQ1 Does every rational matrix admit a restricted NMF with rational factors W and H at the minimal inner dimension?
• RQ2 Can a minimal probabilistic automaton with rational transition probabilities always cover a given LMC, as claimed in 1974?
• RQ3 Is the restricted nonnegative rank of a rational matrix always achievable with rational factor matrices?
• RQ4 What is the minimal number of vertices required for a nested polytope in the NPP when the input polytopes are rational?
• RQ5 Can the existence of irrational entries in RNMF factors be characterized algebraically or geometrically?

Key findings

• A rational matrix of rank 4 exists for which no rational restricted NMF exists at the minimal inner dimension, proving that irrational entries in W and H are necessary.
• The restricted nonnegative rank of this matrix over the rationals is 6, while its nonnegative rank is 5, demonstrating a strict separation between rational and real RNMF.
• The minimal covering probabilistic automaton for a certain LMC cannot have rational transition probabilities, thus falsifying a claim from 1974.
• The minimal vertex solution in the nested polytope problem for this instance requires irrational coordinates, confirming that rational solutions do not always exist.
• For matrices of rank at most 3, rational RNMF always exists at the minimal inner dimension, showing a threshold at rank 4.
• The slack function analysis proves that the x-coordinate of a key vertex in the NPP must be exactly 2−√2, an irrational number, confirming the necessity of irrational entries.