[Paper Review] Constructing a Non-Negative Low Rank and Sparse Graph with Data-Adaptive Features

Liansheng Zhuang, Shenghua Gao+4 more|arXiv (Cornell University)|Sep 3, 2014

Face and Expression Recognition27 references3 citations

TL;DR

This paper proposes a non-negative low-rank and sparse graph (NNLRS-graph) for semi-supervised learning by jointly learning data embeddings and graph construction to capture both global low-rank structures and local sparse neighborhoods. The method outperforms state-of-the-art approaches on face recognition datasets, demonstrating that joint learning of features and graph structure significantly improves classification accuracy and robustness.

ABSTRACT

This paper aims at constructing a good graph for discovering intrinsic data structures in a semi-supervised learning setting. Firstly, we propose to build a non-negative low-rank and sparse (referred to as NNLRS) graph for the given data representation. Specifically, the weights of edges in the graph are obtained by seeking a nonnegative low-rank and sparse matrix that represents each data sample as a linear combination of others. The so-obtained NNLRS-graph can capture both the global mixture of subspaces structure (by the low rankness) and the locally linear structure (by the sparseness) of the data, hence is both generative and discriminative. Secondly, as good features are extremely important for constructing a good graph, we propose to learn the data embedding matrix and construct the graph jointly within one framework, which is termed as NNLRS with embedded features (referred to as NNLRS-EF). Extensive experiments on three publicly available datasets demonstrate that the proposed method outperforms the state-of-the-art graph construction method by a large margin for both semi-supervised classification and discriminative analysis, which verifies the effectiveness of our proposed method.

Motivation & Objective

• To address the limitations of existing graph construction methods in capturing both global low-rank and local sparse structures in high-dimensional data.
• To overcome the issue of negative weights in low-rank representation, which lack physical interpretability for affinity graphs.
• To improve graph-based semi-supervised learning by jointly learning optimal data embeddings and graph structures instead of treating them separately.
• To develop a graph construction method that is both generative (capturing global structure) and discriminative (preserving local geometry).

Proposed method

• Formulates a non-negative low-rank and sparse representation problem to learn a graph where edge weights are non-negative coefficients from linear combinations of other data points.
• Imposes non-negativity to ensure each data point lies in the convex hull of its neighbors, enhancing interpretability and physical meaning.
• Enforces sparsity to capture local linear structures and low-rankness to model the global mixture of subspaces in the data.
• Introduces a joint optimization framework, NNLRS-EF, that simultaneously learns the data embedding matrix and constructs the NNLRS-graph within a single objective function.
• Solves the optimization problem using an alternating direction method of multipliers (ADMM) to handle the non-convex, non-smooth, and non-negative constraints.
• Employs a parameterized regularization with β balancing sparsity and low-rankness, and λ handling gross corruption errors in data.

Experimental results

Research questions

• RQ1 Can a graph constructed via non-negative low-rank and sparse representation better capture both global and local data structures than existing methods?
• RQ2 How does the joint learning of data embeddings and graph construction improve performance compared to sequential processing?
• RQ3 What is the impact of the β parameter on the trade-off between sparsity and low-rankness in graph construction?
• RQ4 Does incorporating data embedding within the graph learning framework lead to better classification accuracy and robustness?
• RQ5 How sensitive is the NNLRS-graph to parameter settings like β and λ in real-world semi-supervised learning tasks?

Key findings

• The NNLRS-graph achieves significantly lower classification error rates than state-of-the-art methods on the YaleB and PIE face databases, with error rates dropping to 5.52% at 50% labeled samples on PIE.
• Joint learning via NNLRS-EF outperforms the baseline PCA+NNLRS method, demonstrating that end-to-end optimization of features and graph structure yields better performance.
• Parameter sensitivity analysis shows that both sparsity (β) and low-rankness are critical: performance degrades when β is too large (e.g., β=100) or too small (e.g., β=0), with β=0.2 yielding optimal results.
• The NNLRS-graph maintains strong performance across varying label proportions, with error rates decreasing steadily from 38.93% (10% labeled) to 6.52% (60% labeled) on the PIE dataset.
• The method is robust to data corruption, as evidenced by stable performance when λ is fixed at 10, indicating effective handling of gross errors.
• The proposed framework achieves state-of-the-art performance in both semi-supervised classification and discriminative analysis, confirming its effectiveness in revealing intrinsic data structures.