[Paper Review] KiWi: A Scalable Subspace Clustering Algorithm for Gene Expression Analysis

Obi L. Griffith, Byron J. Gao+4 more|ArXiv.org|Apr 13, 2009

Gene expression and cancer classification29 references4 citations

TL;DR

KiWi is a scalable subspace clustering algorithm designed for gene expression data that efficiently discovers order-preserving sub-matrix (OPSM) clusters, including computationally expensive 'twig clusters' with few genes and many experiments. It leverages a novel indexing and pruning strategy to enable detection of both positive and negative correlations, significantly outperforming prior methods in scalability and biological relevance on large datasets.

ABSTRACT

Subspace clustering has gained increasing popularity in the analysis of gene expression data. Among subspace cluster models, the recently introduced order-preserving sub-matrix (OPSM) has demonstrated high promise. An OPSM, essentially a pattern-based subspace cluster, is a subset of rows and columns in a data matrix for which all the rows induce the same linear ordering of columns. Existing OPSM discovery methods do not scale well to increasingly large expression datasets. In particular, twig clusters having few genes and many experiments incur explosive computational costs and are completely pruned off by existing methods. However, it is of particular interest to determine small groups of genes that are tightly coregulated across many conditions. In this paper, we present KiWi, an OPSM subspace clustering algorithm that is scalable to massive datasets, capable of discovering twig clusters and identifying negative as well as positive correlations. We extensively validate KiWi using relevant biological datasets and show that KiWi correctly assigns redundant probes to the same cluster, groups experiments with common clinical annotations, differentiates real promoter sequences from negative control sequences, and shows good association with cis-regulatory motif predictions.

Motivation & Objective

• To address the scalability limitations of existing OPSM discovery methods on large-scale gene expression datasets.
• To enable the detection of 'twig clusters'—small gene groups with many experiments—often pruned by prior methods due to high computational cost.
• To identify both positive and negative correlations in gene expression patterns across experimental conditions.
• To improve biological interpretability by correctly grouping redundant probes, clinically annotated experiments, and regulatory sequences.
• To provide a scalable solution capable of handling massive gene expression datasets while preserving subspace cluster patterns.

Proposed method

• KiWi employs a novel indexing structure based on row-wise ordering of columns to efficiently represent and query potential OPSM clusters.
• It uses a bottom-up approach to incrementally build candidate clusters, starting from individual genes and expanding to larger submatrices.
• A pruning strategy eliminates non-promising candidates early by leveraging the order-preserving property and frequency thresholds.
• The algorithm integrates a dynamic thresholding mechanism to balance sensitivity and computational cost, especially for sparse or high-dimensional data.
• It supports both positive and negative correlation detection by considering reverse orderings of column indices in the same matrix.
• KiWi leverages database-style indexing and query optimization techniques to scale to large datasets, reducing exponential growth in search space.

Experimental results

Research questions

• RQ1 Can a subspace clustering algorithm efficiently discover small, highly coregulated gene clusters (twig clusters) across many experimental conditions?
• RQ2 How can order-preserving sub-matrices (OPSMs) be discovered at scale without prohibitive computational cost?
• RQ3 To what extent can KiWi detect both positive and negative correlations in gene expression patterns?
• RQ4 Does KiWi improve biological coherence in clustering results compared to existing methods?
• RQ5 Can KiWi correctly group redundant probes and link clusters to clinical annotations and regulatory motifs?

Key findings

• KiWi successfully identified and grouped redundant probes into the same cluster, demonstrating consistency with known biological redundancy.
• The algorithm correctly grouped experiments with common clinical annotations, indicating alignment with biological metadata.
• KiWi differentiated real promoter sequences from negative control sequences, showing biological specificity in cluster assignments.
• It showed strong association with cis-regulatory motif predictions, supporting functional relevance of discovered clusters.
• KiWi achieved significant scalability improvements, enabling discovery of previously intractable twig clusters that were pruned by earlier methods.
• The method demonstrated robust performance on large-scale gene expression datasets, maintaining high precision and recall in subspace cluster detection.