[Paper Review] Adjacency-constrained hierarchical clustering of a band similarity matrix with application to Genomics

Christophe Ambroise, Alia Dehman+3 more|arXiv (Cornell University)|Feb 5, 2019

Gene expression and cancer classificationBiochemistry, Genetics and Molecular Biology39 references33 citations

TL;DR

The paper presents an adjacency-constrained Ward HAC algorithm with banded similarity matrices to achieve quasi-linear time and linear space complexity, enabling scalable genomic segmentation. It also provides an R package adjclust for GWAS and Hi-C data analyses.

ABSTRACT

Motivation: Genomic data analyses such as Genome-Wide Association Studies (GWAS) or Hi-C studies are often faced with the problem of partitioning chromosomes into successive regions based on a similarity matrix of high-resolution, locus-level measurements. An intuitive way of doing this is to perform a modified Hierarchical Agglomerative Clustering (HAC), where only adjacent clusters (according to the ordering of positions within a chromosome) are allowed to be merged. A major practical drawback of this method is its quadratic time and space complexity in the number of loci, which is typically of the order of 10^4 to 10^5 for each chromosome. Results: By assuming that the similarity between physically distant objects is negligible, we propose an implementation of this adjacency-constrained HAC with quasi-linear complexity. Our illustrations on GWAS and Hi-C datasets demonstrate the relevance of this assumption, and show that this method highlights biologically meaningful signals. Thanks to its small time and memory footprint, the method can be run on a standard laptop in minutes or even seconds. Availability and Implementation: Software and sample data are available as an R package, adjclust, that can be downloaded from the Comprehensive R Archive Network (CRAN).

Motivation & Objective

• Motivate the need to partition chromosomes into regions based on locus-level similarity matrices in GWAS and Hi-C studies.
• Develop an adjacency-constrained hierarchical clustering method that is scalable to large genomic datasets.
• Leverage a banded similarity assumption to achieve near-linear time and linear space complexity.
• Provide an efficient implementation and practical guidance for applying the method to genomic data.

Proposed method

• Extend hierarchical agglomerative clustering by allowing merges only between adjacent clusters along a predefined genomic order.
• Adopt Ward’s linkage and express linkage updates in terms of pre-computed similarity sums, enabling constant-time linkage calculations per candidate merge via a pencil-sum representation.
• Impose a banded similarity assumption where s_ij = 0 for |i-j| >= h, reducing storage to O(ph) and time to O(p(h+log p)).
• Store candidate merges in a min-heap to efficiently identify and update the best adjacent merge at each step, achieving overall complexity O(p(h+log p)).
• Provide an R implementation adjclust (with C core) and guidance for model selection (broken stick, slope heuristic) to choose clustering levels.

Experimental results

Research questions

• RQ1 Does an adjacency-constrained HAC with a banded similarity matrix yield biologically meaningful partitions in genomics?
• RQ2 Can the banded assumption (distance-based sparsity) provide substantial computational gains without compromising clustering quality?
• RQ3 How does the proposed method perform on GWAS LD blocks and Hi-C data in terms of dendrogram structure and interpretability?
• RQ4 What practical guidance can be given for choosing the bandwidth h and for model selection in genomic contexts?

Key findings

• The adjacency-constrained Ward HAC with a banded similarity matrix achieves quasi-linear time and linear space complexity, O(p(h+log p)) and O(ph) respectively.
• Pre-computing pencil-based sums allows constant-time calculation of Ward linkages for candidate merges.
• Using a min-heap to manage candidate fusions yields an overall efficient algorithm suitable for large genomic datasets.
• Empirical results on GWAS LD blocks and Hi-C data show that the banded approach preserves dendrogram structure and yields biologically meaningful partitions, with higher efficiency than full-band methods.
• For GWAS, reducing bandwidth still produces nearly identical dendrograms to the full bandwidth when h is moderately large (e.g., h ≥ 2000).
• The Hi-C analyses demonstrate that sparse, banded clustering significantly reduces computation time and memory while maintaining interpretability related to TAD-like structures and DI enrichment at boundaries.