[论文解读] Clustering with Deep Learning: Taxonomy and New Methods

Elie Aljalbout, Vladimir Golkov+3 more|arXiv (Cornell University)|Jan 23, 2018

Anomaly Detection Techniques and Applications参考文献 27被引用 194

一句话总结

本文提出了一个用于使用深度神经网络的聚类方法的系统化分类法，并通过一个案例研究证实，在 MNIST 上达到具有竞争力的、甚至有时是最先进的聚类性能。

ABSTRACT

Clustering methods based on deep neural networks have proven promising for clustering real-world data because of their high representational power. In this paper, we propose a systematic taxonomy of clustering methods that utilize deep neural networks. We base our taxonomy on a comprehensive review of recent work and validate the taxonomy in a case study. In this case study, we show that the taxonomy enables researchers and practitioners to systematically create new clustering methods by selectively recombining and replacing distinct aspects of previous methods with the goal of overcoming their individual limitations. The experimental evaluation confirms this and shows that the method created for the case study achieves state-of-the-art clustering quality and surpasses it in some cases.

研究动机与目标

• Develop a unified taxonomy for clustering methods that rely on deep neural networks.
• Identify configurable building blocks to enable systematic design of new clustering methods.
• Demonstrate the taxonomy’s utility through a case study that builds a novel method.
• Show that recombining building blocks can overcome limitations of existing methods.

提出的方法

• Define a modular taxonomy with building blocks: neural network architecture, deep feature set, non-clustering loss, clustering loss, loss combination, cluster updates, and post-training re-evaluation (2.1–2.7).
• Survey existing deep-learning clustering methods and map them to the taxonomy to analyze strengths and limitations.
• Propose a case-study method using a CNN-based encoder with autoencoder-style reconstruction loss, a clustering-specific loss (cluster hardening), and a final re-run of k-means on learned representations.
• Use a two-phase training process: phase one pretraining with reconstruction loss, phase two fine-tuning with both reconstruction and clustering losses.
• Evaluate via ACC and NMI on MNIST and COIL20 to illustrate improved clustering quality and balanced performance across datasets.

实验结果

研究问题

• RQ1 How can clustering methods leveraging deep networks be systematically categorized?
• RQ2 Can composing building blocks from the taxonomy yield new methods with improved clustering performance?
• RQ3 Does a case-study method built from the taxonomy achieve state-of-the-art or competitive results on standard benchmarks (e.g., MNIST)?

主要发现

• The taxonomy enables systematic method construction by recombining building blocks.
• The case-study method achieves 0.923 NMI on MNIST, surpassing state-of-the-art in that metric.
• The proposed approach yields balanced results across MNIST and COIL20 compared to other methods.
• The visualization shows clustering-friendly latent spaces after applying the proposed method.
• The study demonstrates that taxonomy-guided design can outperform prior approaches on at least one benchmark.
• The authors report that the method outperforms previous state-of-the-art on MNIST in NMI and offers balanced performance across datasets.