[論文レビュー] Deep Discriminative Clustering Analysis

Jianlong Chang, Yiwen Guo+4 more|arXiv (Cornell University)|May 5, 2019

Domain Adaptation and Few-Shot Learning参考文献 75被引用数 42

ひとこと要約

DDC は、小さなミニバッチ学習でグローバルおよびローカル制約を用いて判別表現とパターン間の関係を共同学習し、事前に指定されたクラスタ数を必要とせず、画像、テキスト、音声データセットにおけるクラスタリングで最先端を達成します。

ABSTRACT

Traditional clustering methods often perform clustering with low-level indiscriminative representations and ignore relationships between patterns, resulting in slight achievements in the era of deep learning. To handle this problem, we develop Deep Discriminative Clustering (DDC) that models the clustering task by investigating relationships between patterns with a deep neural network. Technically, a global constraint is introduced to adaptively estimate the relationships, and a local constraint is developed to endow the network with the capability of learning high-level discriminative representations. By iteratively training the network and estimating the relationships in a mini-batch manner, DDC theoretically converges and the trained network enables to generate a group of discriminative representations that can be treated as clustering centers for straightway clustering. Extensive experiments strongly demonstrate that DDC outperforms current methods on eight image, text and audio datasets concurrently.

研究の動機と目的

• learnable, discriminative representations rather than fixed, low-level features.
• Propose a deep network framework that jointly learns representations and inter-pattern relationships under global and local constraints.
• Enable automatic estimation of cluster structure (including number of clusters) without predefined k.
• Provide theoretical guarantees on convergence and discuss practical mini-batch optimization.

提案手法

• Model clustering as learning pairwise relationships R with a deep network f(x; w) producing discriminative indicator features I = f(x; w).
• Impose a global constraint via reflexivity, symmetry, and transitivity to align R with the network-generated similarities.
• Decompose similarity as g(x_i, x_j; w) = f(x_i; w) · f(x_j; w) to ensure symmetry.
• Introduce a local non-negativity constraint I_h >= 0 to encourage discriminative indicators.
• Optimize a joint objective combining a binary cross-entropy on R with a penalty term ε(R, R̄) that links R to the coarse similarity R̄ from f.
• Utilize an alternating mini-batch optimization that updates w and R and leverages spectral clustering on I for R when w is fixed.
• Provide a constraint-layer implementation to realize the indicator features and ensure reflexivity and non-negativity.

実験結果

リサーチクエスチョン

• RQ1 Can a deep network learn discriminative clustering centers while simultaneously estimating the inter-pattern relationships in an unsupervised setting?
• RQ2 Do global (relationship-level) and local (indicator-level) constraints enable convergence and improve clustering quality across diverse data modalities?
• RQ3 Can the number of clusters be automatically inferred from data without predefining k?
• RQ4 Is mini-batch optimization sufficient to scale to large datasets and maintain convergence guarantees?

主な発見

• DDC achieves best performance across eight datasets spanning images, text, and audio compared with traditional, representation-based, and single-stage deep clustering methods.
• Representation learning integrated with clustering yields superior performance, validating the joint learning premise.
• DDC handles large-scale datasets and high-dimensional features via mini-batch optimization and deep representations.
• Theoretical results (Theorems 1–3) establish that discriminative clustering centers are learned, that a global minimum is attainable when the indicator feature dimension is at least k, and that the algorithm converges under identically distributed mini-batches.
• Ablation studies indicate the model can infer the (unobservable) number of clusters and remains robust to initialization.