[Paper Review] Auto-Encoder based Co-Training Multi-View Representation Learning

Run‐kun Lu, Jian–wei Liu+3 more|arXiv (Cornell University)|Jan 9, 2022

Domain Adaptation and Few-Shot Learning17 references4 citations

TL;DR

This paper proposes Auto-Encoder based Co-Training Multi-View Learning (ACMVL), a neural network-based method that jointly learns consistent and complementary representations across multiple views by co-training view-specific auto-encoders with a shared supervised network. The approach improves representation learning through weight-sharing and alternating co-training, achieving state-of-the-art performance in classification and clustering tasks on benchmark datasets.

ABSTRACT

Multi-view learning is a learning problem that utilizes the various representations of an object to mine valuable knowledge and improve the performance of learning algorithm, and one of the significant directions of multi-view learning is sub-space learning. As we known, auto-encoder is a method of deep learning, which can learn the latent feature of raw data by reconstructing the input, and based on this, we propose a novel algorithm called Auto-encoder based Co-training Multi-View Learning (ACMVL), which utilizes both complementarity and consistency and finds a joint latent feature representation of multiple views. The algorithm has two stages, the first is to train auto-encoder of each view, and the second stage is to train a supervised network. Interestingly, the two stages share the weights partly and assist each other by co-training process. According to the experimental result, we can learn a well performed latent feature representation, and auto-encoder of each view has more powerful reconstruction ability than traditional auto-encoder.

Motivation & Objective

• To address the limitations of existing multi-view subspace learning methods that rely on matrix factorization and struggle with large-scale data.
• To jointly model both consistency (shared features) and complementarity (view-specific features) in multi-view data using deep neural networks.
• To develop a scalable, mini-batch compatible training strategy that improves auto-encoder performance through co-training with a supervised network.
• To accelerate convergence and enhance representation quality by enabling partial parameter sharing between auto-encoders and the supervised network.

Proposed method

• The framework consists of view-specific auto-encoders and a shared supervised network, with the joint latent representation extracted from the first hidden layer of the supervised network.
• Each view's auto-encoder learns a view-specific latent representation, which is mapped into a shared space via weight sharing to enforce consistency.
• An alternating co-training strategy is employed where the supervised network provides gradient signals to improve the auto-encoders’ encoders, and auto-encoder features enhance the supervised network's training.
• The method uses mini-batch stochastic gradient descent, enabling scalability to large-scale datasets unlike previous matrix-factorization-based approaches.
• The joint representation is formed by summing the weighted, shared-space projections of each view’s special features, followed by a nonlinear activation.
• The model is trained end-to-end with reconstruction loss on auto-encoders and cross-entropy loss on the supervised network.

Experimental results

Research questions

• RQ1 Can a deep learning-based co-training framework effectively learn joint latent representations that capture both consistency and complementarity in multi-view data?
• RQ2 How does alternating co-training with shared parameters improve the performance of individual view auto-encoders and the overall joint representation?
• RQ3 Can the proposed method scale efficiently to large-scale datasets using mini-batch training, unlike prior matrix-factorization-based approaches?
• RQ4 To what extent does the joint latent representation outperform individual view representations in downstream classification and clustering tasks?

Key findings

• The ACMVL framework significantly improves the reconstruction ability of individual view auto-encoders compared to standard auto-encoders, as shown in ablation studies.
• In classification tasks, the joint latent representation (LR-ACMVL) achieves higher accuracy and F1 scores than individual view representations (LR-AE) and the co-trained auto-encoder features (LR-AE-ACMVL).
• In clustering tasks, the joint latent representation (GMM-ACMVL) achieves substantially higher NMI and lower JC scores than individual view representations and auto-encoder-only baselines.
• For the WebKB dataset, the joint representation achieved an NMI of 0.5809 and JC of 0.0310, outperforming the best individual view (NMI: 0.4739, JC: 0.1593).
• On the 20NG dataset, the joint representation achieved an NMI of 0.3171 and JC of 0.5100, significantly outperforming the best individual view (NMI: 0.1968, JC: 0.4533).
• The co-training strategy accelerates convergence and enhances the learning performance of each auto-encoder, demonstrating the effectiveness of parameter sharing and mutual supervision.