[Paper Review] Representation Learning with Contrastive Predictive Coding

Aäron van den Oord, Yazhe Li+1 more|arXiv (Cornell University)|Jul 10, 2018

Multimodal Machine Learning Applications4,505 citations

TL;DR

CPC learns unsupervised representations by predicting future latent codes with a contrastive loss, achieving strong results across speech, images, text, and reinforcement learning without labels.

ABSTRACT

While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging endeavor for artificial intelligence. In this work, we propose a universal unsupervised learning approach to extract useful representations from high-dimensional data, which we call Contrastive Predictive Coding. The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models. We use a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples. It also makes the model tractable by using negative sampling. While most prior work has focused on evaluating representations for a particular modality, we demonstrate that our approach is able to learn useful representations achieving strong performance on four distinct domains: speech, images, text and reinforcement learning in 3D environments.

Motivation & Objective

• Motivate unsupervised representation learning as a pathway to data-efficient, robust features.
• Propose a universal CPC framework that compresses data into latent representations and predicts future steps in latent space.
• Demonstrate that a contrastive, MI-maximizing objective yields useful features across multiple modalities.

Proposed method

• Encode input sequences into latent representations z_t with a non-linear encoder g_enc.
• Summarize past latents with an autoregressive model g_ar to produce a context c_t.
• Model a density ratio f_k(x_{t+k}, c_t) proportional to p(x_{t+k}|c_t)/p(x_{t+k}) to preserve mutual information.
• Train using the InfoNCE loss: L_N = -E log [ f_k(x_{t+k}, c_t) / sum_{x_j in X} f_k(x_j, c_t) ].
• Use negative sampling to enable tractable training and optimize jointly the encoder and autoregressive model.
• Any encoder and autoregressive architecture can be used; downstream tasks can use z_t or c_t (with pooling as needed).

Experimental results

Research questions

• RQ1 Can a single unsupervised CPC framework learn transferable representations across diverse data modalities (speech, images, text, RL)?
• RQ2 Does maximizing the mutual information via the InfoNCE loss yield representations that capture high-level, predictive structure over multiple timesteps?
• RQ3 How does the number of future steps predicted and the choice of negative samples affect representation quality?
• RQ4 Can CPC representations speed up or improve learning in downstream tasks such as reinforcement learning?

Key findings

• CPC achieves strong linear separability on phone and speaker classification from LibriSpeech, outperforming MFCC features and approaching supervised performance.
• In vision, CPC with a ResNet-v2-101 encoder yields top-1 accuracy of 48.7% and top-5 of 73.6% on ImageNet in unsupervised evaluation, surpassing several prior unsupervised methods.
• In NLP, CPC performs competitively with skip-thought vectors on sentiment and classification benchmarks using transfer learning from BookCorpus.
• In reinforcement learning, adding CPC as an auxiliary loss improves performance on 4 of 5 DeepMind Lab tasks, with one task showing no effect, indicating task-dependent benefits.
• Ablation studies show predicting multiple future steps improves representations, and the negative sampling strategy influences downstream performance (e.g., mixed vs same speaker) in audio experiments.