[Paper Review] On Linear Identifiability of Learned Representations

Geoffrey Roeder, Luke Metz+1 more|arXiv (Cornell University)|Jul 1, 2020

Neural Networks and Applications32 references16 citations

TL;DR

This paper establishes that a broad family of discriminative representation learning models—including state-of-the-art architectures like BERT, GPT-2, GPT-3, and CPC—are identifiable up to a linear transformation in function space. The authors prove that, under mild conditions, different training runs converge to representations that are equivalent via linear transformation, enabling reproducible and stable representations with sufficient data and capacity.

ABSTRACT

Identifiability is a desirable property of a statistical model: it implies that the true model parameters may be estimated to any desired precision, given sufficient computational resources and data. We study identifiability in the context of representation learning: discovering nonlinear data representations that are optimal with respect to some downstream task. When parameterized as deep neural networks, such representation functions typically lack identifiability in parameter space, because they are overparameterized by design. In this paper, building on recent advances in nonlinear ICA, we aim to rehabilitate identifiability by showing that a large family of discriminative models are in fact identifiable in function space, up to a linear indeterminacy. Many models for representation learning in a wide variety of domains have been identifiable in this sense, including text, images and audio, state-of-the-art at time of publication. We derive sufficient conditions for linear identifiability and provide empirical support for the result on both simulated and real-world data.

Motivation & Objective

• To establish theoretical conditions under which deep representation learning models are identifiable in function space, despite overparameterization in parameter space.
• To bridge the gap between recent nonlinear ICA identifiability results and practical discriminative models used in representation learning.
• To demonstrate that state-of-the-art models in NLP, vision, and audio are identifiable in function space up to linear transformations.
• To provide empirical evidence that increasing data size, model capacity, and optimization progress lead to representations that converge to linearly equivalent forms.
• To advocate for identifiable models as a path toward reproducible, efficient, and ethically transparent representation learning.

Proposed method

• The paper defines a general discriminative model family parameterized by deep neural networks, where the representation function fθ maps input x to a latent representation, and the model is trained to predict y given x and a set S of possible y-values.
• It introduces a canonical mathematical form for models that subsumes supervised, self-supervised, and contrastive learning frameworks, enabling unified theoretical analysis.
• The authors derive sufficient conditions for linear identifiability, showing that under mild regularity and diversity conditions, different representations learned from the same data distribution are related by a linear transformation in the limit of infinite data.
• The proof relies on analyzing the structure of the representation function and its interaction with the prediction objective, leveraging tools from nonlinear ICA and function space analysis.
• Empirical validation is performed on both synthetic and real-world data (e.g., LM1B, image, audio), measuring linear alignment between representations from independent training runs.
• The method includes training multiple instances of the same model on the same data with different random seeds and evaluating the degree of linear equivalence between their learned representations using linear probing and alignment metrics.

Experimental results

Research questions

• RQ1 Can deep representation learning models be identifiable in function space despite overparameterization in parameter space?
• RQ2 What conditions ensure that different representations learned from the same data distribution are equivalent up to a linear transformation?
• RQ3 Do state-of-the-art models like BERT, GPT-2, GPT-3, and CPC satisfy these identifiability conditions?
• RQ4 How does increasing data size, model capacity, and optimization progress affect the convergence of representations to linearly equivalent forms?
• RQ5 What are the practical and ethical implications of linear identifiability for reproducibility and bias in representation learning?

Key findings

• The paper proves that a broad class of discriminative models, including those used in state-of-the-art NLP and vision systems, are identifiable in function space up to a linear transformation under mild conditions.
• Empirical results show that representations from independent training runs on the same data distribution become increasingly linearly aligned as dataset size and model capacity grow.
• Models such as BERT, GPT-2, GPT-3, and CPC are shown to be members of the identifiable model family, supporting their theoretical identifiability.
• The degree of linear alignment between representations improves with more data and better optimization, indicating convergence toward a stable, unique solution in function space.
• The results suggest that identifiable models require only one training run to reach optimal, reproducible representations, reducing computational waste and resource consumption.
• The work highlights a risk: identifiable models may fix biased representations if the training data or objective is biased, underscoring the need for ethical model cards and bias audits.