[Paper Review] Contrastive losses as generalized models of global epistasis

David H. Brookes, Jakub Otwinowski+1 more|arXiv (Cornell University)|May 4, 2023

Evolution and Genetic Dynamics4 citations

TL;DR

This paper proposes using supervised contrastive losses—particularly the Bradley-Terry loss—as a data-efficient method to recover the latent ranking function underlying fitness functions in biological sequences, especially when global epistasis distorts the data. It demonstrates that contrastive losses outperform mean squared error (MSE) loss in learning sparse, globally epistatic fitness landscapes, with consistent improvements on benchmark tasks like FLIP, even under severe data sparsity.

ABSTRACT

Fitness functions map large combinatorial spaces of biological sequences to properties of interest. Inferring these multimodal functions from experimental data is a central task in modern protein engineering. Global epistasis models are an effective and physically-grounded class of models for estimating fitness functions from observed data. These models assume that a sparse latent function is transformed by a monotonic nonlinearity to emit measurable fitness. Here we demonstrate that minimizing supervised contrastive loss functions, such as the Bradley-Terry loss, is a simple and flexible technique for extracting the sparse latent function implied by global epistasis. We argue by way of a fitness-epistasis uncertainty principle that the nonlinearities in global epistasis models can produce observed fitness functions that do not admit sparse representations, and thus may be inefficient to learn from observations when using a Mean Squared Error (MSE) loss (a common practice). We show that contrastive losses are able to accurately estimate a ranking function from limited data even in regimes where MSE is ineffective and validate the practical utility of this insight by demonstrating that contrastive loss functions result in consistently improved performance on benchmark tasks.

Motivation & Objective

• To address the challenge of learning fitness functions from limited experimental data in high-dimensional biological sequence spaces.
• To investigate why mean squared error (MSE) loss fails in regimes affected by global epistasis, where fitness functions become dense in epistatic representation.
• To propose and validate contrastive losses—especially Bradley-Terry—as a more data-efficient alternative for learning latent ranking functions.
• To establish a theoretical link between global epistasis and the incompatibility of sparse epistatic representations with observed fitness data.
• To demonstrate practical superiority of contrastive losses over MSE in real-world protein fitness prediction benchmarks.

Proposed method

• The paper formulates global epistasis as a monotonic nonlinearity transforming a sparse latent fitness function into observed fitness, leading to non-sparse representations in epistatic bases.
• It introduces a fitness-epistasis uncertainty principle, showing that concentrated fitness values in the fitness domain imply dense representations in epistatic (Graph Fourier) bases, making sparse modeling infeasible.
• The core method is minimizing supervised contrastive loss (specifically Bradley-Terry) to recover the latent ranking function without assuming a parametric form for the nonlinearity or latent function.
• The approach is validated via simulations under varying data sparsity and compared to MSE training on the FLIP benchmark with multiple protein fitness datasets.
• The method is applied to convolutional neural networks (CNNs) trained on sequence data, with ablation on inclusion of non-mutated sequence positions.
• Performance is evaluated using Spearman correlation and top-10% recall on test sets to assess both ranking and high-precision prediction.

Experimental results

Research questions

• RQ1 Can contrastive losses such as Bradley-Terry outperform MSE in learning fitness functions when global epistasis distorts the data?
• RQ2 Why does MSE fail to learn fitness functions affected by global epistasis, particularly under data scarcity?
• RQ3 To what extent can the latent ranking function be recovered from observed fitness data using contrastive learning, even without assumptions on the nonlinearity or latent function?
• RQ4 How does the fitness-epistasis uncertainty principle explain the incompatibility between sparse epistatic models and globally epistatic fitness functions?
• RQ5 Does contrastive training improve high-precision prediction (e.g., top 10% fitness sequences) compared to MSE in real-world protein engineering benchmarks?

Key findings

• Contrastive losses, especially Bradley-Terry, consistently outperform MSE across all FLIP benchmark tasks in terms of Spearman correlation, with no case showing worse performance.
• The Bradley-Terry loss achieves higher top-10% recall than MSE, indicating superior performance in identifying the highest-fitness sequences.
• Simulations confirm that global epistasis produces fitness functions that are dense in epistatic bases, making them hard to learn with MSE under data scarcity.
• The fitness-epistasis uncertainty principle explains why such functions cannot be represented sparsely in epistatic terms, justifying the need for alternative learning objectives.
• The method remains robust to noise, as shown in a pathological noise scenario in Appendix F, suggesting practical viability in real experimental settings.
• Excluding non-mutated sequence positions often improves performance and reduces computational cost, with only minor gains in specific splits.