[Paper Review] DeepDiff: Deep-learning for predicting Differential gene expression from histone modifications

Arshdeep Sekhon, Ritambhara Singh+1 more|arXiv (Cornell University)|Jul 10, 2018

Genomics and Chromatin Dynamics9 references3 citations

TL;DR

DeepDiff introduces an attention-based deep learning model that predicts differential gene expression from histone modification signals across cell types. By leveraging hierarchical LSTMs and dual-level attention, it captures spatial dependencies and combinatorial effects among histone marks, significantly outperforming state-of-the-art methods in differential expression prediction on Roadmap Epigenomics data.

ABSTRACT

Computational methods that predict differential gene expression from histone modification signals are highly desirable for understanding how histone modifications control the functional heterogeneity of cells through influencing differential gene regulation. Recent studies either failed to capture combinatorial effects on differential prediction or primarily only focused on cell type-specific analysis. In this paper, we develop a novel attention-based deep learning architecture, DeepDiff, that provides a unified and end-to-end solution to model and to interpret how dependencies among histone modifications control the differential patterns of gene regulation. DeepDiff uses a hierarchy of multiple Long short-term memory (LSTM) modules to encode the spatial structure of input signals and to model how various histone modifications cooperate automatically. We introduce and train two levels of attention jointly with the target prediction, enabling DeepDiff to attend differentially to relevant modifications and to locate important genome positions for each modification. Additionally, DeepDiff introduces a novel deep-learning based multi-task formulation to use the cell-type-specific gene expression predictions as auxiliary tasks, encouraging richer feature embeddings in our primary task of differential expression prediction. Using data from Roadmap Epigenomics Project (REMC) for ten different pairs of cell types, we show that DeepDiff significantly outperforms the state-of-the-art baselines for differential gene expression prediction. The learned attention weights are validated by observations from previous studies about how epigenetic mechanisms connect to differential gene expression. Codes and results are available at \url{deepchrome.org}

Motivation & Objective

• To develop a unified, end-to-end deep learning framework for predicting differential gene expression from histone modification signals across cell types.
• To model the combinatorial interactions among multiple histone modifications and their spatial dependencies along the genome.
• To improve feature representation through a multi-task learning formulation using cell-type-specific gene expression as an auxiliary task.
• To enable interpretability by learning attention weights that highlight relevant histone marks and critical genomic positions.
• To validate the model’s biological relevance by aligning attention patterns with known epigenetic mechanisms from prior studies.

Proposed method

• Employs a hierarchy of Long Short-Term Memory (LSTM) networks to encode the spatial structure of histone modification signals across 10,000 bp regions centered on transcription start sites (TSS).
• Introduces two levels of attention: one to attend to relevant histone modifications and another to locate important genomic positions within each mark’s signal.
• Uses a multi-task learning framework where cell-type-specific gene expression prediction is used as an auxiliary task to enrich feature embeddings for the primary differential expression prediction task.
• Applies a Siamese network architecture with a contrastive loss to align representations across cell types, enhancing generalization.
• Trains the model end-to-end with joint optimization of the main regression task (differential expression) and auxiliary tasks (regression or classification of cell-type-specific expression).
• Uses Pearson Correlation Coefficient (PCC) as the primary evaluation metric to compare performance against baseline models.

Experimental results

Research questions

• RQ1 Can a deep learning model effectively predict differential gene expression from genome-wide histone modification signals while capturing combinatorial and spatial dependencies?
• RQ2 How do attention mechanisms in DeepDiff highlight biologically relevant histone marks and genomic regions for differential regulation?
• RQ3 To what extent does incorporating cell-type-specific gene expression as an auxiliary task improve the performance and robustness of differential expression prediction?
• RQ4 Do the attention weights learned by DeepDiff align with known epigenetic mechanisms and prior biological observations about histone mark functions?
• RQ5 How does DeepDiff compare to existing state-of-the-art methods in terms of predictive accuracy and interpretability?

Key findings

• DeepDiff significantly outperforms state-of-the-art baselines in differential gene expression prediction across ten cell-type pairs from the Roadmap Epigenomics Project.
• The model achieves a mean PCC of 179.17 and median PCC of 192.33 when using classification-based cell-type-specific auxiliary tasks, outperforming the two-layer SVR baseline.
• Attention weights learned by DeepDiff align with established biological knowledge: H3K4me3 is prioritized near promoters, H3K4me1 near enhancers, and H3K27me3 near repressed regions.
• The use of multi-task learning with cell-type-specific expression as an auxiliary task improves feature representation, leading to higher predictive accuracy.
• The hierarchical LSTM architecture effectively models long-range dependencies in histone modification signals across 10 kb genomic regions centered on TSS.
• The model’s attention mechanism successfully identifies key regulatory regions and histone marks, providing interpretable insights into differential gene regulation mechanisms.