[Paper Review] Incorporating Prior Knowledge in Deep Learning Models via Pathway Activity Autoencoders

Pedro Avelar, Min Wu+1 more|arXiv (Cornell University)|Jun 9, 2023

RNA and protein synthesis mechanismsBiochemistry, Genetics and Molecular Biology3 citations

TL;DR

This paper proposes Pathway Activity Autoencoders (PAAE) and their generative variant PAVAE, which integrate biological pathway knowledge into deep autoencoders for RNA-seq data to improve interpretability and predictive performance in cancer prognosis. By constraining the latent space using known pathways, the models achieve superior reconstruction and classification accuracy even with fewer input features, outperforming standard autoencoders and VAEs, while enabling direct biological interpretation of learned representations.

ABSTRACT

Motivation: Despite advances in the computational analysis of high-throughput molecular profiling assays (e.g. transcriptomics), a dichotomy exists between methods that are simple and interpretable, and ones that are complex but with lower degree of interpretability. Furthermore, very few methods deal with trying to translate interpretability in biologically relevant terms, such as known pathway cascades. Biological pathways reflecting signalling events or metabolic conversions are Small improvements or modifications of existing algorithms will generally not be suitable, unless novel biological results have been predicted and verified. Determining which pathways are implicated in disease and incorporating such pathway data as prior knowledge may enhance predictive modelling and personalised strategies for diagnosis, treatment and prevention of disease. Results: We propose a novel prior-knowledge-based deep auto-encoding framework, PAAE, together with its accompanying generative variant, PAVAE, for RNA-seq data in cancer. Through comprehensive comparisons among various learning models, we show that, despite having access to a smaller set of features, our PAAE and PAVAE models achieve better out-of-set reconstruction results compared to common methodologies. Furthermore, we compare our model with equivalent baselines on a classification task and show that they achieve better results than models which have access to the full input gene set. Another result is that using vanilla variational frameworks might negatively impact both reconstruction outputs as well as classification performance. Finally, our work directly contributes by providing comprehensive interpretability analyses on our models on top of improving prognostication for translational medicine.

Motivation & Objective

• To address the lack of interpretability in deep learning models for high-throughput omics data by incorporating biologically meaningful prior knowledge.
• To improve patient stratification and survival prediction in cancer using pathway-informed autoencoders that learn latent representations directly interpretable in biological terms.
• To evaluate whether using pathway information as a structural inductive bias enhances both reconstruction and classification performance compared to standard autoencoders and VAEs.
• To demonstrate that pathway-activity-based representations are consistent across datasets and informative for downstream tasks like classification and survival analysis.
• To provide a framework that enables clinicians and researchers to interpret model outputs through known biological pathways, supporting translational medicine.

Proposed method

• Propose a non-generative autoencoder framework, PAAE, that uses known biological pathways as a structural inductive bias to constrain the latent space.
• Define pathway activity vectors as the activation levels of predefined pathways across samples, using gene set enrichment scores to compute them.
• Train the autoencoder to reconstruct input gene expression data using only the pathway activity vectors as latent representations, thereby enforcing biological interpretability.
• Extend PAAE to a generative variant, PAVAE, using variational inference to enable sampling from the learned latent space and improve robustness.
• Apply a reparameterization trick in PAVAE to allow backpropagation through the pathway activity sampling process, enabling end-to-end training.
• Use UMAP and cosine distance visualization to analyze the topology of the learned pathway activity space and assess class separability.

Experimental results

Research questions

• RQ1 Can integrating known biological pathways as inductive bias improve the reconstruction performance of autoencoders on RNA-seq data?
• RQ2 Does using pathway-activity-based latent representations lead to better classification performance than models using the full gene set?
• RQ3 How does the performance of PAAE and PAVAE compare to standard autoencoders and VAEs in terms of reconstruction and classification accuracy?
• RQ4 To what extent are the learned pathway activities consistent across independent datasets like TCGA and METABRIC?
• RQ5 Can the model's latent space be meaningfully interpreted in terms of known biological pathways relevant to cancer subtypes?

Key findings

• PAAE and PAVAE achieve better out-of-sample reconstruction performance than standard autoencoders and VAEs, despite using only pathway-activity features instead of the full gene set.
• The models outperform baselines in classification tasks, even when the baselines have access to all input genes, demonstrating the value of prior knowledge in representation learning.
• Vanilla VAEs performed worse than PAAE and PAVAE in both reconstruction and classification, suggesting that standard variational frameworks may not be optimal for pathway-informed learning.
• The 32 pathways with highest mutual information with disease subtypes in the training set were also among the most informative in the test set, indicating consistent and robust feature learning across datasets.
• Visualizations of the pathway activity space using UMAP and feature maps show clear clustering by clinical phenotype (e.g., BRCA subtypes), confirming the biological relevance of the learned representations.
• Interpretability analyses reveal that individual pathways such as P53 signaling, mismatch repair, and valine-leucine-isoleucine degradation are highly discriminative and biologically plausible in the context of cancer classification.