[Paper Review] Autoencoders as Weight Initialization of Deep Classification Networks for Cancer versus Cancer Studies

Mafalda Falcão Ferreira, Rui Camacho+1 more|arXiv (Cornell University)|Jan 15, 2020

AI in cancer detection9 references5 citations

TL;DR

This study evaluates denoising autoencoders (DAEs) as a pre-training method for deep neural networks in classifying gene expression data from thyroid, skin, and stomach cancers. By importing the complete pre-trained DAE into the classification network and fine-tuning all weights, the model achieved an F1 score of 98.04% ± 1.09 in detecting thyroid cancer, outperforming fixed-weight initialization and demonstrating the effectiveness of end-to-end fine-tuning in this biomedical classification task.

ABSTRACT

Cancer is still one of the most devastating diseases of our time. One way of automatically classifying tumor samples is by analyzing its derived molecular information (i.e., its genes expression signatures). In this work, we aim to distinguish three different types of cancer: thyroid, skin, and stomach. For that, we compare the performance of a Denoising Autoencoder (DAE) used as weight initialization of a deep neural network. Although we address a different domain problem in this work, we have adopted the same methodology of Ferreira et al.. In our experiments, we assess two different approaches when training the classification model: (a) fixing the weights, after pre-training the DAE, and (b) allowing fine-tuning of the entire classification network. Additionally, we apply two different strategies for embedding the DAE into the classification network: (1) by only importing the encoding layers, and (2) by inserting the complete autoencoder. Our best result was the combination of unsupervised feature learning through a DAE, followed by its full import into the classification network, and subsequent fine-tuning through supervised training, achieving an F1 score of 98.04% +/- 1.09 when identifying cancerous thyroid samples.

Motivation & Objective

• To evaluate the effectiveness of denoising autoencoders (DAEs) as a weight initialization strategy for deep learning in cancer classification.
• To compare two training approaches: fixing pre-trained DAE weights versus fine-tuning the entire network after initialization.
• To assess two embedding strategies: using only the encoding layers or the full autoencoder (encoding and decoding) in the classification network.
• To determine whether unsupervised pre-training with DAEs improves classification performance on gene expression data from thyroid, skin, and stomach cancers.
• To investigate whether the performance gain from DAE pre-training is consistent across different cancer types and training strategies.

Proposed method

• Pre-trained a denoising autoencoder (DAE) on gene expression data from The Cancer Genome Atlas (TCGA) to minimize reconstruction error using mean squared error (MSE) loss.
• Used the pre-trained DAE weights to initialize the top layers of a deep feedforward neural network for classification.
• Evaluated two strategies: (1) only importing the encoding layers of the DAE, and (2) importing the complete DAE (encoding and decoding layers).
• Trained the classification network using two approaches: (a) fixing the DAE weights after initialization, and (b) fine-tuning all weights during supervised training.
• Applied 5-fold stratified cross-validation to evaluate performance, using loss, accuracy, precision, recall, and F1 score as metrics.
• Selected the best-performing model per fold based on F1 score and reported average results with standard deviation.

Experimental results

Research questions

• RQ1 Does using a denoising autoencoder (DAE) as a pre-training method improve classification performance on gene expression data for cancer subtypes?
• RQ2 Is fine-tuning the entire network after DAE pre-training more effective than fixing the pre-trained weights?
• RQ3 Does importing the full DAE (encoding and decoding) yield better results than importing only the encoding layers?
• RQ4 How does the performance of DAE-based initialization compare across different cancer types (thyroid, skin, stomach)?
• RQ5 To what extent does unsupervised pre-training with DAEs capture biologically relevant features for cancer classification?

Key findings

• Fine-tuning the entire network after DAE pre-training led to a 10–20% improvement in F1 score compared to fixing the pre-trained weights, contrary to prior findings.
• The best performance was achieved by importing the complete pre-trained DAE and fine-tuning all weights, yielding an F1 score of 98.04% ± 1.09 for thyroid cancer detection.
• For skin cancer, the best F1 score was 97.81% ± 1.76 using the complete DAE with fine-tuning, while for stomach cancer it reached 95.63% ± 3.23.
• The model using only the encoding layers of the DAE achieved lower performance across all cancer types, with the best F1 score of 97.79% ± 1.13 for thyroid cancer.
• The results indicate that full DAE integration and fine-tuning are critical for optimal performance, suggesting that the decoding layers contribute to better feature representation.
• Despite strong performance, the study found no conclusive evidence that DAE pre-training consistently captures the most relevant biological features across all cancer types, as indicated by inconsistent validation curve behavior.