[Paper Review] Prediction of breast cancer recurrence using Classification Restricted Boltzmann Machine with Dropping

Jakub M. Tomczak|arXiv (Cornell University)|Aug 28, 2013

AI in cancer detection9 references11 citations

TL;DR

This paper proposes a novel probabilistic framework, 'Dropping,' for training Classification Restricted Boltzmann Machines (ClassRBM) to predict breast cancer recurrence and identify relevant clinical features. It introduces DropPart—a generalization of DropConnect using Beta-distributed connection masks—showcasing improved performance over DropConnect and competitive results with DropOut on a real dataset of 949 cases from the Institute of Oncology Ljubljana.

ABSTRACT

In this paper, we apply Classification Restricted Boltzmann Machine (ClassRBM) to the problem of predicting breast cancer recurrence. According to the Polish National Cancer Registry, in 2010 only, the breast cancer caused almost 25% of all diagnosed cases of cancer in Poland. We propose how to use ClassRBM for predicting breast cancer return and discovering relevant inputs (symptoms) in illness reappearance. Next, we outline a general probabilistic framework for learning Boltzmann machines with masks, which we refer to as Dropping. The fashion of generating masks leads to different learning methods, i.e., DropOut, DropConnect. We propose a new method called DropPart which is a generalization of DropConnect. In DropPart the Beta distribution instead of Bernoulli distribution in DropConnect is used. At the end, we carry out an experiment using real-life dataset consisting of 949 cases, provided by the Institute of Oncology Ljubljana.

Motivation & Objective

• To develop a new learning framework, 'Dropping,' for training ClassRBM with probabilistic connection masking.
• To apply ClassRBM to predict breast cancer recurrence using real clinical data.
• To discover clinically relevant inputs (symptoms) associated with disease recurrence through conditional probability analysis.
• To propose and evaluate DropPart, a generalization of DropConnect using Beta-distributed masks.
• To compare the performance of DropOut, DropConnect, and DropPart in terms of classification accuracy and feature relevance.

Proposed method

• Proposes a general probabilistic framework called 'Dropping' for learning Boltzmann machines with connection masks.
• Introduces DropPart, where connection masks are drawn from a Beta distribution (with parameters a, b ≤ 1), generalizing the Bernoulli-based DropConnect.
• Uses ClassRBM to model the joint distribution of visible inputs (clinical features), hidden units, and output classes (recurrence or no recurrence).
• Employs the conditional distribution p(y|x) for prediction, derived from the energy-based model and normalized via softmax over output classes.
• Applies p(xi=1|x\i=0,y) to compute input relevance, identifying features most predictive of recurrence under absence of other inputs.
• Implements stochastic gradient descent with mini-batch updates and uses the sum over masks for approximate inference in training.

Experimental results

Research questions

• RQ1 Can the proposed 'Dropping' framework improve generalization and predictive performance in ClassRBM for breast cancer recurrence prediction?
• RQ2 How does DropPart, using Beta-distributed masks, compare to DropConnect (Bernoulli masks) and DropOut in terms of classification accuracy?
• RQ3 Which clinical features are most relevant for predicting breast cancer recurrence, as identified by the model’s conditional input probabilities?
• RQ4 Does the ClassRBM with Dropping methods outperform traditional models like SVM, Random Forest, and human oncologists in prediction accuracy?
• RQ5 Can the model effectively identify biologically meaningful features, such as tumor type or hormone receptor levels, as key predictors of recurrence?

Key findings

• DropOut achieved the highest classification accuracy (0.750) among all methods tested, outperforming DropConnect and DropPart.
• DropPart with Beta(0.1, 0.1) showed performance comparable to DropConnect, but with slightly better results in some configurations, particularly at lower learning rates.
• The ClassRBM with DropPart successfully identified clinically relevant features: histological tumor type (ductal, lobular, other) and low progesterone receptor levels (<10 fmol/mg) as highly relevant for recurrence prediction.
• Tumor size >50 mm (input No. 5) was found to be non-predictive, which may challenge conventional assumptions and warrants clinical validation.
• The model's predictive performance (0.735–0.750 AUC) was competitive with or superior to traditional models like SVM (0.710), Naive Bayes (0.730), and Random Forest (0.650), and even outperformed two human oncologists (0.630 and 0.630) on a subset of 100 cases.
• The method demonstrated strong explanatory capabilities by quantitatively ranking input features based on their conditional relevance, offering interpretable insights into recurrence predictors.