[Paper Review] A multi-stage machine learning model on diagnosis of esophageal manometry

Wenjun Kou, Dustin A. Carlson+5 more|arXiv (Cornell University)|Jun 25, 2021

Dysphagia Assessment and Management18 references27 citations

TL;DR

This study proposes a multi-stage machine learning framework that automatically diagnoses esophageal motility disorders from high-resolution manometry (HRM) data using raw pressure waveforms. It employs convolutional neural networks (CNNs) to classify swallow types, pressurization patterns, and integrated relaxation pressure (IRP), followed by model-agnostic blending of xgboost, artificial neural network (ANN), and rule-based models to predict study-level Chicago Classification diagnoses with 81% top-1 accuracy and 92% top-2 accuracy.

ABSTRACT

High-resolution manometry (HRM) is the primary procedure used to diagnose esophageal motility disorders. Its interpretation and classification includes an initial evaluation of swallow-level outcomes and then derivation of a study-level diagnosis based on Chicago Classification (CC), using a tree-like algorithm. This diagnostic approach on motility disordered using HRM was mirrored using a multi-stage modeling framework developed using a combination of various machine learning approaches. Specifically, the framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. In the swallow-level stage, three models based on convolutional neural networks (CNNs) were developed to predict swallow type, swallow pressurization, and integrated relaxation pressure (IRP). At the study-level stage, model selection from families of the expert-knowledge-based rule models, xgboost models and artificial neural network(ANN) models were conducted, with the latter two model designed and augmented with motivation from the export knowledge. A simple model-agnostic strategy of model balancing motivated by Bayesian principles was utilized, which gave rise to model averaging weighted by precision scores. The averaged (blended) models and individual models were compared and evaluated, of which the best performance on test dataset is 0.81 in top-1 prediction, 0.92 in top-2 predictions. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data. Moreover, the proposed modeling framework could be easily extended to multi-modal tasks, such as diagnosis of esophageal patients based on clinical data from both HRM and functional luminal imaging probe panometry (FLIP).

Motivation & Objective

• To develop an automated, AI-driven diagnostic system for esophageal motility disorders using high-resolution manometry (HRM) data.
• To reduce inter-rater variability and interpretation subjectivity inherent in manual HRM interpretation based on the Chicago Classification (CC).
• To integrate deep learning at the swallow-level and ensemble machine learning at the study-level for improved diagnostic accuracy.
• To propose a model-agnostic, precision-weighted blending strategy for combining diverse model families (xgboost, ANN, rule-based) to enhance robustness and performance.

Proposed method

• A two-stage modeling framework: (1) swallow-level analysis using three CNNs to predict swallow type (accuracy: 0.88), swallow pressurization (accuracy: 0.93), and IRP (regression error: 4.49 mmHg).
• Study-level diagnosis prediction using three model families: expert-knowledge-based rule models, xgboost, and artificial neural networks (ANNs), trained on derived swallow-level features.
• A model-agnostic blending strategy based on Bayesian principles, where models are averaged using precision scores as weights to balance performance across classes.
• Hyperparameter tuning and model selection performed via validation set accuracy, with feature augmentation (14 features) and class-weighting to address data imbalance.
• Blended models were evaluated using top-1 and top-2 prediction accuracy, with precision scores used as weighting factors for model averaging.
• The framework is extensible to multi-modal data integration, such as combining HRM with functional luminal imaging probe (FLIP) data.

Experimental results

Research questions

• RQ1 Can a multi-stage machine learning model accurately predict esophageal motility diagnoses from raw HRM data without manual feature extraction?
• RQ2 How does model blending based on precision scores improve performance over individual models in diagnosing complex, multi-class HRM outcomes?
• RQ3 To what extent can deep learning models at the swallow-level enhance downstream study-level diagnosis accuracy compared to traditional rule-based or statistical models?
• RQ4 Can a hybrid approach combining deep learning, tree-based models (xgboost), and neural networks (ANN) outperform any single model type in HRM diagnosis?
• RQ5 Is the proposed model-agnostic blending strategy effective in mitigating class imbalance and improving generalization across diverse motility diagnoses?

Key findings

• The proposed multi-stage model achieved 81% top-1 accuracy and 92% top-2 accuracy in predicting Chicago Classification diagnoses on the test set.
• The swallow-level CNNs achieved 88% accuracy for swallow type classification and 93% for pressurization pattern recognition, with IRP regression error of 4.49 mmHg.
• The optimal xgboost model (xgb-5) used max depth 5, rule-based base learner, and 14 augmented features, achieving 79% validation accuracy.
• The optimal ANN model (ANN-1) had 6 layers, width factor 1, and 14 input features, achieving 77% validation accuracy.
• Model blending using precision-weighted averaging outperformed individual models, demonstrating the effectiveness of the proposed strategy.
• The study highlights limitations in current CC classification, particularly for EGJ outflow obstruction (EGJOO), due to overlapping hemodynamic patterns and dataset imbalance.