[Paper Review] Enhanced Prediction of Ventilator-Associated Pneumonia in Patients with Traumatic Brain Injury Using Advanced Machine Learning Techniques

Negin Ashrafi, Armin Abdollahi+1 more|arXiv (Cornell University)|Aug 2, 2024

COVID-19 diagnosis using AIMedicine3 citations

TL;DR

This study enhances prediction of ventilator-associated pneumonia (VAP) in traumatic brain injury (TBI) patients using advanced machine learning on the MIMIC-III database. By applying feature selection, SMOTE for class imbalance, and rigorous hyperparameter tuning, XGBoost achieved an AUC of 0.940 and accuracy of 0.875—23.4% and 23.5% higher than prior literature, respectively.

ABSTRACT

Background: Ventilator-associated pneumonia (VAP) in traumatic brain injury (TBI) patients poses a significant mortality risk and imposes a considerable financial burden on patients and healthcare systems. Timely detection and prognostication of VAP in TBI patients are crucial to improve patient outcomes and alleviate the strain on healthcare resources. Methods: We implemented six machine learning models using the MIMIC-III database. Our methodology included preprocessing steps, such as feature selection with CatBoost and expert opinion, addressing class imbalance with the Synthetic Minority Oversampling Technique (SMOTE), and rigorous model tuning through 5-fold cross-validation to optimize hyperparameters. Key models evaluated included SVM, Logistic Regression, Random Forest, XGBoost, ANN, and AdaBoost. Additionally, we conducted SHAP analysis to determine feature importance and performed an ablation study to assess feature impacts on model performance. Results: XGBoost outperformed the baseline models and the best existing literature. We used metrics, including AUC, Accuracy, Specificity, Sensitivity, F1 Score, PPV, and NPV. XGBoost demonstrated the highest performance with an AUC of 0.940 and an Accuracy of 0.875, which are 23.4% and 23.5% higher than the best results in the existing literature, with an AUC of 0.706 and an Accuracy of 0.640, respectively. This enhanced performance underscores the models' effectiveness in clinical settings. Conclusions: This study enhances the predictive modeling of VAP in TBI patients, improving early detection and intervention potential. Refined feature selection and advanced ensemble techniques significantly boosted model accuracy and reliability, offering promising directions for future clinical applications and medical diagnostics research.

Motivation & Objective

• Improve early prediction of ventilator-associated pneumonia (VAP) in traumatic brain injury (TBI) patients to reduce mortality and healthcare burden.
• Address the challenge of class imbalance and limited predictive performance in existing VAP prediction models for TBI patients.
• Enhance model interpretability and clinical utility through feature importance analysis and ablation studies.
• Develop a robust, generalizable machine learning pipeline for VAP prediction using real-world ICU data.
• Demonstrate the superiority of ensemble methods, particularly XGBoost, over traditional models and prior literature in VAP prediction accuracy.

Proposed method

• Utilized the MIMIC-III database containing 40,000+ patients for data extraction and cohort selection based on ICD-9 codes for TBI.
• Applied exclusion criteria including missing Glasgow Coma Scale (GCS) or vital signs at admission and mechanical ventilation duration <48 hours.
• Performed feature selection using CatBoost and expert clinical input to reduce 52 initial features to 15 key predictors.
• Addressed class imbalance using Synthetic Minority Oversampling Technique (SMOTE) to improve model generalization.
• Trained and tuned six models—SVM, Logistic Regression, Random Forest, XGBoost, ANN, and AdaBoost—via 5-fold cross-validation.
• Conducted SHAP analysis for model interpretability and ablation studies to evaluate feature impact on performance.

Figure 1: Flow diagram of the patient selection process

Experimental results

Research questions

• RQ1 Can advanced machine learning models significantly improve the prediction of VAP in TBI patients compared to existing literature?
• RQ2 Which machine learning model architecture yields the highest performance in terms of AUC, accuracy, sensitivity, and specificity for VAP prediction in TBI?
• RQ3 How does refined feature selection and SMOTE-based resampling impact model performance and generalizability?
• RQ4 Which clinical features are most predictive of VAP development in TBI patients, and how do they compare to known risk factors?
• RQ5 To what extent does model interpretability via SHAP analysis enhance clinical trust and applicability of predictive models?

Key findings

• XGBoost achieved the highest performance with an AUC of 0.940 and accuracy of 0.875, representing a 23.4% improvement in AUC and 23.5% in accuracy over the best prior study (AUC 0.706, accuracy 0.640).
• All baseline models outperformed the previous best study across all metrics, including specificity, sensitivity, F1 score, PPV, and NPV.
• Feature selection reduced the number of input features from 52 to 15 without sacrificing model performance, enhancing interpretability and reducing overfitting.
• SHAP analysis revealed that features such as GCS, lactate levels, and respiratory rate were among the most influential predictors of VAP in TBI patients.
• The ablation study confirmed that removing key features significantly degraded model performance, validating the importance of the selected feature set.
• The model demonstrated strong robustness and generalizability, with consistent high performance across multiple evaluation metrics in cross-validation.

Figure 2: Top 15 features based on CatBoost feature importance scores, highlighting the most impactful ones.