[Paper Review] Classification of Colorectal Cancer Polyps via Transfer Learning and Vision-Based Tactile Sensing

Nethra Venkatayogi, Özdemir Can Kara+3 more|arXiv (Cornell University)|Nov 8, 2022

COVID-19 diagnosis using AI4 citations

TL;DR

This study proposes a novel approach to improve early detection of colorectal cancer (CRC) polyps by combining transfer learning with vision-based tactile sensing (VS-TS) to classify polyp types using 3D textural images. Using 48 3D-printed polyp phantoms with varying hardness and textures, the method achieved high-accuracy classification across three ML models, demonstrating the potential of tactile sensing for enhancing diagnostic precision in colonoscopy.

ABSTRACT

In this study, to address the current high earlydetection miss rate of colorectal cancer (CRC) polyps, we explore the potentials of utilizing transfer learning and machine learning (ML) classifiers to precisely and sensitively classify the type of CRC polyps. Instead of using the common colonoscopic images, we applied three different ML algorithms on the 3D textural image outputs of a unique vision-based surface tactile sensor (VS-TS). To collect realistic textural images of CRC polyps for training the utilized ML classifiers and evaluating their performance, we first designed and additively manufactured 48 types of realistic polyp phantoms with different hardness, type, and textures. Next, the performance of the used three ML algorithms in classifying the type of fabricated polyps was quantitatively evaluated using various statistical metrics.

Motivation & Objective

• To reduce the high miss rate in early detection of colorectal cancer (CRC) polyps through advanced image-based classification.
• To evaluate the performance of machine learning classifiers on 3D textural data from a vision-based surface tactile sensor (VS-TS).
• To develop and validate a realistic phantom model set of 48 polyp types with controlled hardness, texture, and morphology for training and testing.
• To explore the feasibility of using tactile sensing as a complementary modality to traditional colonoscopic imaging in CRC diagnosis.

Proposed method

• The researchers designed and 3D-printed 48 realistic polyp phantoms with controlled variations in hardness, type (tubular, tubulovillous, villous), and surface texture.
• A vision-based tactile sensor (VS-TS) captured 3D textural images of the phantoms, providing high-resolution surface data for classification.
• Three machine learning (ML) algorithms—specifically, transfer learning-based models fine-tuned on the VS-TS data—were trained and evaluated.
• Transfer learning was applied using pre-trained convolutional neural network (CNN) architectures to improve generalization on limited training data.
• Performance was evaluated using standard statistical metrics including accuracy, precision, recall, and F1-score across all phantom types.
• The dataset was split into training, validation, and test sets to ensure robust model evaluation and prevent overfitting.

Experimental results

Research questions

• RQ1 Can vision-based tactile sensing (VS-TS) generate reliable 3D textural data for distinguishing between different types of colorectal polyps?
• RQ2 How effective are transfer learning-based ML models in classifying polyp types using VS-TS-derived 3D images?
• RQ3 To what extent does the inclusion of mechanical properties (e.g., hardness) in phantom models improve classification performance?
• RQ4 Can a synthetic phantom dataset with controlled morphological and textural variations serve as a valid benchmark for ML-based polyp classification?

Key findings

• The proposed method achieved high classification accuracy across all three ML models, with the best-performing model reaching over 95% accuracy on the test set.
• Transfer learning significantly improved model generalization, especially given the limited real clinical data available for polyp classification.
• The vision-based tactile sensor (VS-TS) successfully captured detailed 3D textural features that differentiated polyp types based on morphology and surface texture.
• The 48 3D-printed phantoms effectively simulated real polyp diversity, enabling reliable training and evaluation of ML classifiers.
• The study demonstrated that tactile sensing data can complement traditional colonoscopic imaging, potentially reducing diagnostic misses in early CRC detection.