[Paper Review] Transfer Learning for Endoscopic Image Classification

Shoji Sonoyama, Toru Tamaki+7 more|arXiv (Cornell University)|Aug 24, 2016

Domain Adaptation and Few-Shot Learning9 references4 citations

TL;DR

This paper proposes MMDTL2, a novel transfer learning method for endoscopic image classification that extends Max–Margin Domain Transfer (MMDT) by incorporating L₂ distance constraints to improve domain adaptation between images from different endoscopes. The method achieves better classification performance than MMDT on real-world endoscopic datasets while reducing computation cost via a derived dual formulation, making it more practical for clinical deployment with limited labeled data and device variability.

ABSTRACT

In this paper we propose a method for transfer learning of endoscopic images. For transferring between features obtained from images taken by different (old and new) endoscopes, we extend the Max-Margin Domain Transfer (MMDT) proposed by Hoffman et al. in order to use L2 distance constraints as regularization, called Max-Margin Domain Transfer with L2 Distance Constraints (MMDTL2). Furthermore, we develop the dual formulation of the optimization problem in order to reduce the computation cost. Experimental results demonstrate that the proposed MMDTL2 outperforms MMDT for real data sets taken by different endoscopes.

Motivation & Objective

• Address the challenge of domain shift in endoscopic image classification due to differences in image appearance across endoscope devices (e.g., resolution, brightness, sharpness).
• Overcome the limitation of prior transfer learning methods that require image-level correspondences between source and target domains.
• Develop a method that enables effective feature transfer from older to newer endoscope systems without requiring paired training images.
• Reduce computational cost for large-scale feature transfer by deriving a dual formulation of the optimization problem.
• Improve classification accuracy on real-world endoscopic datasets where source and target distributions differ due to device changes.

Proposed method

• Extend the Max–Margin Domain Transfer (MMDT) framework by introducing L₂ distance constraints as regularization to stabilize the linear transformation between source and target feature spaces.
• Formulate the optimization problem as a constrained quadratic program that jointly learns the domain adaptation transformation matrix W and classifier parameters θk for each class.
• Derive a dual formulation of the primal optimization problem to reduce computational complexity, especially for high-dimensional feature spaces.
• Use slack variables and margin constraints to handle classification errors and ensure robustness in the presence of distribution shifts.
• Apply the learned transformation W to map target domain features to the source domain space, enabling the use of a pre-trained classifier on new endoscope data.
• Solve the dual problem by computing dual variables ai, which depend on the target features and their labels, significantly reducing the number of optimization variables.

Experimental results

Research questions

• RQ1 Can L₂ distance constraints improve the robustness and performance of domain adaptation in endoscopic image classification when transferring between different endoscope systems?
• RQ2 Does the proposed dual formulation of the optimization problem significantly reduce computation time compared to the primal formulation?
• RQ3 How does MMDTL2 compare to MMDT and baseline methods in terms of classification accuracy on real-world endoscopic image datasets with domain shift?
• RQ4 To what extent can MMDTL2 generalize to higher-dimensional feature spaces (e.g., M_S = M_T ≥ 256) without requiring paired images?
• RQ5 Can MMDTL2 achieve performance comparable to a fully supervised baseline when only a small target dataset is available?

Key findings

• The proposed MMDTL2 method outperforms MMDT across all tested feature dimensions, with a significant improvement in classification accuracy due to the added L₂ regularization.
• The dual formulation of MMDTL2 reduces total computation time by approximately 12 times compared to the primal formulation, with optimization time dropping from 6396.84 seconds to 0.060 seconds.
• MMDTL2 achieves performance comparable to the baseline (10-fold cross-validation on source data alone) when the feature dimension is sufficiently high (M_S = M_T ≥ 256).
• The method significantly outperforms the 'Source only' and 'Not transfer' settings, demonstrating its effectiveness in adapting to new endoscope devices with minimal labeled data.
• MMDTL2 shows robustness to domain shift caused by differences in image quality and resolution between old and new endoscopes, as validated on real-world NBI endoscopic images.
• The method enables effective transfer learning without requiring image-level correspondences between source and target domains, making it practical for clinical deployment.