[論文レビュー] LesionAid: Vision Transformers-based Skin Lesion Generation and Classification

Ghanta Sai Krishna, Kundrapu Supriya+2 more|arXiv (Cornell University)|Feb 2, 2023

Cutaneous Melanoma Detection and Management被引用数 8

ひとこと要約

A multi-phase framework using Vision Transformers and ViTGANs to generate synthetic skin lesion images for class balancing, augment data, and perform real-time classification with explainable AI for edge computing.

ABSTRACT

Skin cancer is one of the most prevalent forms of human cancer. It is recognized mainly visually, beginning with clinical screening and continuing with the dermoscopic examination, histological assessment, and specimen collection. Deep convolutional neural networks (CNNs) perform highly segregated and potentially universal tasks against a classified finegrained object. This research proposes a novel multi-class prediction framework that classifies skin lesions based on ViT and ViTGAN. Vision transformers-based GANs (Generative Adversarial Networks) are utilized to tackle the class imbalance. The framework consists of four main phases: ViTGANs, Image processing, and explainable AI. Phase 1 consists of generating synthetic images to balance all the classes in the dataset. Phase 2 consists of applying different data augmentation techniques and morphological operations to increase the size of the data. Phases 3 & 4 involve developing a ViT model for edge computing systems that can identify patterns and categorize skin lesions from the user's skin visible in the image. In phase 3, after classifying the lesions into the desired class with ViT, we will use explainable AI (XAI) that leads to more explainable results (using activation maps, etc.) while ensuring high predictive accuracy. Real-time images of skin diseases can capture by a doctor or a patient using the camera of a mobile application to perform an early examination and determine the cause of the skin lesion. The whole framework is compared with the existing frameworks for skin lesion detection.

研究の動機と目的

• 皮膚病変データセットのクラス不均衡をビジョントランスフォーマーとGANを用いて解消する。
• データを balance し、画像を拡張し、エッジデバイス上でリアルタイムの病変分類を実現するマルチフェーズ・パイプラインを開発する。
• 説明可能なAI（XAI）を組み込み、正確な予測とともに解釈可能な結果を提供する。
• 合成データ生成からモバイルに優しいプラットフォームへの展開まで、エンドツーエンドのワークフローを可能にする。

提案手法

• Phase 1: Generate synthetic images with ViTGANs to balance all classes.
• Phase 2: Apply data augmentation techniques and morphological operations to expand the dataset.
• Phase 3: Develop a ViT-based model for edge computing to classify lesions from user-provided images.
• Phase 4: Integrate explainable AI (XAI) methods, such as activation maps, to enhance interpretability without sacrificing accuracy.

実験結果

リサーチクエスチョン

• RQ1 Can ViTGAN-generated images effectively balance imbalanced skin lesion datasets?
• RQ2 Does a ViT-based classifier achieve high accuracy on edge devices for skin lesion prediction?
• RQ3 Can XAI methods provide meaningful explanations for ViT-based skin lesion classifications?
• RQ4 How does the integrated pipeline compare to existing frameworks in skin lesion detection and classification?

主な発見

• The framework combines ViT and ViTGAN to address class imbalance in skin lesion datasets.
• Data augmentation and morphological processing expand the training data beyond the original samples.
• An edge-optimized ViT classifier is proposed for real-time skin lesion identification from user-provided images.
• Explainable AI components offer interpretable insights via activation maps alongside predictive results.
• The whole framework is benchmarked against existing skin lesion detection frameworks.