[论文解读] LesionAid: Vision Transformers-based Skin Lesion Generation and Classification

Ghanta Sai Krishna, Kundrapu Supriya+2 more|arXiv (Cornell University)|Feb 2, 2023

Cutaneous Melanoma Detection and Management被引用 8

一句话总结

一个多阶段框架，使用 Vision Transformers 和 ViTGANs 生成合成皮肤病变图像以实现类别平衡、扩增数据，并进行端到端的可解释 AI 实时分类，适用于边缘计算。

ABSTRACT

Skin cancer is one of the most prevalent forms of human cancer. It is recognized mainly visually, beginning with clinical screening and continuing with the dermoscopic examination, histological assessment, and specimen collection. Deep convolutional neural networks (CNNs) perform highly segregated and potentially universal tasks against a classified finegrained object. This research proposes a novel multi-class prediction framework that classifies skin lesions based on ViT and ViTGAN. Vision transformers-based GANs (Generative Adversarial Networks) are utilized to tackle the class imbalance. The framework consists of four main phases: ViTGANs, Image processing, and explainable AI. Phase 1 consists of generating synthetic images to balance all the classes in the dataset. Phase 2 consists of applying different data augmentation techniques and morphological operations to increase the size of the data. Phases 3 & 4 involve developing a ViT model for edge computing systems that can identify patterns and categorize skin lesions from the user's skin visible in the image. In phase 3, after classifying the lesions into the desired class with ViT, we will use explainable AI (XAI) that leads to more explainable results (using activation maps, etc.) while ensuring high predictive accuracy. Real-time images of skin diseases can capture by a doctor or a patient using the camera of a mobile application to perform an early examination and determine the cause of the skin lesion. The whole framework is compared with the existing frameworks for skin lesion detection.

研究动机与目标

• 使用 Vision Transformers 和 GAN 来解决皮肤病变数据集中类别不平衡问题。
• 开发一个多阶段管线，平衡数据、扩增图像，并在边缘设备上实现皮肤病变的实时分类。
• 结合可解释的 AI（XAI），提供可解释的结果与准确预测并存。
• 实现从合成数据生成到移动友好平台部署的端到端工作流。

提出的方法

• Phase 1: Generate synthetic images with ViTGANs to balance all classes.
• Phase 2: Apply data augmentation techniques and morphological operations to expand the dataset.
• Phase 3: Develop a ViT-based model for edge computing to classify lesions from user-provided images.
• Phase 4: Integrate explainable AI (XAI) methods, such as activation maps, to enhance interpretability without sacrificing accuracy.

实验结果

研究问题

• RQ1 Can ViTGAN-generated images effectively balance imbalanced skin lesion datasets?
• RQ2 Does a ViT-based classifier achieve high accuracy on edge devices for skin lesion prediction?
• RQ3 Can XAI methods provide meaningful explanations for ViT-based skin lesion classifications?
• RQ4 How does the integrated pipeline compare to existing frameworks in skin lesion detection and classification?

主要发现

• The framework combines ViT and ViTGAN to address class imbalance in skin lesion datasets.
• Data augmentation and morphological processing expand the training data beyond the original samples.
• An edge-optimized ViT classifier is proposed for real-time skin lesion identification from user-provided images.
• Explainable AI components offer interpretable insights via activation maps alongside predictive results.
• The whole framework is benchmarked against existing skin lesion detection frameworks.