[Paper Review] LesionAid: Vision Transformers-based Skin Lesion Generation and Classification

Ghanta Sai Krishna, Kundrapu Supriya+2 more|arXiv (Cornell University)|Feb 2, 2023

Cutaneous Melanoma Detection and ManagementMedicine8 citations

TL;DR

A multi-phase framework using Vision Transformers and ViTGANs to generate synthetic skin lesion images for class balancing, augment data, and perform real-time classification with explainable AI for edge computing.

ABSTRACT

Skin cancer is one of the most prevalent forms of human cancer. It is recognized mainly visually, beginning with clinical screening and continuing with the dermoscopic examination, histological assessment, and specimen collection. Deep convolutional neural networks (CNNs) perform highly segregated and potentially universal tasks against a classified finegrained object. This research proposes a novel multi-class prediction framework that classifies skin lesions based on ViT and ViTGAN. Vision transformers-based GANs (Generative Adversarial Networks) are utilized to tackle the class imbalance. The framework consists of four main phases: ViTGANs, Image processing, and explainable AI. Phase 1 consists of generating synthetic images to balance all the classes in the dataset. Phase 2 consists of applying different data augmentation techniques and morphological operations to increase the size of the data. Phases 3 & 4 involve developing a ViT model for edge computing systems that can identify patterns and categorize skin lesions from the user's skin visible in the image. In phase 3, after classifying the lesions into the desired class with ViT, we will use explainable AI (XAI) that leads to more explainable results (using activation maps, etc.) while ensuring high predictive accuracy. Real-time images of skin diseases can capture by a doctor or a patient using the camera of a mobile application to perform an early examination and determine the cause of the skin lesion. The whole framework is compared with the existing frameworks for skin lesion detection.

Motivation & Objective

• Address class imbalance in skin lesion datasets using vision transformers and GANs.
• Develop a multi-phase pipeline that balances data, augments images, and enables real-time lesion classification on edge devices.
• Incorporate explainable AI (XAI) to provide interpretable results alongside accurate predictions.
• Enable end-to-end workflow from synthetic data generation to deployment on mobile-friendly platforms.

Proposed method

• Phase 1: Generate synthetic images with ViTGANs to balance all classes.
• Phase 2: Apply data augmentation techniques and morphological operations to expand the dataset.
• Phase 3: Develop a ViT-based model for edge computing to classify lesions from user-provided images.
• Phase 4: Integrate explainable AI (XAI) methods, such as activation maps, to enhance interpretability without sacrificing accuracy.

Experimental results

Research questions

• RQ1 Can ViTGAN-generated images effectively balance imbalanced skin lesion datasets?
• RQ2 Does a ViT-based classifier achieve high accuracy on edge devices for skin lesion prediction?
• RQ3 Can XAI methods provide meaningful explanations for ViT-based skin lesion classifications?
• RQ4 How does the integrated pipeline compare to existing frameworks in skin lesion detection and classification?

Key findings

• The framework combines ViT and ViTGAN to address class imbalance in skin lesion datasets.
• Data augmentation and morphological processing expand the training data beyond the original samples.
• An edge-optimized ViT classifier is proposed for real-time skin lesion identification from user-provided images.
• Explainable AI components offer interpretable insights via activation maps alongside predictive results.
• The whole framework is benchmarked against existing skin lesion detection frameworks.