[Paper Review] Artificial Intelligence-Based Detection, Classification and Prediction/Prognosis in PET Imaging: Towards Radiophenomics

Fereshteh Yousefirizi, Pierre Decasez+4 more|arXiv (Cornell University)|Oct 20, 2021

Radiomics and Machine Learning in Medical Imaging1 citations

TL;DR

This paper proposes an AI-driven framework for automated detection, classification, and prognosis in oncological PET and PET/CT imaging using radiomics and deep learning. It integrates handcrafted and deep radiomics with AI to extract subtle image patterns for noninvasive tumor characterization, while highlighting clinical translation challenges and complementary techniques like NLP and neuro-symbolic AI.

ABSTRACT

Artificial intelligence (AI) techniques have significant potential to enable effective, robust, and automated image phenotyping including identification of subtle patterns. AI-based detection searches the image space to find the regions of interest based on patterns and features. There is a spectrum of tumor histologies from benign to malignant that can be identified by AI-based classification approaches using image features. The extraction of minable information from images gives way to the field of radiomics and can be explored via explicit (handcrafted/engineered) and deep radiomics frameworks. Radiomics analysis has the potential to be utilized as a noninvasive technique for the accurate characterization of tumors to improve diagnosis and treatment monitoring. This work reviews AI-based techniques, with a special focus on oncological PET and PET/CT imaging, for different detection, classification, and prediction/prognosis tasks. We also discuss needed efforts to enable the translation of AI techniques to routine clinical workflows, and potential improvements and complementary techniques such as the use of natural language processing on electronic health records and neuro-symbolic AI techniques.

Motivation & Objective

• To develop AI-based methods for automated detection of regions of interest in PET/CT images using pattern recognition.
• To enable accurate tumor classification across a spectrum of histologies—from benign to malignant—using image-derived features.
• To advance noninvasive tumor characterization through radiomics, combining handcrafted and deep learning-based feature extraction.
• To support improved diagnosis and treatment monitoring by enabling prediction and prognosis using imaging phenotypes.
• To identify barriers and solutions for integrating AI into routine clinical workflows, including multimodal data fusion.

Proposed method

• Employ AI-based detection to scan PET/CT images and identify regions of interest based on learned image patterns and features.
• Apply AI-based classification models trained on image features to distinguish between benign and malignant tumor histologies.
• Utilize explicit (handcrafted) radiomics and deep radiomics frameworks to extract minable, discriminative image features.
• Integrate radiomics with AI to enable noninvasive characterization of tumor phenotypes from medical images.
• Combine AI with natural language processing (NLP) on electronic health records to enhance clinical context and model interpretability.
• Explore neuro-symbolic AI techniques to improve reasoning, explainability, and integration of clinical knowledge with imaging data.

Experimental results

Research questions

• RQ1 How can AI effectively detect subtle regions of interest in PET/CT images using pattern recognition?
• RQ2 To what extent can AI-based classification accurately differentiate tumor histologies using image features?
• RQ3 What is the role of handcrafted versus deep radiomics in extracting prognostic and diagnostic information from PET images?
• RQ4 How can AI models be translated into routine clinical workflows for oncological imaging?
• RQ5 In what ways can NLP and neuro-symbolic AI enhance the interpretability and clinical utility of AI in PET imaging?

Key findings

• AI-based detection successfully identifies regions of interest in PET/CT images by learning complex image patterns.
• AI-based classification models demonstrate potential for distinguishing tumor histologies across a benign-to-malignant spectrum.
• Radiomics, especially when combined with deep learning, enables robust noninvasive characterization of tumors.
• The integration of AI with electronic health records via NLP enhances contextual understanding and model performance.
• Neuro-symbolic AI techniques offer a pathway to improve interpretability and clinical reasoning in AI-driven imaging.
• Clinical translation of AI in PET imaging requires addressing standardization, validation, and workflow integration challenges.