[Paper Review] Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy

Stanislav Nikolov, Sam Blackwell+27 more|arXiv (Cornell University)|Sep 12, 2018

Advanced Radiotherapy TechniquesPhysics and Astronomy85 references248 citations

TL;DR

A 3D U-Net-based model segments 21 head and neck organs at risk in CT planning scans at expert radiographer level, with a new surface Dice similarity coefficient to reflect clinical correction effort, and demonstrated generalisability across multiple datasets.

ABSTRACT

Over half a million individuals are diagnosed with head and neck cancer each year worldwide. Radiotherapy is an important curative treatment for this disease, but it requires manual time consuming delineation of radio-sensitive organs at risk (OARs). This planning process can delay treatment, while also introducing inter-operator variability with resulting downstream radiation dose differences. While auto-segmentation algorithms offer a potentially time-saving solution, the challenges in defining, quantifying and achieving expert performance remain. Adopting a deep learning approach, we demonstrate a 3D U-Net architecture that achieves expert-level performance in delineating 21 distinct head and neck OARs commonly segmented in clinical practice. The model was trained on a dataset of 663 deidentified computed tomography (CT) scans acquired in routine clinical practice and with both segmentations taken from clinical practice and segmentations created by experienced radiographers as part of this research, all in accordance with consensus OAR definitions. We demonstrate the model's clinical applicability by assessing its performance on a test set of 21 CT scans from clinical practice, each with the 21 OARs segmented by two independent experts. We also introduce surface Dice similarity coefficient (surface DSC), a new metric for the comparison of organ delineation, to quantify deviation between OAR surface contours rather than volumes, better reflecting the clinical task of correcting errors in the automated organ segmentations. The model's generalisability is then demonstrated on two distinct open source datasets, reflecting different centres and countries to model training. With appropriate validation studies and regulatory approvals, this system could improve the efficiency, consistency, and safety of radiotherapy pathways.

Motivation & Objective

• Address the time, variability, and safety challenges in manual head and neck organ-at-risk delineation for radiotherapy planning.
• Develop and validate a deep learning segmentation model that reaches expert radiographer performance on clinically representative data.
• Propose a clinically meaningful evaluation metric (surface DSC) that reflects the effort to correct automated segmentations.

Proposed method

• Employ a 3D U-Net architecture to delineate 21 organs-at-risk on planning CT scans.
• Train on 663 deidentified CT scans from routine practice with clinically sourced and radiographer-created segmentations.
• Evaluate against two independent experts on a test set of 21 CT scans from the same hospital, plus open-source datasets to test generalisability.
• Introduce surface Dice similarity coefficient (surface DSC) to measure surface overlap within organ-specific tolerances, focusing on editable boundary regions rather than volumes.
• Compare model performance to experienced radiographers and to oncologist-ground-truth adjudications to establish expert-level performance.

Experimental results

Research questions

• RQ1 Can a deep learning model achieve expert-level delineation of 21 head and neck OARs on planning CT scans?
• RQ2 Does the model generalise to external datasets with different demographics and imaging protocols?
• RQ3 Does a surface-focused evaluation metric (surface DSC) better reflect clinical correction effort than volumetric metrics?
• RQ4 What is the comparative performance of the model versus radiographers and oncologists on representative test sets?

Key findings

• The model achieved surface DSC performance similar to radiographers for all 21 OARs on the UCLH test set within organ-specific tolerances.
• On the TCIA open-source test set, the model matched radiographers in 19 of 21 OARs, with two organs (brainstem and right lens) below radiographer performance likely due to image quality.
• A novel surface Dice similarity coefficient (surface DSC) was introduced to quantify surface-level agreement within organ-specific tolerances, showing clinically relevant assessment of corrections.
• Generalisation was demonstrated across three test cohorts (UCLH, TCIA, PDDCA), indicating robustness to different centres, demographics, and scanner/protocol variations.
• The authors released their TCIA-labelled dataset to support objective comparisons and future research in auto-segmentation for radiotherapy planning.