[Paper Review] ECONet: Efficient Convolutional Online Likelihood Network for Scribble-based Interactive Segmentation

Muhammad Asad, Lucas Fidon+1 more|arXiv (Cornell University)|Jan 12, 2022

COVID-19 diagnosis using AI6 citations

TL;DR

ECONet proposes a lightweight, fully convolutional online learning network for scribble-based interactive segmentation of COVID-19 lung lesions in CT scans. It enables real-time adaptation using only user-provided scribbles via patch-based online training with weighted cross-entropy, achieving 16% higher Dice score than state-of-the-art methods while reducing inference time by 3× and requiring 9,000 fewer labeled voxels.

ABSTRACT

Automatic segmentation of lung lesions associated with COVID-19 in CT images requires large amount of annotated volumes. Annotations mandate expert knowledge and are time-intensive to obtain through fully manual segmentation methods. Additionally, lung lesions have large inter-patient variations, with some pathologies having similar visual appearance as healthy lung tissues. This poses a challenge when applying existing semi-automatic interactive segmentation techniques for data labelling. To address these challenges, we propose an efficient convolutional neural networks (CNNs) that can be learned online while the annotator provides scribble-based interaction. To accelerate learning from only the samples labelled through user-interactions, a patch-based approach is used for training the network. Moreover, we use weighted cross-entropy loss to address the class imbalance that may result from user-interactions. During online inference, the learned network is applied to the whole input volume using a fully convolutional approach. We compare our proposed method with state-of-the-art using synthetic scribbles and show that it outperforms existing methods on the task of annotating lung lesions associated with COVID-19, achieving 16% higher Dice score while reducing execution time by 3$\ imes$ and requiring 9000 lesser scribbles-based labelled voxels. Due to the online learning aspect, our approach adapts quickly to user input, resulting in high quality segmentation labels. Source code for ECONet is available at: https://github.com/masadcv/ECONet-MONAILabel.

Motivation & Objective

• To address the challenge of time-intensive, expert-dependent manual segmentation of COVID-19 lung lesions in CT volumes.
• To overcome limitations of appearance-based models in distinguishing lesions from healthy lung tissue due to overlapping HU intensity distributions.
• To develop an efficient, online learning framework that adapts quickly to user scribbles without pre-training.
• To reduce the number of required labeled voxels while maintaining high segmentation accuracy.
• To enable rapid data labeling for emerging pathologies with ambiguous visual features.

Proposed method

• ECONet employs a lightweight fully convolutional neural network trained in real-time using only patches containing user-provided scribble labels.
• The network uses a patch-based training strategy to accelerate learning from sparse user interactions.
• Weighted cross-entropy loss is applied to mitigate class imbalance caused by sparse scribbles.
• Online inference is performed via a fully convolutional forward pass on the full 3D volume.
• The model is trained end-to-end without pre-training, enabling immediate adaptation to new annotations.
• A dynamic learning strategy updates the network weights incrementally after each user interaction.

Experimental results

Research questions

• RQ1 Can a deep learning model achieve high segmentation accuracy for ambiguous lung lesions using only minimal scribble annotations?
• RQ2 How does online learning from sparse user interactions compare to offline or pre-trained models in terms of speed and accuracy?
• RQ3 Can a patch-based training strategy reduce computational cost while maintaining performance in interactive segmentation?
• RQ4 To what extent does weighted cross-entropy loss improve model robustness under class imbalance from sparse labeling?
• RQ5 Can an end-to-end online learning framework outperform existing online likelihood models in both accuracy and efficiency?

Key findings

• ECONet achieved a 16% higher Dice score than the state-of-the-art DybaORF-Haar-Like method on synthetic scribbles.
• ECONet reduced inference time by a factor of 3 compared to DybaORF-Haar-Like, achieving online training and inference in ~2 seconds.
• ECONet required 9,000 fewer labeled voxels than DybaORF-Haar-Like to achieve comparable performance.
• 70% of the dataset achieved a Dice score above 80% using ECONet, compared to only 15% for DybaORF-Haar-Like.
• ECONet outperformed GMM and histogram-based methods, which failed to generalize and even deteriorated with more labeled voxels.
• Qualitative results using non-expert scribbles showed ECONet produced segmentations close to ground truth, indicating robustness to noisy user input.