[Paper Review] Benchmarking off-the-shelf statistical shape modeling tools in clinical applications

Anupama Goparaju, Alexandre Bône+9 more|arXiv (Cornell University)|Sep 6, 2020

Morphological variations and asymmetry87 references47 citations

TL;DR

The paper benchmarks three open-source statistical shape modeling tools (ShapeWorks, Deformetrica, SPHARM-PDM) using extrinsic validation frameworks for clinical relevance.

ABSTRACT

Statistical shape modeling (SSM) is widely used in biology and medicine as a new generation of morphometric approaches for the quantitative analysis of anatomical shapes. Technological advancements of in vivo imaging have led to the development of open-source computational tools that automate the modeling of anatomical shapes and their population-level variability. However, little work has been done on the evaluation and validation of such tools in clinical applications that rely on morphometric quantifications (e.g., implant design and lesion screening). Here, we systematically assess the outcome of widely used, state-of-the-art SSM tools, namely ShapeWorks, Deformetrica, and SPHARM-PDM. We use both quantitative and qualitative metrics to evaluate shape models from different tools. We propose validation frameworks for anatomical landmark/measurement inference and lesion screening. We also present a lesion screening method to objectively characterize subtle abnormal shape changes with respect to learned population-level statistics of controls. Results demonstrate that SSM tools display different levels of consistencies, where ShapeWorks and Deformetrica models are more consistent compared to models from SPHARM-PDM due to the groupwise approach of estimating surface correspondences. Furthermore, ShapeWorks and Deformetrica shape models are found to capture clinically relevant population-level variability compared to SPHARM-PDM models.

Motivation & Objective

• Motivate evaluation and validation of SSM tools in clinical contexts such as implant design, planning, and lesion screening.
• Compare the consistency and population-level variability capture of ShapeWorks, Deformetrica, and SPHARM-PDM.
• Propose validation frameworks for anatomical landmark inference and lesion screening.
• Demonstrate how population-level statistics can inform objective clinical decisions.

Proposed method

• Review and summarize the modeling approaches of ShapeWorks, Deformetrica, and SPHARM-PDM.
• Develop extrinsic evaluation and validation frameworks that do not rely on ground-truth correspondences.
• Use data preprocessing, data splitting with importance sampling, and hyperparameter tuning for each tool.
• Evaluate shape models with compactness, generalization, and specificity metrics.
• Perform qualitative assessment via modes of variation and cluster analysis to identify clinically relevant patterns.

Experimental results

Research questions

• RQ1 How do open-source SSM tools differ in producing consistent correspondences across a population?
• RQ2 Which tools best capture clinically relevant population-level variability?
• RQ3 Can extrinsic validation frameworks assess the clinical usefulness of shape models when ground-truth correspondences are unavailable?
• RQ4 Can SSM-based landmark inference and lesion screening provide objective guidance for clinical decisions?
• RQ5 What are the effects of tool-specific assumptions (groupwise vs pairwise) on shape modeling outcomes?

Key findings

• ShapeWorks and Deformetrica show higher consistency than SPHARM-PDM due to groupwise correspondence estimation.
• ShapeWorks and Deformetrica tend to capture population-level variability more effectively than SPHARM-PDM.
• The proposed validation frameworks enable assessment of landmark inference and lesion screening even without ground-truth correspondences.
• Extrinsic evaluation reveals differences in compactness, generalization, and specificity across tools.
• Qualitative analyses of modes of variation and clustering reveal clinically interpretable patterns in shape populations.