[Paper Review] Disentangling Prompt Dependence to Evaluate Segmentation Reliability in Gynecological MRI

Elodie Germani, Krystel Nyangoh-Timoh+2 more|arXiv (Cornell University)|Mar 9, 2026

Fetal and Pediatric Neurological Disorders0 citations

TL;DR

The paper introduces a framework to separate prompt ambiguity from local sensitivity in promptable segmentation, enabling robust reliability assessment in gynecological MRI. It uses a conditional MDN to model plausible prompts and a stability-margin predictor to gauge local prompt sensitivity, evaluated on two pelvic MRI datasets.

ABSTRACT

Promptable segmentation models (e.g., the Segment Anything Models) enable generalizable, zero-shot segmentation across diverse domains. Although predictions are deterministic for a fixed image-prompt pair, the robustness of these models to variations in user prompts, referred to as prompt dependence, remains underexplored. In safety-critical workflows with substantial inter-user variability, interpretable and informative frameworks are needed to evaluate prompt dependence. In this work, we assess the reliability of promptable segmentation by analyzing and measuring its sensitivity to prompt variability. We introduce the first formulation of prompt dependence that explicitly disentangles prompt ambiguity (inter-user variability) from local sensitivity (interaction imprecision), offering an interpretable view of segmentation robustness. Experiments on two female pelvic MRI datasets for uterus and bladder segmentation reveal a strong negative correlation between both metrics and segmentation performance, highlighting the value of our framework for assessing robustness. The two metrics have low mutual correlation, supporting the disentangled design of our formulation, and provide meaningful indicators of prompt-related failure modes.

Motivation & Objective

• Motivate robust evaluation of promptable segmentation in safety-critical medical workflows.
• Disentangle inter-user prompt ambiguity from interaction-imprecision (local sensitivity).
• Model image-conditioned prompts with a Mixture Density Network (MDN) to quantify prompt ambiguity.
• Define and estimate a stability margin to capture local sensitivity under prompt perturbations.
• Demonstrate the framework on two female pelvic MRI datasets with uterus and bladder segmentation.

Proposed method

• Formulate prompt dependence as variance decomposition into prompt ambiguity and local sensitivity.
• Model the distribution of valid prompts P_valid(b|I) with a conditional MDN to estimate prompt ambiguity U_amb = tr(Cov_p_theta(b|x)).
• Define a perturbation delta around a reference prompt and quantify the stability margin delta* as the smallest perturbation that crosses a mask-discrepancy threshold tau.
• Train a stability-margin predictor r_psi to fast-approximate delta*(I,b) without iterative sampling.
• Compute pixel-wise uncertainty maps by sampling 1000 prompts from p_theta(b|x) and aggregating resulting masks, yielding entropy maps aligned with boundaries.
• Evaluate with two SAM-based architectures (MobileSAM, MedSAM) on two pelvic MRI datasets (UT-EndoMRI, MOGaMBO).

Experimental results

Research questions

• RQ1 Can prompt ambiguity and local sensitivity be disentangled to provide an interpretable robustness assessment for promptable segmentation in medical MRI?
• RQ2 How does the image-conditioned prompt distribution relate to pixel-level uncertainty and segmentation quality?
• RQ3 Do prompt ambiguity and local sensitivity correlate with Dice scores, and are they largely independent?
• RQ4 Can a stability-margin predictor efficiently approximate the minimal prompt perturbation needed to alter the segmentation?
• RQ5 Are the proposed metrics useful for identifying and characterizing failure modes in prompt-based medical segmentation?

Key findings

• Prompt ambiguity and local sensitivity show low mutual correlation, supporting their disentangled interpretation.
• On both datasets and models, higher prompt ambiguity and higher local sensitivity are associated with lower Dice scores, indicating reduced segmentation quality.
• The conditional MDN accurately captures plausible prompts and enables pixel-level uncertainty maps that align with segmentation errors.
• Stability margins correlate with segmentation robustness, with larger delta* indicating more robust prompts.
• Uncertainty maps derived from the conditional prompt distribution provide spatially coherent, boundary-aligned uncertainty.
• The framework reveals dataset- and model-specific differences in how prompt dependence impacts performance.