[Paper Review] Depth Anything in Medical Images: A Comparative Study

John Han, Ayberk Acar+2 more|arXiv (Cornell University)|Jan 29, 2024

AI in cancer detection4 citations

TL;DR

This study evaluates the zero-shot performance of the Depth Anything foundation model on medical endoscopic and laparoscopic images, comparing it against general-scene and in-domain monocular depth estimation (MDE) models. Despite strong zero-shot capabilities, Depth Anything does not consistently outperform prior models in accuracy or speed, with in-domain models like Endo-Depth and MiDaS showing superior results on specific anatomical sequences, though Depth Anything offers favorable inference speeds for real-time applications.

ABSTRACT

Monocular depth estimation (MDE) is a critical component of many medical tracking and mapping algorithms, particularly from endoscopic or laparoscopic video. However, because ground truth depth maps cannot be acquired from real patient data, supervised learning is not a viable approach to predict depth maps for medical scenes. Although self-supervised learning for MDE has recently gained attention, the outputs are difficult to evaluate reliably and each MDE's generalizability to other patients and anatomies is limited. This work evaluates the zero-shot performance of the newly released Depth Anything Model on medical endoscopic and laparoscopic scenes. We compare the accuracy and inference speeds of Depth Anything with other MDE models trained on general scenes as well as in-domain models trained on endoscopic data. Our findings show that although the zero-shot capability of Depth Anything is quite impressive, it is not necessarily better than other models in both speed and performance. We hope that this study can spark further research in employing foundation models for MDE in medical scenes.

Motivation & Objective

• To assess the zero-shot generalization of the Depth Anything foundation model on medical endoscopic and laparoscopic scenes.
• To compare the accuracy and inference speed of Depth Anything against general-scene MDE models (MiDaS, ZoeDepth) and in-domain models (Endo-SfM, Endo-Depth).
• To evaluate whether foundation models like Depth Anything can reliably generalize across diverse anatomical structures without fine-tuning.
• To provide a benchmark for future research on foundation models in medical monocular depth estimation.
• To explore the feasibility of using pre-trained foundation models for real-time surgical navigation and 3D reconstruction.

Proposed method

• The study evaluates Depth Anything, MiDaS, ZoeDepth, Endo-SfM, Endo-Depth, and LapDepth on two public medical datasets: EndoSLAM and rectified Hamlyn Dataset.
• Zero-shot inference is performed without any fine-tuning on medical data, using the original pre-trained weights of each model.
• Quantitative metrics include absolute relative error (Abs. Rel.), squared relative error (Sq. Rel.), root mean square error (RMSE), and delta1 (δ1) for depth accuracy.
• Inference speed is measured on an Intel i9-13900K CPU and NVIDIA RTX 4090 GPU using input resolution (320, 320).
• Performance is analyzed across four sequences representing different anatomical regions: colon, small intestine, stomach, and rectum.
• The study compares results across models, highlighting trade-offs between accuracy and speed, and discusses limitations due to imperfect ground truth in the Hamlyn dataset.

Experimental results

Research questions

• RQ1 Does the Depth Anything foundation model achieve superior zero-shot performance on medical endoscopic scenes compared to general-scene MDE models?
• RQ2 How does the zero-shot performance of Depth Anything compare to in-domain MDE models trained specifically on endoscopic data?
• RQ3 What is the inference speed of Depth Anything relative to other MDE models, and is it suitable for real-time surgical navigation systems?
• RQ4 To what extent do ground truth depth maps in public medical datasets affect the reliability of performance evaluation?
• RQ5 Can foundation models like Depth Anything generalize effectively across diverse anatomical structures without fine-tuning?

Key findings

• On the EndoSLAM dataset, MiDaS outperformed Depth Anything in the colon sequence, and Endo-SfMLearner and MiDaS surpassed all Depth Anything variants in the small intestine sequence.
• For the stomach sequence, DAM-Large achieved the highest accuracy among all models, with an Abs. Rel. of 0.078 and δ1 of 0.953.
• On the rectified Hamlyn Dataset, ZoeDepth and Depth Anything-Indoor/Outdoor achieved comparable performance, with ZoeDepth scoring 0.083 Abs. Rel. and 0.949 δ1 on sequence 1.
• Endo-Depth was the fastest model, achieving 249.38 FPS on the same hardware, outperforming even the smallest Depth Anything variant (120.34 FPS).
• ZoeDepth had the slowest inference time at 82.15 ms (12.17 FPS), but still within real-time capability for surgical applications.
• The Hamlyn dataset's ground truth depth maps were found to be imperfect, potentially affecting the reliability of performance comparisons, especially in sequences 4, 19, and 20.