[Paper Review] CauCLIP: Bridging the Sim-to-Real Gap in Surgical Video Understanding via Causality-Inspired Vision-Language Modeling

Yuxin He, An Li+1 more|arXiv (Cornell University)|Feb 6, 2026

Surgical Simulation and Training0 citations

TL;DR

CauCLIP proposes a causality-guided, CLIP-based framework for robust surgical phase recognition that generalizes across sim-to-real domain shifts without target-domain data, using frequency-based augmentation and a causal suppression loss.

ABSTRACT

Surgical phase recognition is a critical component for context-aware decision support in intelligent operating rooms, yet training robust models is hindered by limited annotated clinical videos and large domain gaps between synthetic and real surgical data. To address this, we propose CauCLIP, a causality-inspired vision-language framework that leverages CLIP to learn domain-invariant representations for surgical phase recognition without access to target domain data. Our approach integrates a frequency-based augmentation strategy to perturb domain-specific attributes while preserving semantic structures, and a causal suppression loss that mitigates non-causal biases and reinforces causal surgical features. These components are combined in a unified training framework that enables the model to focus on stable causal factors underlying surgical workflows. Experiments on the SurgVisDom hard adaptation benchmark demonstrate that our method substantially outperforms all competing approaches, highlighting the effectiveness of causality-guided vision-language models for domain-generalizable surgical video understanding.

Motivation & Objective

• Address the sim-to-real domain gap in surgical phase recognition due to limited annotated real videos.
• Develop a causality-inspired vision-language framework built on CLIP to learn domain-invariant representations.
• Introduce a frequency-based augmentation and a causal suppression loss to reduce non-causal biases.

Proposed method

• Build on CLIP-based video-text alignment for surgical phase recognition as a video-text matching task.
• Introduce frequency-domain augmentation that perturbs high-frequency, non-causal cues while preserving semantics.
• Implement a causal suppression module that enforces similarity between original and frequency-augmented representations and decorrelates non-causal features.
• Add an augmented alignment loss to preserve semantic consistency across augmented views.
• Train with a total loss combining the original CLIP loss, augmented alignment loss, and suppression loss.

Experimental results

Research questions

• RQ1 How can causality-inspired components improve cross-domain generalization for surgical phase recognition without target-domain access?
• RQ2 Do frequency-domain augmentations and causal suppression jointly improve robustness to domain shifts compared to baseline CLIP-based methods?
• RQ3 Can a vision-language approach leverage multimodal supervision to exceed domain-adapted baselines on SurgVisDom hard adaptation?
• RQ4 What is the contribution of each proposed component to overall performance?

Key findings

• CauCLIP achieves state-of-the-art performance on the SurgVisDom hard adaptation benchmark across weighted F1, unweighted F1, global F1, and balanced accuracy.
• It outperforms multiple baselines including Rand, SK, Parakeet, ResNet-50, ViT-B/16, and SDA-CLIP on all reported metrics.
• Ablation studies show both the causality-inspired suppression (L_sup) and frequency-domain augmentation (L_aug) provide gains, with the full model performing best.
• The combination of augmentation and suppression yields complementary benefits, improving robustness to style variations and emphasizing causal surgical semantics.