[Paper Review] Benchmarking Zero-Shot Robustness of Multimodal Foundation Models: A Pilot Study

Chenguang Wang, Ruoxi Jia+2 more|arXiv (Cornell University)|Mar 15, 2024

Infrastructure Maintenance and Monitoring4 citations

TL;DR

This paper introduces RoZ, a comprehensive benchmark to evaluate the zero-shot robustness of multimodal foundation models like CLIP across 7 natural, 3 synthetic distribution shifts, and 11 adversarial attacks. Despite strong zero-shot accuracy, CLIP shows significant robustness degradation—averaging -11.8% on synthetic shifts and -34.7% on typographic attacks—highlighting a critical robustness gap compared to supervised ImageNet models.

ABSTRACT

Pre-training image representations from the raw text about images enables zero-shot vision transfer to downstream tasks. Through pre-training on millions of samples collected from the internet, multimodal foundation models, such as CLIP, produce state-of-the-art zero-shot results that often reach competitiveness with fully supervised methods without the need for task-specific training. Besides the encouraging performance on classification accuracy, it is reported that these models close the robustness gap by matching the performance of supervised models trained on ImageNet under natural distribution shift. Because robustness is critical to real-world applications, especially safety-critical ones, in this paper, we present a comprehensive evaluation based on a large-scale robustness benchmark covering 7 natural, 3 synthetic distribution shifts, and 11 adversarial attacks. We use CLIP as a pilot study. We show that CLIP leads to a significant robustness drop compared to supervised ImageNet models on our benchmark, especially under synthetic distribution shift and adversarial attacks. Furthermore, data overlap analysis suggests that the observed robustness under natural distribution shifts could be attributed, at least in part, to data overlap. In summary, our evaluation shows a comprehensive evaluation of robustness is necessary; and there is a significant need to improve the robustness of zero-shot multimodal models.

Motivation & Objective

• To systematically evaluate the robustness of zero-shot multimodal foundation models beyond natural distribution shifts.
• To identify whether observed robustness on natural shifts is due to data overlap or genuine model generalization.
• To develop a comprehensive benchmark (RoZ) that includes natural, synthetic, and adversarial robustness evaluations.
• To assess the impact of prompt engineering and automatic prompt generation on robustness in zero-shot settings.
• To highlight the urgent need for improved robustness in zero-shot multimodal models for safety-critical applications.

Proposed method

• Constructed RoZ benchmark with 7 natural distribution shifts (e.g., ImageNet-V2, ObjectNet), 3 synthetic shifts (e.g., ImageNet-C, Stylized ImageNet), and 11 adversarial attacks.
• Evaluated CLIP with multiple vision encoders (ViT-B/16, ResNet50x16) and CLIP-Auto, which uses automatic prompt generation.
• Performed data overlap analysis between test sets and ImageNet to assess whether robustness on natural shifts stems from dataset overlap.
• Introduced a novel typographic attack test set (ImageNet-T, CIFAR-10-T) targeting multimodal model vulnerabilities in prompt-based inference.
• Compared zero-shot CLIP performance against standard ImageNet models under all robustness conditions.
• Used standardized zero-shot prompting with natural language templates to evaluate model generalization without fine-tuning.

(a) Seven natural distribution shifts.

Experimental results

Research questions

• RQ1 To what extent does CLIP maintain robustness under synthetic distribution shifts and adversarial attacks compared to supervised ImageNet models?
• RQ2 Is the observed robustness of CLIP on natural distribution shifts due to data overlap with the ImageNet training set?
• RQ3 How does automatic prompt generation (CLIP-Auto) affect robustness compared to standard CLIP and supervised models?
• RQ4 What vulnerabilities emerge in multimodal zero-shot models under novel adversarial attacks like typographic perturbations?
• RQ5 Can a comprehensive benchmark like RoZ reliably identify robustness weaknesses in zero-shot multimodal foundation models?

Key findings

• CLIP exhibits a significant robustness drop of -11.8% on average across synthetic distribution shifts (e.g., ImageNet-C, Stylized ImageNet) compared to supervised ImageNet models.
• On adversarial attacks, CLIP underperforms standard ImageNet models, with a robustness degradation of -34.7% on the newly introduced typographic attack sets (ImageNet-T and CIFAR-10-T).
• Data overlap analysis reveals that observed robustness on natural distribution shifts (e.g., ImageNet-A, ImageNet-R) is at least partially attributable to overlapping samples with the ImageNet training set.
• CLIP-Auto, which uses automatic prompt generation, performs similarly to standard CLIP across all test sets, indicating no robustness improvement from prompt automation.
• Despite strong zero-shot accuracy, CLIP fails to generalize robustly under non-natural distribution shifts, exposing a critical vulnerability in real-world deployment.
• The RoZ benchmark successfully identifies robustness weaknesses in zero-shot multimodal models, demonstrating the need for systematic robustness evaluation beyond standard zero-shot accuracy.

(b) Robustness test sets except for the distribution shifts in (a).