[Paper Review] CARLA-GeAR: a Dataset Generator for a Systematic Evaluation of Adversarial Robustness of Vision Models

Federico Nesti, Giulio Rossolini+3 more|arXiv (Cornell University)|Jun 9, 2022

Adversarial Robustness in Machine Learning4 citations

TL;DR

CARLA-GeAR is a dataset generator built on the CARLA simulator that creates photo-realistic synthetic datasets for evaluating adversarial robustness of vision models in autonomous driving scenarios. It enables systematic benchmarking of physical adversarial patches via automated generation of datasets for semantic segmentation, object detection, and depth estimation, with experimental validation showing significant performance degradation under attacks and varying defense effectiveness.

ABSTRACT

Adversarial examples represent a serious threat for deep neural networks in several application domains and a huge amount of work has been produced to investigate them and mitigate their effects. Nevertheless, no much work has been devoted to the generation of datasets specifically designed to evaluate the adversarial robustness of neural models. This paper presents CARLA-GeAR, a tool for the automatic generation of photo-realistic synthetic datasets that can be used for a systematic evaluation of the adversarial robustness of neural models against physical adversarial patches, as well as for comparing the performance of different adversarial defense/detection methods. The tool is built on the CARLA simulator, using its Python API, and allows the generation of datasets for several vision tasks in the context of autonomous driving. The adversarial patches included in the generated datasets are attached to billboards or the back of a truck and are crafted by using state-of-the-art white-box attack strategies to maximize the prediction error of the model under test. Finally, the paper presents an experimental study to evaluate the performance of some defense methods against such attacks, showing how the datasets generated with CARLA-GeAR might be used in future work as a benchmark for adversarial defense in the real world. All the code and datasets used in this paper are available at http://carlagear.retis.santannapisa.it.

Motivation & Objective

• Address the lack of standardized benchmarks for evaluating adversarial robustness of vision models in real-world autonomous driving scenarios.
• Overcome practical and safety limitations of real-world physical adversarial attacks by enabling fully controllable, photo-realistic simulation-based evaluation.
• Systematically compare the performance of different adversarial defense and detection methods under realistic physical attack conditions.
• Provide a reusable, open-source tool for generating datasets with adversarial patches attached to real-world objects like billboards and trucks.
• Establish a foundation for future benchmarking of adversarial robustness and defense methods through standardized, reproducible evaluation.

Proposed method

• Leverages the CARLA simulator’s Python API to generate high-definition, photo-realistic RGB images and ground-truth labels for urban driving scenarios.
• Automatically attaches adversarial patches to surfaces such as billboards or truck backs using state-of-the-art white-box attack strategies to maximize model prediction error.
• Supports four vision tasks: semantic segmentation, 2D and 3D stereo object detection, and monocular depth estimation, with synchronized camera and scene metadata.
• Generates patches via optimization on a pre-collected dataset without patches, using a loss function tailored to maximize misclassification under physical constraints.
• Collects multiple datasets across different attack scenarios and object placements to enable systematic evaluation of model robustness and defense mechanisms.
• Provides pre-computed datasets with patches and supports custom generation upon request to reduce computational overhead.

Experimental results

Research questions

• RQ1 How effective are state-of-the-art defense methods (e.g., Z-Mask, LGS) in mitigating physical adversarial attacks on vision models in autonomous driving scenarios?
• RQ2 To what extent do distributional shifts between CARLA-generated data and real-world training data (e.g., COCO, CityScapes) affect model performance on generated adversarial datasets?
• RQ3 Can a simulated, photo-realistic environment like CARLA enable reliable and systematic evaluation of adversarial robustness across diverse attack configurations?
• RQ4 How do different vision tasks (e.g., semantic segmentation vs. depth estimation) respond to the same adversarial patch in terms of performance degradation?
• RQ5 What are the computational and practical challenges in generating large-scale, realistic adversarial datasets for benchmarking?

Key findings

• The use of adversarial patches significantly degrades model performance across all tasks, with average performance drops ranging from -25.48% to -45.41% in the worst-case scenarios.
• Defensive methods like Z-Mask and LGS show mixed results: Z-Mask improves performance in some cases (e.g., +8.84% on Billboard04 for Stereo R-CNN) but worsens it in others (e.g., -33.17% on Billboard06 for DDRNet).
• LGS defense improves performance on Billboard07 (13.35% relative gain) and Billboard08 (15.84% gain), indicating task- and scenario-dependent effectiveness.
• The model performance on CARLA-generated datasets is substantially worse than on real-world training data, indicating a significant distributional shift due to synthetic data characteristics.
• The tool enables consistent and reproducible evaluation of adversarial robustness, with datasets available for download and custom generation upon request.
• The study demonstrates that CARLA-GeAR can serve as a foundation for future benchmarking, though standardized metrics and evaluation platforms remain to be developed.