[Paper Review] CPPE-5: Medical Personal Protective Equipment Dataset

Rishit Dagli, Ali Mustufa Shaikh|arXiv (Cornell University)|Dec 15, 2021

Advanced Neural Network Applications4 citations

TL;DR

This paper introduces CPPE-5, a novel object detection dataset focused on subordinate categorization of medical personal protective equipment (PPE) such as masks, gloves, goggles, face shields, and coveralls. Collected from real-world scenes with non-iconic, complex views, the dataset contains 5,000+ images with dense, multi-instance bounding box annotations. The key contribution is establishing a benchmark for detecting PPE in realistic, cluttered environments, with state-of-the-art models achieving a box AP of 52.9% on the dataset.

ABSTRACT

We present a new challenging dataset, CPPE - 5 (Medical Personal Protective Equipment), with the goal to allow the study of subordinate categorization of medical personal protective equipments, which is not possible with other popular data sets that focus on broad-level categories (such as PASCAL VOC, ImageNet, Microsoft COCO, OpenImages, etc). To make it easy for models trained on this dataset to be used in practical scenarios in complex scenes, our dataset mainly contains images that show complex scenes with several objects in each scene in their natural context. The image collection for this dataset focuses on: obtaining as many non-iconic images as possible and making sure all the images are real-life images, unlike other existing datasets in this area. Our dataset includes 5 object categories (coveralls, face shields, gloves, masks, and goggles), and each image is annotated with a set of bounding boxes and positive labels. We present a detailed analysis of the dataset in comparison to other popular broad category datasets as well as datasets focusing on personal protective equipments, we also find that at present there exist no such publicly available datasets. Finally, we also analyze performance and compare model complexities on baseline and state-of-the-art models for bounding box results. Our code, data, and trained models are available at https://git.io/cppe5-dataset.

Motivation & Objective

• To address the lack of publicly available datasets focused on subordinate categorization of medical PPE.
• To provide a realistic benchmark for object detection in complex, natural scenes with multiple PPE items per image.
• To support the development of robust computer vision models for critical real-world applications such as pandemic response.
• To promote research in detecting PPE in non-canonical, challenging viewpoints and occluded conditions.

Proposed method

• Images were collected primarily from Flickr and a subset from Google Images to maximize diversity and realism.
• Each image was annotated with oriented bounding boxes and class labels using crowd-sourcing with multiple reviewers for quality control.
• The dataset includes five PPE categories: coveralls, face shields, gloves, masks, and goggles, with annotations for multiple objects per image.
• A baseline YOLOv3 model was trained with a cosine decay schedule and focal loss for classification and smooth L1 for localization.
• State-of-the-art models including Deformable DETR, FCOS, and Double Heads were fine-tuned and evaluated using standard COCO-style metrics.
• Training used a warmup phase of 2000 iterations with an initial base learning rate of 4/300 and weight decay of 0.04.

Experimental results

Research questions

• RQ1 Can existing object detection models generalize to the detection of medical PPE in complex, real-world scenes with multiple overlapping items?
• RQ2 How does the performance of state-of-the-art models vary on a dataset emphasizing non-iconic, real-life PPE images compared to standard benchmarks?
• RQ3 To what extent does the inclusion of diverse, non-canonical viewpoints and occlusions affect detection accuracy in medical PPE detection?
• RQ4 Is there a significant performance gap between models trained on generic datasets and those fine-tuned on CPPE-5 for medical PPE detection?

Key findings

• The CPPE-5 dataset is the first publicly available benchmark focused on subordinate categorization of medical PPE, filling a critical gap in existing datasets.
• The dataset contains 5,000+ real-world images with an average of more than one PPE object per image, emphasizing complex, cluttered scenes.
• The best-performing model, TridentNet, achieved a box AP of 52.9%, significantly outperforming baseline models and demonstrating the dataset's challenge.
• Models like Empirical Attention and Double Heads achieved high performance (AP > 50%) on CPPE-5, indicating strong generalization to non-iconic views.
• The dataset's emphasis on non-iconic images and natural context makes it more representative of real-world deployment than standard datasets like COCO or VOC.
• The results show that CPPE-5 is more challenging than OpenImages and COCO due to complex scenes, varied viewpoints, and occlusions.