[Paper Review] EgoBlur: Responsible Innovation in Aria

Nikhil Raina, Guruprasad Somasundaram+13 more|arXiv (Cornell University)|Aug 24, 2023

Ethics and Social Impacts of AISocial Sciences3 citations

TL;DR

EgoBlur is a privacy-preserving anonymization system for egocentric video recordings from Meta's Aria glasses, using a FasterRCNN-based detector to identify faces and license plates, followed by Gaussian blur to obscure personal identification information. It achieves state-of-the-art performance with over 93.8% average recall on the CCV2 dataset and 99.2% recall on license plate detection, demonstrating strong robustness across diverse real-world conditions and responsible AI attributes.

ABSTRACT

Project Aria pushes the frontiers of Egocentric AI with large-scale real-world data collection using purposely designed glasses with privacy first approach. To protect the privacy of bystanders being recorded by the glasses, our research protocols are designed to ensure recorded video is processed by an AI anonymization model that removes bystander faces and vehicle license plates. Detected face and license plate regions are processed with a Gaussian blur such that these personal identification information (PII) regions are obscured. This process helps to ensure that anonymized versions of the video is retained for research purposes. In Project Aria, we have developed a state-of-the-art anonymization system EgoBlur. In this paper, we present extensive analysis of EgoBlur on challenging datasets comparing its performance with other state-of-the-art systems from industry and academia including extensive Responsible AI analysis on recently released Casual Conversations V2 dataset.

Motivation & Objective

• To develop a scalable, privacy-first system for anonymizing personal identification information (PII) in real-world egocentric video recordings from Aria glasses.
• To ensure robust performance across diverse demographic and environmental conditions, including low-light, occluded, and non-frontal faces.
• To establish a strong baseline for vehicle license plate detection and anonymization in egocentric video streams.
• To evaluate the system’s fairness and performance across responsible AI attributes such as age, gender, skin tone, and country of origin.
• To enable flexible replacement of the underlying object detector while maintaining high accuracy and privacy guarantees.

Proposed method

• Employed a weakly supervised training approach using RetinaFace as a teacher model to generate pseudo-ground truth annotations for face detection on a large-scale image corpus.
• Trained a FasterRCNN-based object detector with a ResNeXt-101-32x8-FPN backbone using Detectron2, leveraging long-term training schedules and increased grayscale image augmentation.
• Applied Gaussian blur to detected face and license plate regions to anonymize PII, ensuring no residual identification information remains.
• Collected and manually annotated two benchmark datasets: 259,656 face bounding boxes on CCV2 and 23,242 on the Aria Pilot Dataset with fine-grained attributes (e.g., occluded, wearing glasses, low-light).
• Used Intersection over Union (IoU) with 0.5 threshold, along with Average Precision (AP) and Average Recall (AR) as evaluation metrics for both face and license plate detection.
• Conducted multi-annotator, majority-vote labeling to minimize bias in attribute annotations for fine-grained performance analysis.

Experimental results

Research questions

• RQ1 How does EgoBlur’s face detection performance compare to state-of-the-art systems (e.g., MediaPipe, RetinaFace) across diverse demographic and environmental attributes on the CCV2 dataset?
• RQ2 What is the performance of EgoBlur in detecting faces under challenging egocentric conditions such as occlusion, truncation, low lighting, and non-frontal views on the Aria Pilot Dataset?
• RQ3 Can a generic, task-agnostic FasterRCNN detector outperform specialized face detectors like RetinaFace and MediaPipe in real-world egocentric video settings?
• RQ4 How effective is the EgoBlur system in detecting vehicle license plates across varying recording conditions (e.g., distance, angle, time of day) in real-world Aria recordings?
• RQ5 To what extent does the system maintain fairness and consistency across demographic buckets (e.g., skin tone, gender, country of origin) in face detection performance?

Key findings

• EgoBlur achieved 93.8% average recall and 89.5% average precision on the CCV2 dataset, outperforming both MediaPipe and RetinaFace across all demographic and environmental subgroups.
• On the Aria Pilot Dataset, EgoBlur achieved 95.7% recall for faces with glasses, 92.8% for non-frontal views, and 89.2% for occluded faces—surpassing both MediaPipe and RetinaFace in all fine-grained categories.
• In grayscale conditions, EgoBlur achieved 91.4% recall for faces with glasses and 88.0% for non-frontal views, demonstrating strong robustness to low-light and monochrome imaging.
• For license plate detection, EgoBlur achieved 98.2% recall and 96.3% average precision on grayscale streams, and 99.2% recall with 92.9% AP on RGB streams, indicating high reliability across lighting and color conditions.
• The system demonstrated consistent performance across all demographic buckets in the CCV2 dataset, with no significant performance drop in any subgroup, indicating strong fairness and generalization.
• The use of a generic FasterRCNN detector outperformed specialized face detectors like RetinaFace and MediaPipe in egocentric settings, validating the effectiveness of a unified, scalable detection pipeline.