[Paper Review] Privacy-Protection Drone Patrol System based on Face Anonymization

Harim Lee, Myeung Un Kim+3 more|arXiv (Cornell University)|May 29, 2020

Face recognition and analysis1 references4 citations

TL;DR

This paper proposes a privacy-preserving drone patrol system that uses a modified CycleGAN and GauGAN-based generative adversarial network (GAN) framework to anonymize faces in real time on a drone's companion computer. The system transforms original faces into photorealistic but different identities while preserving visual features essential for robot perception, achieving effective privacy protection without degrading SLAM performance.

ABSTRACT

The robot market has been growing significantly and is expected to become 1.5 times larger in 2024 than what it was in 2019. Robots have attracted attention of security companies thanks to their mobility. These days, for security robots, unmanned aerial vehicles (UAVs) have quickly emerged by highlighting their advantage: they can even go to any hazardous place that humans cannot access. For UAVs, Drone has been a representative model and has several merits to consist of various sensors such as high-resolution cameras. Therefore, Drone is the most suitable as a mobile surveillance robot. These attractive advantages such as high-resolution cameras and mobility can be a double-edged sword, i.e., privacy infringement. Surveillance drones take videos with high-resolution to fulfill their role, however, those contain a lot of privacy sensitive information. The indiscriminate shooting is a critical issue for those who are very reluctant to be exposed. To tackle the privacy infringement, this work proposes face-anonymizing drone patrol system. In this system, one person's face in a video is transformed into a different face with facial components maintained. To construct our privacy-preserving system, we have adopted the latest generative adversarial networks frameworks and have some modifications on losses of those frameworks. Our face-anonymzing approach is evaluated with various public face-image and video dataset. Moreover, our system is evaluated with a customized drone consisting of a high-resolution camera, a companion computer, and a drone control computer. Finally, we confirm that our system can protect privacy sensitive information with our face-anonymzing algorithm while preserving the performance of robot perception, i.e., simultaneous localization and mapping.

Motivation & Objective

• To address growing concerns about privacy infringement from surveillance drones equipped with high-resolution cameras.
• To develop a real-time face anonymization system that protects personal identity without compromising drone perception capabilities.
• To integrate a privacy-preserving vision pipeline into a functional drone patrol system using onboard deep learning inference.
• To evaluate the system’s effectiveness on public datasets and real-world drone video recordings.
• To ensure that face anonymization does not degrade vision-based robot perception, such as ORB-SLAM2 performance.

Proposed method

• The system combines CycleGAN for domain translation and GauGAN for semantic image synthesis, creating a two-stage GAN framework for face anonymization.
• A modified loss function is applied to the generators to preserve facial structure while replacing identity, ensuring photorealism and privacy.
• The anonymization pipeline is deployed on a Jetson Xavier companion computer mounted on a custom drone with a ZED stereo camera.
• Video frames are processed in real time using the ROS framework, enabling end-to-end integration of perception and privacy protection.
• The system processes each frame to replace detected faces with new, non-identifiable faces while maintaining spatial and geometric consistency.
• ORB-SLAM2 is used to evaluate whether anonymization affects visual odometry and feature extraction, ensuring compatibility with standard robot perception stacks.

Experimental results

Research questions

• RQ1 Can a GAN-based face anonymization system effectively protect personal identity in real-time drone video streams?
• RQ2 To what extent does face anonymization impact the performance of vision-based robot perception systems like ORB-SLAM2?
• RQ3 Can a hybrid GAN architecture combining CycleGAN and GauGAN achieve high-quality, identity-preserving face anonymization with structural fidelity?
• RQ4 How well does the system generalize across diverse face datasets and real-world drone video scenarios?
• RQ5 Is it feasible to deploy such a privacy-preserving system on an embedded platform like Jetson Xavier for real-time operation on a drone?

Key findings

• The proposed face-anonymizing system successfully transforms original faces into new, non-identifiable faces while preserving facial components and structure.
• Evaluation on multiple datasets, including CelebA-HQ, Helen, and FaceScrub, confirmed consistent and high-quality anonymization results across diverse facial appearances.
• ORB-SLAM2 feature extraction performance remained nearly identical between original and anonymized video frames, with minimal deviation in detected feature points.
• The distance measurements between the ZED camera and detected features (e.g., face region) were preserved within acceptable margins, confirming no degradation in depth estimation.
• The system achieved real-time processing on the Jetson Xavier companion computer, demonstrating feasibility for deployment on mobile drones.
• The integration of the anonymization pipeline with ROS enabled seamless communication between the camera, processing unit, and perception system on the drone.