[Paper Review] Enable Portrait Privacy Protection in Photo Capturing and Sharing

Lan Zhang, Kebin Liu+4 more|arXiv (Cornell University)|Oct 24, 2014

Face recognition and analysis8 references3 citations

TL;DR

This paper proposes InvisibleMe, a privacy-preserving protocol enabling automatic portrait removal in photos based on users' explicit privacy preferences. It uses a lightweight, privacy-preserving vector matching scheme to associate privacy requests with individuals in images without exposing facial data, achieving 95.1% accuracy and sub-second matching time in real-world tests.

ABSTRACT

The wide adoption of wearable smart devices with onboard cameras greatly increases people's concern on privacy infringement. Here we explore the possibility of easing persons from photos captured by smart devices according to their privacy protection requirements. To make this work, we need to address two challenges: 1) how to let users explicitly express their privacy protection intention, and 2) how to associate the privacy requirements with persons in captured photos accurately and efficiently. Furthermore, the association process itself should not cause portrait information leakage and should be accomplished in a privacy-preserving way. In this work, we design, develop, and evaluate a protocol, that enables a user to flexibly express her privacy requirement and empowers the photo service provider (or image taker) to exert the privacy protection policy.Leveraging the visual distinguishability of people in the field-of-view and the dimension-order-independent property of vector similarity measurement, we achieves high accuracy and low overhead. We implement a prototype system, and our evaluation results on both the trace-driven and real-life experiments confirm the feasibility and efficiency of our system.

Motivation & Objective

• Address the growing concern of privacy infringement due to ubiquitous smart cameras and wearable devices like smart glasses.
• Enable users to express privacy preferences without visible markers, ensuring aesthetic and practical usability.
• Accurately and efficiently associate privacy requests with individuals in captured photos while preventing portrait information leakage.
• Design a system that operates transparently with minimal overhead on mobile devices and untrusted cloud servers.
• Support both automatic portrait removal and privacy-preserving tagging of individuals in photos.

Proposed method

• Encode users' privacy preferences into visual features embedded in their appearance, enabling invisible expression of intent.
• Use dimension-order-independent vector similarity measurement to match privacy request vectors with image regions without revealing facial details.
• Outsource complex matching computations to untrusted cloud servers using a privacy-preserving protocol that prevents leakage.
• Implement a two-phase protocol: InvisibleMe-Basic for baseline performance and InvisibleMe-Advanced for optimized speed using hashed feature vectors.
• Leverage secure multi-party computation principles to enable non-interactive, private vector distance computation between client and cloud.
• Design a system model that supports three threat models: privacy leakage during transmission, exposure during matching, and unauthorized access to image data.

Experimental results

Research questions

• RQ1 How can users express their portrait privacy preferences in a non-intrusive, invisible, and flexible manner during photo capture?
• RQ2 How can a system accurately and efficiently associate privacy requests with individuals in a captured image without exposing facial data?
• RQ3 What mechanisms ensure that the matching process itself does not leak portrait information to untrusted parties?
• RQ4 How can the system maintain low computational and communication overhead on mobile devices while ensuring privacy?
• RQ5 What is the performance and accuracy trade-off of the proposed privacy-preserving matching protocol in real-world scenarios?

Key findings

• The system achieves a false negative rate of 1.4% and a false positive rate of 0.9% when invisible users are present in photos.
• When no invisible users are present, the false positive rate increases to 4.9% due to lack of true matches, indicating a need for higher threshold tuning.
• Portrait graph transmission delay is negligible at around 0.01–0.04 seconds, and total system computation delay is approximately 4 seconds (3s on client, 1s on cloud).
• Matching cost is only 0.01 seconds in the optimized InvisibleMe-Advanced variant, significantly faster than Paillier-based methods (1.8s per match).
• Power consumption from the protocol is minimal—on the order of seconds—compared to the much higher cost of photo capture itself.
• The protocol supports both automatic portrait removal and privacy-preserving tagging, with verification of removal integrity provided.