[Paper Review] A Large-Scale Study of Personal Identifiability of Virtual Reality Motion Over Time

Mark Roman Miller, Eugy Han+4 more|arXiv (Cornell University)|Mar 2, 2023

Virtual Reality Applications and Impacts11 citations

TL;DR

This study analyzes identifiability of VR motion data over time using 232 participants across eight weekly sessions, showing that more sessions and longer per-session durations increase identifiability, while longer training-testing delays reduce it; it also introduces body-space coordinates and evaluates multiclass AUC.

ABSTRACT

In recent years, social virtual reality (VR), sometimes described as the "metaverse," has become widely available. With its potential comes risks, including risks to privacy. To understand these risks, we study the identifiability of participants' motion in VR in a dataset of 232 VR users with eight weekly sessions of about thirty minutes each, totaling 764 hours of social interaction. The sample is unique as we are able to study the effect of user, session, and time independently. We find that the number of sessions recorded greatly increases identifiability, and duration per session increases identifiability as well, but to a lesser degree. We also find that greater delay between training and testing sessions reduces identifiability. Ultimately, understanding the identifiability of VR activities will help designers, security professionals, and consumer advocates make VR safer.

Motivation & Objective

• Assess how identifiability of VR motion data evolves over time and with varying data sizes.
• Quantify how the number of sessions, per-session duration, and time delay between training and testing affect identifiability.
• Introduce body-space coordinates to improve feature alignment across users.
• Demonstrate demographic inferences (e.g., gender and ethnicity) from motion data.
• Establish evaluation metrics (multiclass AUC) to enable comparisons across datasets and class sizes.

Proposed method

• Utilize the Stanford Longitudinal VR Classroom Dataset with 232 participants across two collection periods and eight weekly sessions of ~30 minutes each.
• Record position and rotation of headset and hand controllers during social VR discussions using the ENGAGE platform.
• Engineer 840 features from 42 streams (positions, rotations, and derived relative motions) including body-space coordinates.
• Define a body-space coordinate system relative to each participant’s forward direction to improve invariance to horizontal plane rotations.
• Train a random forest classifier (R/ranger) with an ensemble of 600 trees and aggregate predictions per session for multiclass identification.
• Evaluate using multiclass AUC as the primary metric, supplemented by accuracy for fixed N-class testing sets.

Figure 1: Parallel coordinates plot of classification size, span of time in which data was collected, and total duration of data collected per participant. The current work is the largest or the second-largest on all dimensions. Note all dimensions are log-scaled in order to better scale the variati

Experimental results

Research questions

• RQ1 How does identifiability of VR motion data change with more training data (more sessions) versus longer single sessions?
• RQ2 How does the time delay between training and testing data affect identifiability across weeks?
• RQ3 Does transforming data into body-space coordinates improve identifiability compared to global coordinates?
• RQ4 Can demographic attributes such as gender and ethnicity be inferred from VR motion data, and with what accuracy?

Key findings

• Identifiability increases substantially with more sessions recorded and, to a lesser extent, with longer session duration.
• Delay between training and testing data reduces identifiability across the studied timescales.
• Identifiability is higher within a session than between different sessions, consistent with prior work.
• The study advocates multiclass AUC as a robust metric for comparing identifiability across datasets with varying class sizes.
• Body-space coordinates help align motion features to a user’s forward direction, improving the identification feature set.
• Demographic attributes (gender and ethnicity) can be inferred from motion data with small to medium gains over baseline models.

Figure 2: Participants performing discussion activities in the VR environment. In the top left panel, participants illustrate the environmental impacts of an oil spill with a duck model covered in black smudges representing oil. In the top right panel, several students discuss the experience of the