[论文解读] Inferring Private Personal Attributes of Virtual Reality Users from Head and Hand Motion Data

Vivek Nair, Christian Räck+9 more|arXiv (Cornell University)|May 30, 2023

Sexuality, Behavior, and Technology被引用 9

一句话总结

本文显示仅使用基于 Transformer 的模型，通过VR头部与手部运动数据，就能从 Beat Saber 等友好游戏中推断出超过 40 项私人个人属性，揭示多用户 VR 场景中的隐私风险。

ABSTRACT

Motion tracking "telemetry" data lies at the core of nearly all modern virtual reality (VR) and metaverse experiences. While generally presumed innocuous, recent studies have demonstrated that motion data actually has the potential to uniquely identify VR users. In this study, we go a step further, showing that a variety of private user information can be inferred just by analyzing motion data recorded from VR devices. We conducted a large-scale survey of VR users (N=1,006) with dozens of questions ranging from background and demographics to behavioral patterns and health information. We then obtained VR motion samples of each user playing the game "Beat Saber," and attempted to infer their survey responses using just their head and hand motion patterns. Using simple machine learning models, over 40 personal attributes could be accurately and consistently inferred from VR motion data alone. Despite this significant observed leakage, there remains limited awareness of the privacy implications of VR motion data, highlighting the pressing need for privacy-preserving mechanisms in multi-user VR applications.

研究动机与目标

• 证明在非对抗性游戏环境中，可以从 VR 头部和手部运动数据推断出私人用户属性。
• 仅通过运动遥测量化哪些属性在统计显著性上可被推断。
• 提供一个可泛化的机器学习框架，用于从顺序的 VR 运动数据推断二元属性。

提出的方法

• 在 1,006 名 Beat Saber 玩家中收集运动遥测数据（每帧 21 个特征，来自头部和双手），并附带详细问卷。
• 将 50 个选定属性转化为二分类任务（例如年龄组、婚姻状况等）。
• 在 21×1024 帧序列上训练基于 Transformer 的模型，以预测二元属性，使用每类均衡数据集及 Monte Carlo 交叉验证。
• 通过按序列和按用户的准确率以及二项检验的统计显著性（p<0.01, p<0.05）评估推断成功率。
• 使用 Adam 优化和 BCE 损失进行 100 轮训练；报告跨属性的聚合结果。

实验结果

研究问题

• RQ1 在非对抗性 VR 游戏中，是否可以仅从头部和手部运动数据推断出 VR 用户的私人属性？
• RQ2 在 Beat Saber 中，哪些属性可以通过运动遥测达到统计显著的推断？
• RQ3 基于 Transformer 的模型在这类顺序运动数据推断任务中的效果如何？

主要发现

属性	总数	测试数量	准确率	显著性	总数	测试数量	准确率	显著性
StandaloneGrip	31,100	6,000	85.9%	<0.001	311	60	91.7%	<0.001
Height	19,100	6,000	76.5%	<0.001	191	60	86.7%	<0.001
Controller	33,200	6,000	81.2%	<0.001	332	60	85.0%	<0.001
Weight	9,800	6,000	73.6%	<0.001	98	60	85.0%	<0.001
FootSize	9,100	6,000	73.2%	<0.001	91	60	85.0%	<0.001
Country	33,300	6,000	60.3%	<0.001	333	60	81.7%	<0.001
RhythmGames	10,900	6,000	63.5%	<0.001	109	60	80.0%	<0.001
Age	62,300	6,000	64.9%	<0.001	623	60	78.3%	<0.001
TotalPlayTime	34,400	6,000	67.7%	<0.001	344	60	78.3%	<0.001
Headset	65,000	6,000	66.9%	<0.001	650	60	76.7%	<0.001
LeftArm	10,300	6,000	65.2%	<0.001	103	60	76.7%	<0.001
RightArm	10,200	6,000	64.9%	<0.001	102	60	75.0%	<0.001
Athletics	8,700	6,000	59.1%	<0.001	87	60	75.0%	<0.001
MaritalStatus	81,400	6,000	60.2%	<0.001	814	60	73.3%	<0.001
EmploymentStatus	64,200	6,000	65.1%	<0.001	642	60	71.7%	<0.001
AnyRhythmGames	83,000	6,000	54.8%	<0.001	830	60	70.0%	<0.001
Ethnicity	73,900	6,000	59.7%	<0.001	739	60	70.0%	<0.001
SteamComputerFormFactor	51,300	6,000	58.5%	<0.001	513	60	70.0%	<0.001
Footwear	36,700	6,000	60.5%	<0.001	367	60	70.0%	<0.001
AnyVRRhythmGames	83,000	8,000	56.8%	<0.001	830	80	68.8%	<0.001
Income	76,700	8,000	55.0%	<0.001	767	80	68.8%	<0.001
Wingspan	16,000	8,000	59.9%	<0.001	160	80	68.8%	<0.001
Handedness	71,600	10,000	55.2%	<0.001	716	100	66.0%	<0.001
HandLength	51,000	8,000	58.5%	<0.001	510	80	66.3%	<0.001
SubstanceUse	69,200	10,000	55.9%	<0.001	692	100	64.0%	<0.001
Preparation	39,400	8,000	58.2%	<0.001	394	80	65.0%	<0.001
LowerBody	29,500	8,000	55.9%	<0.001	295	80	65.0%	<0.001
Lenses	80,900	8,000	55.3%	<0.001	809	80	65.0%	<0.001
Languages	80,700	8,000	56.5%	<0.001	807	80	65.0%	<0.001
SteamOperatingSystemVersion	50,800	8,000	58.4%	<0.001	508	80	65.0%	<0.001
Music	29,600	8,000	53.6%	<0.001	296	80	65.0%	<0.001
AnyMentalDisabilities	83,000	10,000	52.6%	<0.001	830	100	63.0%	<0.001
Sex	76,300	10,000	56.5%	<0.001	763	100	63.0%	<0.001
AnyPhysicalDisabilities	83,000	10,000	54.5%	<0.001	830	100	62.0%	<0.001
ReactionTime	9,800	14,000	53.1%	<0.001	98	140	60.0%	<0.001
AnyMusic	83,000	8,000	55.7%	<0.001	830	80	62.5%	<0.001
AnyAthletics	19,900	8,000	55.7%	<0.001	199	80	61.3%	<0.001
EducationalStatus	62,200	8,000	57.1%	<0.001	622	80	60.0%	<0.001
IPD	6,700	8,000	55.8%	<0.001	67	80	60.0%	<0.001
Dance	82,000	10,000	52.3%	<0.001	820	100	59.0%	<0.001
PoliticalOrientation	33,100	10,000	53.5%	<0.001	331	100	58.0%	<0.001
UpperBody	47,200	10,000	52.0%	<0.001	472	100	57.0%	<0.001
SteamProcessorLogicalCores	33,500	10,000	51.0%	<0.001	335	100	56.0%	<0.001
HadCOVID	83,000	10,000	54.4%	<0.001	830	100	55.0%	<0.001
CaffinatedBeverages	40,800	10,000	52.9%	<0.001	408	100	55.0%	<0.001
RoomArea	33,100	8,000	50.5%	p=0.183	331	80	56.3%	p=0.157
PhysicalFitness	7,800	12,000	54.2%	<0.001	78	120	55.0%	p=0.158
SteamProcessorCPUVendor	51,600	10,000	49.2%	p=0.953	516	100	53.0%	p=0.309
SteamLighthouses	5,500	8,000	48.6%	p=0.993	55	80	52.5%	p=0.369
ColorBlindness	79,800	10,000	50.4%	p=0.227	798	100	52.0%	p=382

• 在 50 项属性中，按用户统计显著性高（p<0.01）预测成功的有 33 项，具有中等显著性（p<0.05）的是 8 项。
• 在按序列的层面上，50 项属性中有 45 项具有高度显著性（p<0.01），1 项具有中度显著性（p<0.05）。
• 对虚构输入进行的宏观显著性检验在按序列和按用户评估中均得到 p<0.0001，表明总体显著性很强。
• 结果在真实世界的多样硬件/软件生态中成立，仅使用头部/手部运动数据，且采用最弱的对手模型。