[论文解读] Leaderboard Effects on Player Performance in a Citizen Science Game

Mads Kock Pedersen, Nanna Ravn Rasmussen+2 more|arXiv (Cornell University)|Jul 12, 2017

Educational Games and GamificationPsychology参考文献 3被引用 17

一句话总结

本研究调查了不同排行榜设计——无排行榜、历史前五名排行榜以及个性化无限排行榜——对公民科学游戏《Quantum Moves》中玩家表现的影响。基于高流量发布阶段超过4,500名玩家的数据，研究发现，移除排行榜或使用个性化排名并未降低表现，有时甚至优于标准的前五名排行榜，挑战了排行榜在游戏化系统中普遍提升参与度与表现的假设。

ABSTRACT

Quantum Moves is a citizen science game that investigates the ability of humans to solve complex physics challenges that are intractable for computers. During the launch of Quantum Moves in April 2016 the game's leaderboard function broke down resulting in a "no leaderboard" game experience for some players for a couple of days (though their scores were still displayed). The subsequent quick fix of an all-time Top 5 leaderboard, and the following long-term implementation of a personalized relative-position (infinite) leaderboard provided us with a unique opportunity to compare and investigate the effect of different leaderboard implementations on player performance in a points-driven citizen science game. All three conditions were live sequentially during the game's initial influx of more than 150.000 players that stemmed from global press attention on Quantum Moves due the publication of a Nature paper about the use of Quantum Moves in solving a specific quantum physics problem. Thus, it has been possible to compare the three conditions and their influence on the performance (defined as a player's quality of game play related to a high-score) of over 4500 new players. These 4500 odd players in our three leaderboard-conditions have a similar demographic background based upon the time-window over which the implementations occurred and controlled against Player ID tags. Our results placed Condition 1 experience over condition 3 and in some cases even over condition 2 which goes against the general assumption that leaderboards enhance gameplay and its subsequent overuse as a an oft-relied upon element that designers slap onto a game to enhance said appeal. Our study thus questions the use of leaderboards as general performance enhancers in gamification contexts and brings some empirical rigor to an often under-reported but overused phenomenon.

研究动机与目标

• 调查不同排行榜实现方式对公民科学游戏中玩家表现的影响。
• 评估排行榜是否能提升游戏化科学任务中的游戏质量和持续参与度。
• 评估排行榜设计在超过150,000名玩家参与的高流量发布阶段对表现的影响。
• 提供关于排行榜在游戏化中有效性的实证证据，反驳其能增强表现的普遍假设。
• 比较三种不同的排行榜条件：无排行榜、历史前五名排行榜以及个性化相对名次排行榜。

提出的方法

• 本研究利用《Quantum Moves》发布期间的自然实验，当时排行榜功能临时失效，形成了‘无排行榜’条件。
• 研究人员比较了三种连续的排行榜条件：(1) 无排行榜，(2) 历史前五名排行榜，以及(3) 个性化无限排行榜。
• 表现通过玩家得分衡量，反映其在解决量子物理挑战中的游戏质量。
• 分析通过玩家ID标签控制了玩家人口统计特征和进入时间，以确保各条件下的群体可比性。
• 本研究使用了超过4,500名新玩家在三个不同排行榜阶段加入的大规模数据集。
• 对三种条件之间的表现结果进行了统计比较。

实验结果

研究问题

• RQ1 在公民科学游戏中，无排行榜是否会对玩家表现产生负面影响？
• RQ2 标准化的前五名排行榜与个性化无限排行榜相比，对游戏质量的影响有何差异？
• RQ3 排行榜的存在是否如游戏化设计中普遍假设的那样，始终能提升玩家表现？
• RQ4 不同排行榜设计在高流量游戏发布期间对参与度和表现的影响程度如何？
• RQ5 个性化或无排行榜系统是否能维持甚至优于传统竞争性排行榜的表现？

主要发现

• '无排行榜'条件下的平均玩家表现高于'历史前五名排行榜'条件。
• 在某些情况下，'无排行榜'条件的表现甚至优于个性化无限排行榜，与预期中排行榜能提升表现的假设相矛盾。
• 与'无排行榜'条件相比，个性化无限排行榜并未显著提升表现。
• 当排行榜缺失时，玩家表现保持稳定甚至有所提升，表明竞争性排名可能并非高质量游戏表现的必要条件。
• 研究结果挑战了排行榜作为游戏化默认工具的广泛使用，表明其未必能普遍增强表现。
• 本研究提供了实证证据，表明排行榜并非保证能提升表现的工具，甚至在某些情境下可能适得其反。