[论文解读] Learning Personalized Models of Human Behavior in Chess

Reid McIlroy-Young, Russell Wang+3 more|arXiv (Cornell University)|Aug 23, 2020

Sports Analytics and Performance参考文献 58被引用 9

一句话总结

本文通过在单个玩家的走法序列上微调预训练的AlphaZero智能体，提出了一套个性化的人类国际象棋行为模型。通过迭代微调将模型适配至特定玩家，系统在预测个体人类走法方面实现了高准确率，并实现了精确的风格分析，表明个性化人工智能能够捕捉独特的个体决策模式。

ABSTRACT

Even when machine learning systems surpass human ability in a domain, there are many reasons why AI systems that capture human-like behavior would be desirable: humans may want to learn from them, they may need to collaborate with them, or they may expect them to serve as partners in an extended interaction. Motivated by this goal of human-like AI systems, the problem of predicting human actions -- as opposed to predicting optimal actions -- has become an increasingly useful task. We extend this line of work by developing highly accurate personalized models of human behavior in the context of chess. Chess is a rich domain for exploring these questions, since it combines a set of appealing features: AI systems have achieved superhuman performance but still interact closely with human chess players both as opponents and preparation tools, and there is an enormous amount of recorded data on individual players. Starting with an open-source version of AlphaZero trained on a population of human players, we demonstrate that we can significantly improve prediction of a particular player's moves by applying a series of fine-tuning adjustments. Furthermore, we can accurately perform stylometry -- predicting who made a given set of actions -- indicating that our personalized models capture human decision-making at an individual level.

研究动机与目标

• 开发能够预测人类国际象棋走法而非最优走法的AI模型，以促进人机协作与学习。
• 通过利用玩家历史走法数据对通用AI智能体进行个性化，提升对个体玩家的预测准确率。
• 探究个性化模型是否可通过捕捉个体对弈风格，实现风格分析——即识别走法序列的作者。
• 探索如何对超人类AI模型（AlphaZero）进行微调，使其在个体层面上表现出类人行为。

提出的方法

• 在特定人类玩家的走法序列数据集上微调预训练的AlphaZero智能体，以使其策略网络适配该玩家的行为。
• 利用个体玩家的走法历史进行监督学习，调整模型的策略网络，使其从通用最优对弈转向个性化决策。
• 对单个玩家的批量数据应用迭代微调，逐步使模型行为与观察到的人类选择保持一致。
• 在来自同一玩家的保留走法序列上评估个性化模型，以衡量预测准确率。
• 利用微调后的模型执行风格分析，通过分类判断哪位玩家最可能是给定走法序列的作者。
• 利用国际象棋这一具有丰富人类走法数据和超人类AI基准的领域结构，验证该方法的有效性。

实验结果

研究问题

• RQ1 对超人类AI模型（如AlphaZero）进行微调，能否在国际象棋中准确预测个体人类玩家的走法？
• RQ2 个性化模型在多大程度上能捕捉独特的对弈风格，从而实现对走法序列的准确风格分析？
• RQ3 与基础模型相比，基于个体玩家数据的微调在多大程度上提升了预测性能？
• RQ4 个性化模型是否能很好地泛化到同一玩家的未见走法序列上？

主要发现

• 与基础AlphaZero模型相比，微调显著提升了对个体玩家走法的预测准确率。
• 个性化模型在风格分析中表现出高准确率，能强有力地正确识别走法序列的作者。
• 模型成功捕捉了个体层面上独特的决策模式，表明个性化成功实现。
• 该方法表明，超人类AI模型可被有效调整，以高保真度模仿人类行为。