[论文解读] iWinRNFL: A Simple, Interpretable & Well-Calibrated In-Game Win Probability Model for NFL

Konstantinos Pelechrinis|arXiv (Cornell University)|Apr 1, 2017

Sports Analytics and Performance参考文献 7被引用 3

一句话总结

本文提出 iWinRNFL，一种用于 NFL 比赛中胜率预测的简单、可解释且校准良好的逻辑回归模型，利用过去 7 个赛季的逐次动作数据中的 10 个比赛状态特征。该模型在预测比赛胜者方面达到 75% 的准确率，显著优于基线赛前预测模型，并在所有置信度水平下保持良好的概率校准，而更复杂的模型（如神经网络或贝叶斯分类器）并未带来有意义的性能提升。

ABSTRACT

During the last few sports seasons a lot of discussion has been generated for the several, high-profile, "comebacks" that were observed in almost all sports. The Cavaliers won the championship after being down 3-1 in the 2016 NBA finals' series against the Golden State Warriors, which was exactly the case for Chicago Cubs and the World Series. The Patriots won the Super Bowl in 2016 even though they were trailing by 25 points late in the third quarter, while FC Barcelona in the top-16 round of the 2016-17 Champions League scored 3 goals during the last 7 minutes of the game (including stoppage time) against PSG to advance in the tournament. This has brought the robustness and accuracy of the various probabilistic prediction models under high scrutiny. Many of these models are proprietary, which makes it hard to evaluate. In this paper, we build a simple and open, yet robust and well-calibrated, in-game probability model for predicting the winner in an NFL (iWinRNFL) game. In particular, we build a logistic regression model that utilizes a set of 10 variables to predict the running win probability for the home team. We train our model using detailed play-by-play data from the last 7 NFL seasons obtained through the league's API. Our results indicate that in 75% of the cases iWinRNFL provides an accurate winner projection, as compared to a 63% accuracy of a baseline pre-game win probability model. Most importantly the probabilities that iWinRNFL provides are well-calibrated. Finally, we have also evaluated more complex, non-linear, models using the same set of features, without any significant improvement in performance.

研究动机与目标

• 开发一种透明、开放且可解释的 NFL 比赛中胜率预测模型，避免对落后球队胜率的过度自信。
• 评估简单广义线性模型是否能在预测 NFL 比赛结果方面达到与复杂非线性模型相当的性能。
• 确保模型预测的概率在整个胜率范围内均实现良好校准，特别是在极端情况（如比赛末段逆转）下表现稳健。
• 通过开放、可社区审查的方法论，为未来胜率预测模型的评估与改进提供基准。

提出的方法

• 该模型采用逻辑回归框架，使用 10 个关键比赛状态变量：剩余时间、分差、控球权、当前回合、达阵距离、场地位置、球队实力、主客场优势以及比赛状态（上半场/下半场、半场结束/比赛结束）。
• 模型训练基于 7 个 NFL 赛季（2009–2015）的详细逐次动作数据，数据通过 NFL 官方 API 获取。
• 模型通过标准指标进行评估：准确率、Brier 评分以及在概率分箱中的校准可靠性。
• 使用相同的 10 个特征训练非线性模型（贝叶斯分类器与前馈神经网络），以与逻辑回归基线模型进行性能对比。
• 通过比较各概率区间内预测概率与实际胜率频率，评估模型的校准性能。
• 案例分析聚焦于超级碗 51，测试模型在极端情况下的行为表现，如新英格兰爱国者队在落后 25 分的情况下实现逆转。

实验结果

研究问题

• RQ1 简单、可解释的逻辑回归模型是否能在 NFL 比赛中胜率预测中实现高准确率与良好校准的概率？
• RQ2 iWinRNFL 在准确率与校准性方面相较于基线赛前胜率预测模型表现如何？
• RQ3 当使用相同输入特征时，更复杂的非线性模型（如神经网络、贝叶斯分类器）是否显著优于简单的逻辑回归模型？
• RQ4 该模型在极端场景（如 2017 年超级碗 51 爱国者队的逆转）下的表现如何？
• RQ5 该模型的校准性能是否在整个概率范围内均稳健，特别是在低概率事件（如 1/1000 的机会）下？

主要发现

• iWinRNFL 在预测正确比赛胜者方面达到 75% 的准确率，显著高于基线赛前胜率模型的 63% 准确率。
• 该模型在所有概率分箱中均表现出良好的概率校准，即被赋予 20% 胜率的球队实际获胜频率约为 20%。
• 即使在极端情况（如 2017 年超级碗 51 爱国者队逆转）下，iWinRNFL 在其最低点仍仅给予爱国者队 2.1% 的胜率，避免了某些专有模型中常见的过度自信问题。
• 尽管在节末存在非线性动态的证据，但更复杂的模型（如前馈神经网络，FNN）在 Brier 评分与准确率方面仅带来微小提升，尤其在比赛后期阶段。
• 随着比赛推进，模型性能持续提升，至比赛末期准确率已接近当前最先进水平，这与比赛中动态信息相较于球队实力评级的重要性逐渐增加的趋势一致。
• 研究结论认为，简单且可解释的模型（如 iWinRNFL）已足以实现稳健的比赛中胜率预测，无需采用非线性模型的复杂性即可获得有意义的性能提升。