[论文解读] Qualitative Analysis of Concurrent Mean-payoff Games

Krishnendu Chatterjee, Rasmus Ibsen-Jensen|arXiv (Cornell University)|Jan 1, 2013

Logic, Reasoning, and Knowledge参考文献 40被引用 1

一句话总结

本文对并发均值支付博弈进行了定性分析，建立了定性确定性、最优策略复杂度，并提出了计算几乎必然获胜集和正获胜集的二次时间算法。研究表明，若能解决此类博弈中的定量约束，将同时解决长期悬而未决的难题——即在多项式时间内求解回合制确定性均值支付博弈。

ABSTRACT

We consider concurrent games played by two-players on a finite-state graph, where in every round the players simultaneously choose a move, and the current state along with the joint moves determine the successor state. We study a fundamental objective, namely, mean-payoff objective, where a reward is associated to each transition, and the goal of player 1 is to maximize the long-run average of the rewards, and the objective of player 2 is strictly the opposite. The path constraint for player 1 could be qualitative, i.e., the mean-payoff is the maximal reward, or arbitrarily close to it; or quantitative, i.e., a given threshold between the minimal and maximal reward. We consider the computation of the almost-sure (resp. positive) winning sets, where player 1 can ensure that the path constraint is satisfied with probability 1 (resp. positive probability). Our main results for qualitative path constraints are as follows: (1) we establish qualitative determinacy results that show that for every state either player 1 has a strategy to ensure almost-sure (resp. positive) winning against all player-2 strategies, or player 2 has a spoiling strategy to falsify almost-sure (resp. positive) winning against all player-1 strategies; (2) we present optimal strategy complexity results that precisely characterize the classes of strategies required for almost-sure and positive winning for both players; and (3) we present quadratic time algorithms to compute the almost-sure and the positive winning sets, matching the best known bound of algorithms for much simpler problems (such as reachability objectives). For quantitative constraints we show that a polynomial time solution for the almost-sure or the positive winning set would imply a solution to a long-standing open problem (the value problem for turn-based deterministic mean-payoff games) that is not known to be solvable in polynomial time.

研究动机与目标

• 建立并发均值支付博弈在几乎必然获胜和正获胜条件下的定性确定性。
• 精确刻画双方玩家在几乎必然获胜和正获胜策略下所需的策略复杂度。
• 开发高效算法以计算几乎必然获胜集和正获胜集，其时间复杂度与可达性问题中已知的最佳界限一致。
• 研究并发均值支付博弈中定量路径约束的计算难度。
• 通过缩放与平移技术，将布尔奖励结果推广至有理数值奖励函数。

提出的方法

• 通过基于部件的构造，将并发均值支付博弈（DMPGs）转化为回合制随机博弈，以每条原始转移模拟3M步。
• 利用马尔可夫链的性质分析简化后博弈中的长期平均奖励，特别关注封闭的常返集与期望平均奖励。
• 在简化后的回合制随机博弈中使用位置策略，以推导原始并发博弈中的策略。
• 应用马尔可夫链的基本性质，将简化后博弈中的平均奖励与原始博弈的循环行为关联起来。
• 通过奖励缩放与阈值转换，证明原始博弈与简化博弈中获胜条件的等价性。
• 证明：若能在并发博弈中求解定量获胜集，则可推出回合制确定性均值支付博弈在多项式时间内可解的结论。

实验结果

研究问题

• RQ1 在几乎必然获胜和正获胜条件下，并发均值支付博弈是否具有定性确定性？
• RQ2 并发均值支付博弈中，几乎必然获胜和正获胜策略所需的精确策略复杂度为何？
• RQ3 几乎必然获胜集和正获胜集是否可在二次时间内计算，其复杂度与可达性博弈中已知的最佳界限一致？
• RQ4 求解并发均值支付博弈中的定量路径约束，是否在计算上等价于求解回合制确定性均值支付博弈的值问题？
• RQ5 如何通过缩放与平移技术，将有理数值奖励函数约化为布尔奖励，同时保持定性获胜条件？

主要发现

• 定性确定性成立：对于每个状态，要么玩家1存在一种策略，可确保对所有玩家2策略均实现几乎必然或正获胜；要么玩家2存在一种破坏策略。
• 几乎必然获胜集与正获胜集可在二次时间内计算，其时间复杂度与并发博弈中可达性目标的最佳已知复杂度一致。
• 双方玩家的策略复杂度被精确刻画：在定性约束下，位置策略足以实现几乎必然获胜与正获胜。
• 从并发DMPGs到回合制随机博弈的约化通过3M步模拟保留了获胜条件，从而可借助马尔可夫链性质进行分析。
• 若能求解并发均值支付博弈中的定量获胜集，则可推出长期悬而未决的难题——即在多项式时间内求解回合制确定性均值支付博弈——的解决方案。
• 通过缩放与平移技术，研究结果可从布尔奖励推广至有理数值奖励，同时保持最大奖励目标下的定性获胜条件。