[Paper Review] A Game-Theoretic Utility Network for Cooperative Multi-Agent Decisions in Adversarial Environments

Qin Yang, Ramviyas Parasuraman|arXiv (Cornell University)|Apr 23, 2020

Infrastructure Resilience and Vulnerability Analysis16 references4 citations

TL;DR

This paper proposes the Game-theoretic Utility Tree (GUT), a hierarchical network model that decomposes high-level cooperative strategies into executable low-level actions for multi-agent systems in adversarial environments. By integrating game theory, Bayesian networks, and a needs-based utility function, GUT outperforms QMIX in simulations, achieving higher winning rates and lower system costs through improved cooperation modeling and predictive modeling under incomplete information.

ABSTRACT

Underlying relationships among multi-agent systems (MAS) in hazardous scenarios can be represented as Game-theoretic models. We measure the performance of MAS achieving tasks from the perspective of balancing success probability and system costs. This paper proposes a new network-based model called Game-theoretic Utility Tree (GUT), which decomposes high-level strategies into executable low-level actions for cooperative MAS decisions. This is combined with a new payoff measure based on agent needs for real-time strategy games. We present an Explore game domain to evaluate GUT against the state-of-the-art QMIX decision-making method. Conclusive results on extensive numerical simulations indicate that GUT can organize more complex relationships among MAS cooperation, helping the group achieve challenging tasks with lower costs and a higher winning rate.

Motivation & Objective

• To address the challenge of designing cooperative multi-agent systems that balance task success probability and system cost in adversarial environments.
• To model complex agent relationships and hierarchical decision-making under uncertainty, especially with intentional and unintentional adversaries.
• To develop a novel utility computation framework based on agent needs, inspired by Maslow's hierarchy, to guide cooperative strategy selection.
• To evaluate the GUT framework against state-of-the-art methods like QMIX in a realistic game-theoretic simulation environment.
• To investigate the impact of predictive modeling under incomplete information on system performance and decision robustness.

Proposed method

• GUT is structured as a hierarchical tree of Game-theoretic Utility Computation Units, decomposing high-level strategies into low-level executable actions to reduce strategy space complexity.
• The utility function is computed using a needs-based hierarchy, where agent utility is derived from expectations of survival, energy, and mission success, analogous to human needs pyramids.
• A novel payoff measure integrates agent-specific needs and system-level costs, enabling balanced evaluation of cooperation efficiency and success probability.
• The approach combines principles from Bayesian Networks for probabilistic inference, Game Theory for equilibrium reasoning, and Utility Theory for value-based decision-making.
• Two predictive models—linear and polynomial regression—are used to estimate adversary states (e.g., energy levels) from indirect observations in incomplete information scenarios.
• The system is evaluated in a custom 'Explore' game domain simulating explorers versus aliens, with controlled adversarial dynamics and environmental obstacles.

Experimental results

Research questions

• RQ1 Can a hierarchical game-theoretic utility network improve cooperation among multi-agent systems in adversarial environments compared to existing methods?
• RQ2 How does modeling agent needs as a utility hierarchy affect system-level performance and decision robustness?
• RQ3 To what extent can predictive models based on observable costs (e.g., HP, energy) improve decision-making under incomplete information?
• RQ4 How do intentional and unintentional adversaries jointly impact system cost and success probability in cooperative multi-agent tasks?
• RQ5 Does GUT enable more complex and efficient cooperation patterns than QMIX in challenging, dynamic scenarios?

Key findings

• GUT achieved a higher winning rate than QMIX in the Explore domain, particularly under incomplete information, demonstrating superior strategic coordination.
• The system cost—measured as average HP and energy consumption per round—was significantly lower with GUT compared to QMIX, indicating improved efficiency.
• Linear regression models outperformed polynomial models in predicting adversary energy levels under incomplete information, showing better alignment with ground truth in individual performance metrics.
• The inclusion of unintentional adversaries (e.g., mountains) reduced the winning rate and increased system costs, but GUT maintained better performance than QMIX under these conditions.
• Predictive model parameters required adaptation to scenario context, highlighting the need for experience-based learning to refine utility and prediction functions.
• GUT enabled more complex cooperation relationships among agents, as evidenced by improved group-level outcomes and lower individual and system costs.