[Paper Review] A Unified Approach to Dynamic Decision Problems with Asymmetric Information - Part I: Non-Strategic Agents

Hamidreza Tavafoghi, Yi Ouyang+1 more|arXiv (Cornell University)|Dec 3, 2018

Auction Theory and Applications51 references4 citations

TL;DR

This paper introduces a unified framework for dynamic multi-agent decision problems with asymmetric information and non-strategic agents by proposing sufficient information-based belief (SIB) states that compress private and common information in a time-invariant, mutually consistent manner. The key contribution is a sequential decomposition enabling backward induction via a dynamic program, which yields globally optimal strategies in dynamic teams without loss of optimality.

ABSTRACT

We study a general class of dynamic multi-agent decision problems with asymmetric information and non-strategic agents, which includes dynamic teams as a special case. When agents are non-strategic, an agent's strategy is known to the other agents. Nevertheless, the agents' strategy choices and beliefs are interdependent over times, a phenomenon known as signaling. We introduce the notions of private information that effectively compresses the agents' information in a mutually consistent manner. Based on the notions of sufficient information, we propose an information state for each agent that is sufficient for decision making purposes. We present instances of dynamic multi-agent decision problems where we can determine an information state with a time-invariant domain for each agent. Furthermore, we present a generalization of the policy-independence property of belief in Partially Observed Markov Decision Processes (POMDP) to dynamic multi-agent decision problems. Within the context of dynamic teams with asymmetric information, the proposed set of information states leads to a sequential decomposition that decouples the interdependence between the agents' strategies and beliefs over time, and enables us to formulate a dynamic program to determine a globally optimal policy via backward induction.

Motivation & Objective

• To address the challenge of interdependent strategies and beliefs in dynamic multi-agent systems with asymmetric information and non-strategic agents.
• To develop a general method for compressing agents' private and common information into time-invariant, sufficient information states that preserve decision-making optimality.
• To establish a sequential decomposition that decouples strategy and belief interdependence over time, enabling backward induction.
• To generalize the policy-independence property of belief in POMDPs to multi-agent settings with asymmetric information.
• To formulate a dynamic program for determining globally optimal strategy profiles in dynamic teams with asymmetric information.

Proposed method

• Introduce the notion of sufficient private information (SPI) that compresses agents' private and common information in a mutually consistent way over time.
• Define sufficient information-based belief (SIB) states as the belief over the system state and SPI, which are time-invariant under certain conditions.
• Propose a sequential decomposition that decouples the interdependence between agents' strategies and beliefs, enabling backward induction.
• Formulate a dynamic program using the SIB belief and a time-invariant update rule via Bayes' rule to recursively compute beliefs.
• Generalize the policy-independence belief property from POMDPs to dynamic teams with asymmetric information.
• Establish a Bellman equation for infinite-horizon dynamic teams using stationary SIB strategies and a time-invariant update rule.

Experimental results

Research questions

• RQ1 Can we compress the private and common information of non-strategic agents in dynamic multi-agent systems into a time-invariant, sufficient information state without loss of optimality?
• RQ2 How can we decouple the interdependence between agents' strategies and beliefs over time in dynamic teams with asymmetric information?
• RQ3 Does the policy-independence property of belief in POMDPs extend to multi-agent systems with asymmetric information and non-strategic agents?
• RQ4 Can we formulate a dynamic program for globally optimal strategy selection in dynamic teams with asymmetric information using SIB states?
• RQ5 What conditions allow for the existence of time-invariant sufficient private information in dynamic decision problems?

Key findings

• The proposed SIB belief state is sufficient for decision making and enables a sequential decomposition that decouples strategy and belief interdependence over time.
• Restricting to SIB-based strategies entails no loss of optimality in dynamic decision problems with non-strategic agents.
• For specific instances, the sufficient private information has a time-invariant domain, enabling the use of stationary strategies.
• A dynamic program is formulated using the SIB belief and a time-invariant update rule, allowing backward induction to compute globally optimal strategies.
• The policy-independence property of belief in POMDPs is generalized to dynamic teams with asymmetric information.
• A Bellman equation is derived for infinite-horizon dynamic teams, generalizing the POMDP formulation to multi-agent settings with asymmetric information.