[Paper Review] The Prescription Approach to Decentralized Stochastic Control with Word-of-Mouth Communication

Aditya Dave, Andreas A. Malikopoulos|arXiv (Cornell University)|Jul 28, 2019

Distributed Control Multi-Agent Systems4 citations

TL;DR

This paper introduces the prescription approach to solve decentralized stochastic control problems with word-of-mouth communication, where agents communicate with delays through a network. It derives structural results for optimal control strategies in time-invariant spaces and proves that optimal prescriptions depend only on local information states and complete prescriptions, enabling globally optimal solutions under non-classical information structures.

ABSTRACT

In this paper we analyze a network of agents that communicates through word of mouth. In a word-of-mouth communication system, every agent communicates with its neighbors with delays in communication. This is a non-classical information structure where the topological and temporal restrictions in communication mean that information propagates slowly through the network. We present the prescription approach to derive structural results for such problems. The structural results lead to optimal control strategies with time invariant domain-sizes. We show that these domains are smaller in size than the control strategies derived using the common information approach.

Motivation & Objective

• To address decentralized stochastic control problems where agents communicate via word-of-mouth with delays, leading to non-classical information structures.
• To develop a structural characterization of optimal control strategies in time-invariant spaces under such communication constraints.
• To provide a solution framework that avoids the computational intractability of traditional approaches like the designer’s approach.
• To compare the prescription approach with the common information approach, demonstrating its advantages in deriving globally optimal strategies.

Proposed method

• The prescription approach models agent communication as a network with delayed, neighbor-to-neighbor information exchange, capturing word-of-mouth dynamics.
• It introduces the concept of a 'complete prescription' that encapsulates all future control actions based on current and past information states.
• The method uses dynamic programming on the joint information state of agents, deriving cost-to-go functions that depend on local information states and complete prescriptions.
• It proves that the optimal prescription for each agent depends only on its own information state and the complete prescriptions of other agents, enabling time-invariant strategies.
• The approach leverages conditional expectations and probability distributions of primitive random variables to simplify the cost-to-go expressions.
• A mathematical induction procedure is used to derive the structural result for the optimal prescription strategy across all agents.

Experimental results

Research questions

• RQ1 How can optimal control strategies be derived in decentralized systems with word-of-mouth communication and delayed information exchange?
• RQ2 What structural properties characterize the optimal control strategy under non-classical information structures arising from delayed, neighbor-based communication?
• RQ3 Can the prescription approach yield globally optimal solutions where the person-by-person approach fails?
• RQ4 How does the prescription approach compare to the common information approach in terms of structural simplicity and computational feasibility?
• RQ5 Under what conditions does the optimal prescription depend only on local information states and complete prescriptions of other agents?

Key findings

• The optimal prescription strategy for each agent depends only on its own information state and the complete prescriptions of other agents, enabling time-invariant control laws.
• The cost-to-go function for each agent decomposes into a local cost and an expectation over joint system states, which is tractable due to the structure of the prescription.
• The prescription approach yields globally optimal solutions under non-classical information structures, unlike the person-by-person approach which may only yield locally optimal solutions.
• The method avoids the high computational complexity of the designer’s approach by focusing on structural properties of the optimal strategy.
• The structural result is proven via mathematical induction, showing that the optimal prescription for agent $k$ at time $t$ is a function of the information states $\Pi_t^k, \dots, \Pi_t^K$ and the complete prescriptions $\Theta_t^k, \dots, \Theta_t^K$.
• The approach enables the derivation of optimal control strategies even when agents have asymmetric and delayed access to information, as in drone swarms or connected vehicle networks.