[Paper Review] Cleaning up the neighborhood: A full classification for adversarial partial monitoring

Tor Lattimore, Csaba Szepesvári|arXiv (Cornell University)|May 23, 2018

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper completes the classification of finite adversarial partial monitoring games by introducing a refined algorithm, NeighbourhoodWatch2, that correctly handles degenerate games—previously unresolved cases where actions are optimal only on low-dimensional subspaces. The authors achieve a tight regret bound of $O(F\sqrt{nK_{\text{loc}}\log K})$, improving upon prior work by eliminating dependence on arbitrary constants and simplifying analysis while proving tight lower bounds.

ABSTRACT

Partial monitoring is a generalization of the well-known multi-armed bandit framework where the loss is not directly observed by the learner. We complete the classification of finite adversarial partial monitoring to include all games, solving an open problem posed by Bartok et al. [2014]. Along the way we simplify and improve existing algorithms and correct errors in previous analyses. Our second contribution is a new algorithm for the class of games studied by Bartok [2013] where we prove upper and lower regret bounds that shed more light on the dependence of the regret on the game structure.

Motivation & Objective

• To resolve an open problem in partial monitoring theory by completing the classification of finite adversarial partial monitoring games.
• To address the long-standing issue of degenerate games—actions optimal only on low-dimensional subspaces—previously unhandled by existing algorithms.
• To simplify and correct prior analyses of NeighbourhoodWatch and Bartók’s algorithm, fixing errors in expectation vs. high-probability bounds.
• To establish tight regret bounds that clarify the dependence of regret on game structure, particularly feedback dimension and local action geometry.
• To provide a unified, high-probability regret guarantee for the new algorithm across all game types, including degenerate ones.

Proposed method

• Propose NeighbourhoodWatch2, an improved variant of Foster and Rakhlin’s NeighbourhoodWatch, with enhanced handling of degenerate actions via refined cell decomposition and local observability checks.
• Use a geometric approach to define cells as intersections of action loss sets, enabling identification of locally observable actions and degenerate actions.
• Apply exponential weights with adaptive learning rates, ensuring high-probability regret bounds through martingale concentration and Hoeffding-style inequalities.
• Introduce a novel local game decomposition to analyze regret, distinguishing between local and global feedback structures.
• Correct a critical error in Bartók et al. (2014) involving the interchange of expectation and maximum in localization arguments.
• Prove lower bounds that establish the necessity of linear dependence on the number of feedback symbols $F$ in general.

Experimental results

Research questions

• RQ1 What is the correct classification of all finite adversarial partial monitoring games, including previously unresolved degenerate cases?
• RQ2 Can a single algorithm achieve optimal regret across all game types, including degenerate and non-degenerate games, with high-probability guarantees?
• RQ3 How does the regret depend on structural properties such as the number of feedback symbols $F$, local action dimension $K_{\text{loc}}$, and game geometry?
• RQ4 Why do prior analyses of Bartók’s algorithm fail to yield high-probability bounds, and how can this be corrected without increasing algorithmic complexity?
• RQ5 Is the linear dependence on $F$ in the regret bound unavoidable, and what structural conditions allow for better scaling?

Key findings

• The paper completes the classification of finite adversarial partial monitoring games by resolving the degenerate case, which was an open problem since Bartók et al. (2014).
• NeighbourhoodWatch2 achieves a regret bound of $O(F\sqrt{nK_{\text{loc}}\log K})$, which improves upon Bartók’s (2013) result by eliminating dependence on arbitrary game constants and reducing logarithmic factors.
• The analysis corrects a flaw in Bartók et al. (2014) where expectation and maximum were incorrectly interchanged in the localization argument.
• The paper proves that the linear dependence on $F$ is unavoidable in general, establishing a tight lower bound that matches the upper bound up to logarithmic factors.
• The new algorithm is simpler than Bartók’s and provides high-probability regret guarantees, unlike the original analysis which relied on expectations.
• The study confirms that degenerate actions, though not uniquely optimal, are informative and must be included in the algorithmic design to achieve optimal regret.