[Paper Review] Learning to Recommend in Unknown Games

Arwa Alanqary, Zakaria Baba+2 more|arXiv (Cornell University)|Feb 19, 2026

Advanced Bandit Algorithms Research0 citations

TL;DR

The paper analyzes learning unknown utilities in multi-agent games via moderator recommendations under best-response and quantal-response feedback, proving learnability under QR (up to positive affine transformations) and non-learnability under BR, plus low-regret online recommendations.

ABSTRACT

We study preference learning through recommendations in multi-agent game settings, where a moderator repeatedly interacts with agents whose utility functions are unknown. In each round, the moderator issues action recommendations and observes whether agents follow or deviate from them. We consider two canonical behavioral feedback models-best response and quantal response-and study how the information revealed by each model affects the learnability of agents' utilities. We show that under quantal-response feedback the game is learnable, up to a positive affine equivalence class, with logarithmic sample complexity in the desired precision, whereas best-response feedback can only identify a larger set of agents' utilities. We give a complete geometric characterization of this set. Moreover, we introduce a regret notion based on agents' incentives to deviate from recommendations and design an online algorithm with low regret under both feedback models, with bounds scaling linearly in the game dimension and logarithmically in time. Our results lay a theoretical foundation for AI recommendation systems in strategic multi-agent environments, where recommendation compliances are shaped by strategic interaction.

Motivation & Objective

• Formalize a moderator–agent interaction where utilities are unknown and only action feedback is observed.
• Characterize learnability of utilities under two behavioral models: best response and quantal response.
• Provide an efficient online algorithm achieving low incentive-to-deviate regret under both feedback models.

Proposed method

• Model the game as a finite normal-form game with unknown utilities and a moderator issuing probability distributions over action profiles.
• Use two agent choice models: best response (BR) and quantal response (QR) to define feedback from observed actions.
• Define learnability up to agent-wise positive affine transformations, and characterize indistinguishability sets via polyhedral normal fans (under BR) and sign-based QR feedback.
• Prove that QR feedback makes the utilities learnable up to positive affine equivalence (Theorem 4) while BR feedback does not (Theorem 5).
• Provide an algorithm based on a geometric cutting-plane approach to obtain low regret with regret scaling as O(nM log T) under BR and QR (Theorem 3).
• Quantify learning complexity under QR as O(m n M log(1/ε)) recommendations for ε-precision (Theorem 2).

Experimental results

Research questions

• RQ1 Can the moderator recover unknown agents’ utilities from repeated recommendations and observed actions, and if not, what equivalence classes are identifiable?
• RQ2 Is it possible to design online recommendations with low incentive-to-deviate regret under BR and QR feedback?
• RQ3 How does the feedback model (BR vs QR) affect learnability and the structure of indistinguishable games?
• RQ4 What is the geometric characterization of indistinguishable utilities under BR feedback?
• RQ5 What are the sample complexity and regret bounds for learning and recommending in unknown games?

Key findings

• Under quantal-response feedback, the game utilities are learnable up to an agent-wise positive affine transformation.
• Under best-response feedback, a strictly larger class of utilities remains indistinguishable, and the paper provides a complete geometric characterization of this set.
• The QR-based learning algorithm achieves ε-approximation in utilities with O(m n M log(1/ε)) recommendations, near-linear in the game size.
• An online algorithm achieves regret O(nM log T) under both BR and QR feedback, with linear dependence on the number of actions and logarithmic dependence on time.
• The results establish a theoretical basis for AI recommendation systems in strategic multi-agent environments by leveraging off-equilibrium (feedback) behavior.