[论文解读] A Distributed Learning Dynamics in Social Groups

L. Elisa Celis, P. M. Krafft+1 more|arXiv (Cornell University)|May 8, 2017

Opinion Dynamics and Social Influence参考文献 26被引用 4

一句话总结

本文提出了一种社会群体中的分布式学习动态机制，其中个体基于近期绩效信号模仿他人，证明这种简单、无记忆的过程能快速引导群体收敛至最优选项，且遗憾极低。研究建立有限群体动态实际上实现了乘法权重更新（MWU）方法的随机变体，实现了近乎最优的性能，遗憾边界明确为 $O(\sqrt{\ln m / T})$ 阶。

ABSTRACT

We study a distributed learning process observed in human groups and other social animals. This learning process appears in settings in which each individual in a group is trying to decide over time, in a distributed manner, which option to select among a shared set of options. Specifically, we consider a stochastic dynamics in a group in which every individual selects an option in the following two-step process: (1) select a random individual and observe the option that individual chose in the previous time step, and (2) adopt that option if its stochastic quality was good at that time step. Various instantiations of such distributed learning appear in nature, and have also been studied in the social science literature. From the perspective of an individual, an attractive feature of this learning process is that it is a simple heuristic that requires extremely limited computational capacities. But what does it mean for the group -- could such a simple, distributed and essentially memoryless process lead the group as a whole to perform optimally? We show that the answer to this question is yes -- this distributed learning is highly effective at identifying the best option and is close to optimal for the group overall. Our analysis also gives quantitative bounds that show fast convergence of these stochastic dynamics. Prior to our work the only theoretical work related to such learning dynamics has been either in deterministic special cases or in the asymptotic setting. Finally, we observe that our infinite population dynamics is a stochastic variant of the classic multiplicative weights update (MWU) method. Consequently, we arrive at the following interesting converse: the learning dynamics on a finite population considered here can be viewed as a novel distributed and low-memory implementation of the classic MWU method.

研究动机与目标

• 理解在社会群体中，简单、分布式且无记忆的学习动态是否能导致最优集体决策。
• 分析在具有随机采样和采纳步骤的有限群体中，此类动态的收敛性与效率。
• 为该动态下的群体层面表现建立严格的遗憾边界。
• 弥合社会学习的经验观察与有限、随机设定下的理论分析之间的差距。
• 揭示所提出动态与经典乘法权重更新（MWU）方法之间的联系。

提出的方法

• 本文将动态建模为两步过程：每个个体随机抽样另一名群体成员，并在该成员的近期绩效信号良好时采纳其选择的选项。
• 引入该动态的无限群体随机变体，其分析上可处理，且被证明会强收敛。
• 通过理论分析将无限群体动态与乘法权重更新（MWU）方法的随机版本联系起来。
• 通过将无限群体模型的收敛性与有限群体动态的耦合论证相结合，推导出遗憾边界。
• 使用关键不等式和对数边界，控制群体层面表现与事后最优策略之间的偏差。
• 分析参数调优，特别是置信参数 $\beta$，以优化分布式设置下的遗憾性能。

实验结果

研究问题

• RQ1 在有限社会群体中，分布式、无记忆的学习过程能否以低遗憾收敛至最优选项？
• RQ2 此类过程的性能与事后最优策略相比如何？
• RQ3 该社会学习动态与经典乘法权重更新（MWU）方法之间是否存在理论联系？
• RQ4 在具有随机性的有限群体中，该动态的定量收敛与遗憾边界是什么？
• RQ5 该过程能否被视为 MWU 算法的一种分布式、低内存实现？

主要发现

• 该分布式学习动态在有限群体中实现了 $O(\sqrt{\ln m / T})$ 的遗憾边界，与随机 MWU 方法的最优速率一致。
• 当 $T \geq \ln m / \delta^2$ 时，遗憾被限制在 $3\delta$ 以内，表明其能快速收敛至近乎最优性能。
• 最优选项被选中的概率随时间增加，且对于较大的 $T$，有 $\mathbb{E}[P_1^{t-1}] \geq 1 - \frac{3\delta}{\eta_1 - \eta_2}$，表明对最优选项存在强烈的选择压力。
• 该动态的无限群体极限被证明是乘法权重更新（MWU）方法的随机变体，解释了其有效性。
• 群体层面的动态实际上解决了全信息问题，使得尽管个体仅观察近期信号，仍能实现高集体效率。
• 采纳规则中的参数 $\beta$ 对遗憾有关键影响；最优调优可实现标准的 $O(\sqrt{\ln m / T})$ 遗憾边界，表明现实群体可能隐式优化了该参数。