[论文解读] Avoiding Braess' Paradox through Collective Intelligence

Kagan Tumer, David H. Wolpert|ArXiv.org|Dec 20, 1999

Reinforcement Learning in Robotics参考文献 25被引用 7

一句话总结

本文提出了一种基于集体智能（COIN）的路由算法，通过考虑路由决策的全局副作用，避免了Braess悖论——即增加网络链路反而导致整体性能下降的现象。与仅优化单个数据包成本但损害全局效率的最优最短路径算法（ISPA）不同，该COIN方法采用前瞻性的、全局协调的路由机制，在成本估计不精确的情况下仍优于ISPA，实验与机器学习适配验证了其有效性。

ABSTRACT

In an Ideal Shortest Path Algorithm (ISPA), at each moment each router in a network sends all of its traffic down the path that will incur the lowest cost to that traffic. In the limit of an infinitesimally small amount of traffic for a particular router, its routing that traffic via an ISPA is optimal, as far as cost incurred by that traffic is concerned. We demonstrate though that in many cases, due to the side-effects of one router's actions on another routers performance, having routers use ISPA's is suboptimal as far as global aggregate cost is concerned, even when only used to route infinitesimally small amounts of traffic. As a particular example of this we present an instance of Braess' paradox for ISPA's, in which adding new links to a network decreases overall throughput. We also demonstrate that load-balancing, in which the routing decisions are made to optimize the global cost incurred by all traffic currently being routed, is suboptimal as far as global cost averaged across time is concerned. This is also due to "side-effects", in this case of current routing decision on future traffic. The theory of COllective INtelligence (COIN) is concerned precisely with the issue of avoiding such deleterious side-effects. We present key concepts from that theory and use them to derive an idealized algorithm whose performance is better than that of the ISPA, even in the infinitesimal limit. We present experiments verifying this, and also showing that a machine-learning-based version of this COIN algorithm in which costs are only imprecisely estimated (a version potentially applicable in the real world) also outperforms the ISPA, despite having access to less information than does the ISPA. In particular, this COIN algorithm avoids Braess' paradox.

研究动机与目标

• 解决ISPA在局部最优下导致全局网络性能次优的问题。
• 研究副作用的影响——即某一路由器的路由选择会降低其他路由器的性能——如何导致网络路由中的Braess悖论。
• 构建一个全局优化框架，能够预测未来流量，并缓解当前路由决策导致的长期成本累积。
• 证明即使使用不精确的成本估计，经机器学习增强的该算法版本在实践中仍优于ISPA。
• 为避免去中心化、局部最优路由引发的悖论性网络性能下降，提供理论与实证基础。

提出的方法

• 利用COllective INtelligence（COIN）理论形式化问题，建模代理（路由器）之间的相互依赖关系，以避免有害副作用。
• 设计一种理想化的基于COIN的路由算法，通过考虑当前决策对未来的影响，优化所有流量的总成本，而不仅限于无穷小流。
• 引入一种前瞻性成本模型，反映当前路由选择对未来网络性能的影响，纠正ISPA的短视优化。
• 实现COIN算法的机器学习变体，从噪声或不完整的成本反馈中学习路由策略，提升现实适用性。
• 通过实验仿真，在不同网络拓扑和链路增加场景下，对比COIN算法与ISPA的性能表现。
• 应用多智能体系统与自适应控制技术，平衡局部决策与全局网络效率。

实验结果

研究问题

• RQ1 一种考虑全局副作用的路由算法，是否能在最小化总网络成本方面优于ISPA？
• RQ2 在网络中增加新链路是否总是提升性能，还是可能因路由决策导致悖论性性能下降？
• RQ3 即使成本估计不精确，去中心化、基于学习的路由系统是否仍能避免Braess悖论？
• RQ4 当前的路由决策如何影响未来网络性能，这种时间依赖性是否可建模以提升长期效率？
• RQ5 COIN形式化的集体智能在多大程度上能实现比局部优化更优的全局优化？

主要发现

• ISPA虽然对单个无穷小流是最优的，但由于未考虑对其他路由器的副作用，导致全局性能次优。
• 本文展示了网络路由中Braess悖论的具体实例：在使用ISPA时，增加链路反而降低整体网络吞吐量。
• 仅优化当前流量的负载均衡策略在长期中表现次优，因其忽略了路由决策对未来流量的长期影响。
• 所提出的基于COIN的算法在理想环境与成本估计不精确的场景下，均优于ISPA，证实其鲁棒性与全局最优性。
• 即使使用噪声或不完整成本数据，经机器学习增强的COIN算法版本仍持续优于ISPA，证明其实际可行性。
• COIN框架通过协调路由决策，成功避免了Braess悖论，防止了无益的网络扩展。