[论文解读] Double Whammy - How ICT Projects are Fooled by Randomness and Screwed by Political Intent

Alexander Budzier, Bent Flyvbjerg|arXiv (Cornell University)|Apr 16, 2013

Complex Systems and Decision Making被引用 4

一句话总结

本文分析了1,471项价值2410亿美元的信息与通信技术（ICT）项目，表明成本超支源于两种截然不同的机制：政治操纵（在临界点之前）和厚尾极端值带来的随机性（超过该点之后）。通过累积分布函数，研究揭示了分布形态从指数分布到正态分布，再到幂律分布的转变，证明项目失败既受故意的乐观偏差驱动，也受对罕见高影响事件的系统性盲视影响。

ABSTRACT

The cost-benefit analysis formulates the holy trinity of objectives of project management - cost, schedule, and benefits. As our previous research has shown, ICT projects deviate from their initial cost estimate by more than 10% in 8 out of 10 cases. Academic research has argued that Optimism Bias and Black Swan Blindness cause forecasts to fall short of actual costs. Firstly, optimism bias has been linked to effects of deception and delusion, which is caused by taking the inside-view and ignoring distributional information when making decisions. Secondly, we argued before that Black Swan Blindness makes decision-makers ignore outlying events even if decisions and judgements are based on the outside view. Using a sample of 1,471 ICT projects with a total value of USD 241 billion - we answer the question: Can we show the different effects of Normal Performance, Delusion, and Deception? We calculated the cumulative distribution function (CDF) of (actual-forecast)/forecast. Our results show that the CDF changes at two tipping points - the first one transforms an exponential function into a Gaussian bell curve. The second tipping point transforms the bell curve into a power law distribution with the power of 2. We argue that these results show that project performance up to the first tipping point is politically motivated and project performance above the second tipping point indicates that project managers and decision-makers are fooled by random outliers, because they are blind to thick tails. We then show that Black Swan ICT projects are a significant source of uncertainty to an organisation and that management needs to be aware of.

研究动机与目标

• 调查ICT项目成本超支的双重成因：政治意图与随机性。
• 确定绩效偏差是否遵循与乐观偏差和‘黑天鹅’事件相关的特定统计模式。
• 识别成本超支分布中的临界点，以揭示从政治影响到统计极端值主导的转变信号。
• 证明‘黑天鹅’ICT项目显著增加了组织的不确定性。
• 验证内部视角错觉与外部视角盲视在预测失败中的作用。

提出的方法

• 收集了1,471项ICT项目的数据，总价值为2410亿美元。
• 计算了实际成本与预测成本之差除以预测成本的累积分布函数（CDF）。
• 分析CDF以检测不同绩效水平下分布形态的结构性转变。
• 识别出分布从指数分布转变为正态分布，再转变为幂律分布（指数为2）的两个临界点。
• 使用统计建模区分由政治驱动的超支与由极端随机事件引起的超支。
• 应用内部视角与外部视角框架，评估项目预测中的决策偏差。

实验结果

研究问题

• RQ1 在不同绩效水平下，ICT项目成本超支的统计模式如何呈现？
• RQ2 政治动机与随机性如何共同影响项目成本预测的准确性？
• RQ3 项目绩效在何时从受政治驱动转变为由极端异常值主导？
• RQ4 ‘黑天鹅’事件在多大程度上导致ICT项目的成本超支？
• RQ5 内部视角与外部视角决策框架能否解释观察到的分布形态转变？

主要发现

• 成本超支的累积分布显示出两个显著的临界点，表明存在三种绩效状态。
• 在第一个临界点之前，成本超支遵循指数分布，表明存在政治影响和故意的乐观情绪。
• 在第一个与第二个临界点之间，分布变为正态分布，表明存在中等程度的不确定性和风险。
• 超过第二个临界点后，分布转变为幂律分布，指数为2，表明存在极端异常值和‘黑天鹅’效应。
• 研究证实，‘黑天鹅’ICT项目是不确定性的重要来源，尤其是在第二个临界点之后。
• 结果验证了乐观偏差（错觉）与‘黑天鹅’盲视是ICT项目预测失败的关键驱动因素。