[论文解读] The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research

Nur Ahmed, Muntasir Wahed|arXiv (Cornell University)|Oct 22, 2020

Explainable Artificial Intelligence (XAI)参考文献 58被引用 105

一句话总结

本研究分析深度学习如何加剧有利于大型企业和精英大学的算力鸿沟，挤压中下层机构，并使用广义合成控制方法显示算力获取推动这种分歧。

ABSTRACT

Increasingly, modern Artificial Intelligence (AI) research has become more computationally intensive. However, a growing concern is that due to unequal access to computing power, only certain firms and elite universities have advantages in modern AI research. Using a novel dataset of 171394 papers from 57 prestigious computer science conferences, we document that firms, in particular, large technology firms and elite universities have increased participation in major AI conferences since deep learning's unanticipated rise in 2012. The effect is concentrated among elite universities, which are ranked 1-50 in the QS World University Rankings. Further, we find two strategies through which firms increased their presence in AI research: first, they have increased firm-only publications; and second, firms are collaborating primarily with elite universities. Consequently, this increased presence of firms and elite universities in AI research has crowded out mid-tier (QS ranked 201-300) and lower-tier (QS ranked 301-500) universities. To provide causal evidence that deep learning's unanticipated rise resulted in this divergence, we leverage the generalized synthetic control method, a data-driven counterfactual estimator. Using machine learning based text analysis methods, we provide additional evidence that the divergence between these two groups - large firms and non-elite universities - is driven by access to computing power or compute, which we term as the "compute divide". This compute divide between large firms and non-elite universities increases concerns around bias and fairness within AI technology, and presents an obstacle towards "democratizing" AI. These results suggest that a lack of access to specialized equipment such as compute can de-democratize knowledge production.

研究动机与目标

• 促使关注点：AI研究正越来越多地由拥有大量计算资源的主体主导。
• 量化自2012年深度学习兴起以来，不同信誉等级的企业与大学之间的参与差距。
• 建立深度学习兴起、算力获取与AI研究中观测到的发表与多样性变化之间的因果关系。

提出的方法

• 构建一个由57个知名计算机科学会议的171,394篇论文组成的新数据集。
• 使用广义合成控制方法为深度学习对不同主体参与的影响提供因果反事实。
• 应用基于机器学习的文本分析来研究导致大型企业与非精英大学之间在获取计算资源方面分歧的驱动因素。

实验结果

研究问题

• RQ1 深度学习的兴起是否与企业和精英大学在主要AI会议中的参与度增加相关？
• RQ2 观测到的参与变动主要由企业、精英大学推动，还是两者的结合？
• RQ3 获取计算能力在推动大型企业与非精英大学之间分歧中扮演什么角色？
• RQ4 是否有证据表明算力鸿沟挤压了AI研究中的中等及以下等级的大学？

主要发现

• 自2012年深度学习兴起以来，企业与精英大学在主要AI会议中的存在感提升。
• 这一效应集中在QS世界大学排名中位列1–50的精英大学。
• 企业通过纯企业自刊物以及主要与精英大学的合作来扩张。
• 分歧与获取计算能力的不平等相关，即算力鸿沟，推动偏见性和不更民主的知识生产。
• 结果表明，缺乏对专用硬件的获取可能限制非精英机构的知识生产。