[论文解读] On the Opportunities and Risks of Foundation Models

Rishi Bommasani, Drew A. Hudson+112 more|arXiv (Cornell University)|Aug 16, 2021

Domain Adaptation and Few-Shot Learning参考文献 1,095被引用 2,137

一句话总结

对基础模型（在广泛数据上训练、可适应多任务）的能力、涌现、同质化、生态系统及社会影响进行全面审查，并提出治理与研究方向的考虑。

ABSTRACT

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

研究动机与目标

• 界定基础模型，并把它们的研究动机定位为AI领域的范式转变。
• 分析在语言、视觉、机器人、推理和交互等方面的能力。
• 审视与基础模型相关的技术、数据、系统、安全、评估与理论等方面。
• 讨论在医疗、法律与教育等领域的应用，以及更广泛的社会影响。

提出的方法

• 综合计算机科学、社会科学、经济学与伦理学等跨学科视角，以刻画基础模型。
• 描述从数据创建到部署的生态系统，以推断下游影响。
• 讨论涌现行为及模型同质化对研究与实践的影响。
• 提供治理、伦理学以及学术界与产业界合作方面的规范性指导。
• 突出理解中的空白并提出未来研究与基础设施的方向。

实验结果

研究问题

• RQ1 基础模型在跨模态（语言、视觉、机器人、推理、交互）方面具备哪些能力？
• RQ2 涌现和同质化如何塑造基础模型的能力、风险与社会影响？
• RQ3 从数据创建到部署，基础模型周围的生态系统由哪些要素组成，治理应介入的环节在哪些？
• RQ4 为负责任地开发与部署基础模型需要哪些规范与制度安排（通常需要跨学科合作）？

主要发现

• 基础模型表现出涌现特性，并在大规模上具备上下文学习能力，这些能力并非在训练时显式设计。
• 模型与方法的高度同质化，虽然促进广泛迁移学习，但也带来共同行为失效模式与偏见。
• 涌现能力及这些模型的规模引发的社会、伦理和环境关注，需要审慎治理。
• 全面的社会影响取决于整个生态系统（数据创建、筛选、训练、适应、部署），而不仅仅是训练阶段。
• 学术界与产业界必须合作，学术界提供多样的学科视角与长期公共利益考量。
• 随着模型和训练数据日益成为专有且高成本，再现实性和开放性面临重大障碍。