[Paper Review] Thousands of AI Authors on the Future of AI

Katja Grace, H. T. L. Stewart+5 more|arXiv (Cornell University)|Jan 5, 2024

Big Data and Business Intelligence36 citations

TL;DR

A large-scale survey of 2,778 AI researchers predicts milestones and risks of AI progress, finding earlier timelines in 2023 versus 2022 and broad variance in views on risks and impacts.

ABSTRACT

In the largest survey of its kind, 2,778 researchers who had published in top-tier artificial intelligence (AI) venues gave predictions on the pace of AI progress and the nature and impacts of advanced AI systems The aggregate forecasts give at least a 50% chance of AI systems achieving several milestones by 2028, including autonomously constructing a payment processing site from scratch, creating a song indistinguishable from a new song by a popular musician, and autonomously downloading and fine-tuning a large language model. If science continues undisrupted, the chance of unaided machines outperforming humans in every possible task was estimated at 10% by 2027, and 50% by 2047. The latter estimate is 13 years earlier than that reached in a similar survey we conducted only one year earlier [Grace et al., 2022]. However, the chance of all human occupations becoming fully automatable was forecast to reach 10% by 2037, and 50% as late as 2116 (compared to 2164 in the 2022 survey). Most respondents expressed substantial uncertainty about the long-term value of AI progress: While 68.3% thought good outcomes from superhuman AI are more likely than bad, of these net optimists 48% gave at least a 5% chance of extremely bad outcomes such as human extinction, and 59% of net pessimists gave 5% or more to extremely good outcomes. Between 38% and 51% of respondents gave at least a 10% chance to advanced AI leading to outcomes as bad as human extinction. More than half suggested that "substantial" or "extreme" concern is warranted about six different AI-related scenarios, including misinformation, authoritarian control, and inequality. There was disagreement about whether faster or slower AI progress would be better for the future of humanity. However, there was broad agreement that research aimed at minimizing potential risks from AI systems ought to be prioritized more.

Motivation & Objective

• Assess AI researchers' predictions on the pace of AI progress across tasks, occupations, and general human-level performance.
• Examine expectations for HLMI and FAOL timelines and their framing effects.
• Explore perceived risks, societal impacts, and priorities for AI safety research among AI researchers.
• Compare 2023 forecasts with 2022 and prior surveys to identify shifts in expectations and framing effects.

Proposed method

• Surveyed 2,778 AI researchers who published in top AI venues in the prior year (NeurIPS, ICML, ICLR, AAAI, IJCAI, JMLR).
• Used Likert-scale, probability estimates, and year-based responses with fixed-years and fixed-probabilities framing to elicit predictions.
• Fitted gamma distributions to individual respondent data to obtain aggregate distributions for milestones and HLMI/FAOL.
• Implemented framing controls and randomization to mitigate question framing effects and assess robustness of results.
• Compared 2023 results with 2022 and 2016 surveys to map changes in predicted timelines and attitudes.

Figure 1: Most milestones are predicted to have better than even odds of happening within the next ten years, though with a wide range of plausible dates. The figure shows aggregate distributions over when selected milestones are expected, including 39 tasks, four occupations, and two measures of ge

Experimental results

Research questions

• RQ1 When will AI achieve feasibility for a set of 39 tasks and other milestones (within a year and with specified probability thresholds)?
• RQ2 How soon will HLMI and FAOL be feasible, and how do task vs occupation framing affect these forecasts?
• RQ3 What are AI researchers' views on risks, societal impacts, and the desirability of safety-focused research prioritization?
• RQ4 What explains shifts in predictions from 2022 to 2023, and how do demographic factors influence forecasts?
• RQ5 Do researchers perceive intelligence explosion risks, and how do they evaluate potential enhancements in progress post-HLMI?

Key findings

• Most of the 39 tasks have a >50% chance of being feasible within the next ten years, including autonomously building a payment site and generating a song indistinguishable from a hit artist’s work.
• The 50% milestone dates for HLMI moved from 2060 (2022) to 2047 (2023), while FAOL’s 50% date moved from 2164 (2022) to 2116 (2023).
• Framing effects: fixed-year vs fixed-probability framing yields different 50% timelines, with fixed-probability framing predicting earlier milestones than fixed-year framing.
• A majority (68.3%) view good outcomes from superhuman AI as more likely than bad, yet a substantial minority assign non-negligible probabilities to extreme outcomes such as human extinction (up to 10% or more for some responses).
• Over half of respondents advocate prioritizing AI safety research more than current levels, with support growing since earlier surveys.
• HLMI is predicted to arrive earlier than FAOL by ~70 years on a 50% basis, a difference that persists across years and framing.
• Respondents foresee certain 2043 traits in AI (e.g., surprise in behavior, human-like discussion ability) with high likelihood, though disagreement remains on some traits like taking action to attain power.
• There is substantial concern about AI-enabled misinformation, inequality, and authoritarian misuse, with several scenarios rated as warranting substantial or extreme concern by a sizable minority.

Figure 2: Expected feasibility of many AI milestones moved substantially earlier in the course of one year (between 2022 and 2023). The milestones are sorted (within each scale-adjusted chart) by size of drop from 2022 forecast to 2023 forecast, with the largest change first. The year when the aggre