[Paper Review] Forecasting Transformative AI: An Expert Survey

Ross Gruetzemacher, David Paradice+1 more|arXiv (Cornell University)|Jan 24, 2019

Big Data Technologies and ApplicationsDecision Sciences11 references3 citations

TL;DR

This expert survey forecasts transformative AI timelines by polling AI researchers at ICML, IJCAI, and HLAI conferences in 2018. It finds a 50% chance of AI automating 90% of human tasks in 25 years and 99% in 50 years, with HLAI attendees projecting earlier, more optimistic timelines, highlighting the need for policy planning around transformative AI impacts.

ABSTRACT

Transformative AI technologies have the potential to reshape critical aspects of society in the near future. However, in order to properly prepare policy initiatives for the arrival of such technologies accurate forecasts and timelines are necessary. A survey was administered to attendees of three AI conferences during the summer of 2018 (ICML, IJCAI and the HLAI conference). The survey included questions for estimating AI capabilities over the next decade, questions for forecasting five scenarios of transformative AI and questions concerning the impact of computational resources in AI research. Respondents indicated a median of 21.5% of human tasks (i.e., all tasks that humans are currently paid to do) can be feasibly automated now, and that this figure would rise to 40% in 5 years and 60% in 10 years. Median forecasts indicated a 50% probability of AI systems being capable of automating 90% of current human tasks in 25 years and 99% of current human tasks in 50 years. The conference of attendance was found to have a statistically significant impact on all forecasts, with attendees of HLAI providing more optimistic timelines with less uncertainty. These findings suggest that AI experts expect major advances in AI technology to continue over the next decade to a degree that will likely have profound transformative impacts on society.

Motivation & Objective

• To forecast the timeline for transformative AI systems capable of automating a majority of human tasks.
• To assess expert perceptions of AI capabilities over the next decade and beyond.
• To examine how conference attendance influences forecasted timelines and uncertainty.
• To inform policy and societal preparedness for transformative AI impacts.

Proposed method

• Conducted a survey of AI researchers attending ICML, IJCAI, and HLAI conferences in summer 2018.
• Collected forecasts on AI capabilities for automating human tasks at 5-, 10-, 25-, and 50-year horizons.
• Used median estimates to determine the probability of reaching 90% and 99% automation of human tasks.
• Analyzed the impact of conference type (HLAI vs. ICML/IJCAI) on forecasted timelines and uncertainty.
• Employed statistical testing to assess significance of differences in forecasts across conference types.
• Reported results with confidence intervals and effect sizes for key forecasted milestones.

Experimental results

Research questions

• RQ1 When will AI systems be capable of automating 90% of current human tasks, according to expert estimates?
• RQ2 What is the projected timeline for AI to automate 99% of human tasks, and how certain are experts about this?
• RQ3 How do forecasts from experts at the HLAI conference compare to those from ICML and IJCAI in terms of timing and uncertainty?
• RQ4 What is the current level of AI automation feasibility for human tasks, and how is it expected to evolve over 5 and 10 years?
• RQ5 To what extent does the conference affiliation of experts influence their forecasts of transformative AI timelines?

Key findings

• Experts estimate a 21.5% feasibility of automating all human tasks today, rising to 40% in 5 years and 60% in 10 years.
• The median forecast indicates a 50% probability of AI automating 90% of human tasks by 2045 (25 years from 2018).
• The median forecast projects a 50% chance of 99% automation of human tasks by 2070 (50 years from 2018).
• Attendees of the HLAI conference provided significantly more optimistic forecasts than those from ICML and IJCAI, with lower uncertainty.
• Conference type had a statistically significant impact on all forecasted timelines, indicating differing expectations across expert communities.
• The survey results suggest that transformative AI impacts are expected to emerge within the next few decades, necessitating proactive policy planning.