[Paper Review] When Will AI Exceed Human Performance? Evidence from AI Experts

Katja Grace, John Salvatier+3 more|arXiv (Cornell University)|May 24, 2017

Big Data and Business IntelligenceBusiness, Management and Accounting3 references222 citations

TL;DR

The paper surveys AI researchers to forecast timelines for AI surpassing human performance across tasks and occupations, finding a 50% chance of HLMI within 45 years and regional differences, with many milestones expected within a decade.

ABSTRACT

Advances in artificial intelligence (AI) will transform modern life by reshaping transportation, health, science, finance, and the military. To adapt public policy, we need to better anticipate these advances. Here we report the results from a large survey of machine learning researchers on their beliefs about progress in AI. Researchers predict AI will outperform humans in many activities in the next ten years, such as translating languages (by 2024), writing high-school essays (by 2026), driving a truck (by 2027), working in retail (by 2031), writing a bestselling book (by 2049), and working as a surgeon (by 2053). Researchers believe there is a 50% chance of AI outperforming humans in all tasks in 45 years and of automating all human jobs in 120 years, with Asian respondents expecting these dates much sooner than North Americans. These results will inform discussion amongst researchers and policymakers about anticipating and managing trends in AI.

Motivation & Objective

• Forecast timelines for high-level machine intelligence (HLMI) and automation of all human jobs.
• Assess expected dates for AI surpassing human performance in specific tasks and occupations.
• Examine perceptions of AI safety, risks, and the prioritization of risk-minimizing research.

Proposed method

• Survey of researchers who published at NIPS/ICML in 2015.
• Elicitation of beliefs using gamma CDF fits to individual forecasts.
• Aggregation via mixture distributions and bootstrapped 95% intervals.
• Comparison across regions (e.g., Asia vs North America) and demographics.
• Supplementary analyses include robustness checks and framing effects on predictions.

Experimental results

Research questions

• RQ1 When will HLMI arrive with a 50% probability?
• RQ2 What are the predicted dates for AI to outperform humans in specific tasks or occupations?
• RQ3 Do regional or demographic factors systematically shift AI timelines?
• RQ4 What are researchers’ views on AI safety and potential extreme outcomes?

Key findings

• Average respondent predicted HLMI with a 50% probability within 45 years.
• 10% probability for HLMI within 9 years.
• Asian respondents expect HLMI sooner (about 30 years) than North Americans (about 74 years).
• Mean estimate for 50% probability of full automation of labor is 122 years; 20-year 10% probability.
• About 20% probability of an explosive improvement two years after HLMI; 10% for explosive progress two years after HLMI.
• Approximately 48% think AI safety research should be prioritized more than the status quo.