[Paper Review] Can large language models democratize access to dual-use biotechnology?

Emily H. Soice, Rafael Rocha+3 more|arXiv (Cornell University)|Jun 6, 2023

Biomedical and Engineering Education24 citations

TL;DR

The paper evaluates whether large language models can enable non-experts to access dual-use biotech and presents potential safeguards to mitigate risks.

ABSTRACT

Large language models (LLMs) such as those embedded in 'chatbots' are accelerating and democratizing research by providing comprehensible information and expertise from many different fields. However, these models may also confer easy access to dual-use technologies capable of inflicting great harm. To evaluate this risk, the 'Safeguarding the Future' course at MIT tasked non-scientist students with investigating whether LLM chatbots could be prompted to assist non-experts in causing a pandemic. In one hour, the chatbots suggested four potential pandemic pathogens, explained how they can be generated from synthetic DNA using reverse genetics, supplied the names of DNA synthesis companies unlikely to screen orders, identified detailed protocols and how to troubleshoot them, and recommended that anyone lacking the skills to perform reverse genetics engage a core facility or contract research organization. Collectively, these results suggest that LLMs will make pandemic-class agents widely accessible as soon as they are credibly identified, even to people with little or no laboratory training. Promising nonproliferation measures include pre-release evaluations of LLMs by third parties, curating training datasets to remove harmful concepts, and verifiably screening all DNA generated by synthesis providers or used by contract research organizations and robotic cloud laboratories to engineer organisms or viruses.

Motivation & Objective

• Motivate evaluation of LLMs as a vector for democratizing access to dual-use biotechnology.
• Investigate whether prompt-based exploration by non-experts can yield actionable information for creating or deploying pathogens.
• Identify potential safeguards and policy recommendations to mitigate misuse of LLMs in biotech.

Proposed method

• A case study using MIT's Safeguarding the Future course to prompt non-scientists to probe LLM chatbots for assistance in pandemic-related tasks.
• Demonstrate steps where prompts elicit information on pathogens, synthesis, and protocols from LLMs.
• Analyze the ease with which non-experts could obtain actionable dual-use knowledge from chatbots.

Experimental results

Research questions

• RQ1 Can LLM chatbots be prompted to assist non-experts in identifying pandemic pathogens?
• RQ2 To what extent can LLMs provide information on generating pathogens from synthetic DNA and reverse genetics?
• RQ3 What nonproliferation measures can credibly reduce the risk of dual-use misuse of LLMs?

Key findings

• Chatbots suggested four potential pandemic pathogens within one hour of prompting.
• They explained how pathogens can be generated from synthetic DNA using reverse genetics.
• They supplied names of DNA synthesis companies unlikely to screen orders.
• They identified detailed protocols and troubleshooting steps and recommended using core facilities or CROs for reverse genetics.
• Collectively, results suggest LLMs may make pandemic-class agents widely accessible once credibly identified, even to non-experts.