[Paper Review] Large Language Models for Scientific Synthesis, Inference and Explanation

Yizhen Zheng, Huan Yee Koh+5 more|arXiv (Cornell University)|Oct 12, 2023

Machine Learning in Materials ScienceMaterials Science12 citations

TL;DR

The paper shows how general-purpose large language models can perform scientific synthesis, infer from datasets, and explain predictions, enhancing ML-based molecular property tasks.

ABSTRACT

Large language models are a form of artificial intelligence systems whose primary knowledge consists of the statistical patterns, semantic relationships, and syntactical structures of language1. Despite their limited forms of "knowledge", these systems are adept at numerous complex tasks including creative writing, storytelling, translation, question-answering, summarization, and computer code generation. However, they have yet to demonstrate advanced applications in natural science. Here we show how large language models can perform scientific synthesis, inference, and explanation. We present a method for using general-purpose large language models to make inferences from scientific datasets of the form usually associated with special-purpose machine learning algorithms. We show that the large language model can augment this "knowledge" by synthesizing from the scientific literature. When a conventional machine learning system is augmented with this synthesized and inferred knowledge it can outperform the current state of the art across a range of benchmark tasks for predicting molecular properties. This approach has the further advantage that the large language model can explain the machine learning system's predictions. We anticipate that our framework will open new avenues for AI to accelerate the pace of scientific discovery.

Motivation & Objective

• Motivate the use of general-purpose LLMs for scientific synthesis, inference, and explanation in natural science tasks.
• Demonstrate augmenting conventional ML systems with LLM-derived synthesized and inferred knowledge to improve predictive performance.
• Show that LLMs can provide explanations for machine learning predictions in scientific contexts.
• Highlight potential for AI to accelerate scientific discovery across disciplines.

Proposed method

• Use general-purpose LLMs to infer from scientific datasets typically handled by specialized ML methods.
• Augment ML models with synthesized knowledge from scientific literature to improve task performance.
• Enable the LLM to generate explanations for the model’s predictions.
• Evaluate on benchmark tasks for predicting molecular properties to demonstrate performance gains.
• Provide a framework and discussion for integrating LLM-based synthesis and inference into scientific AI workflows.

Experimental results

Research questions

• RQ1 Can general-purpose LLMs infer from scientific datasets in ways comparable to specialized ML algorithms?
• RQ2 Does augmenting ML systems with LLM-synthesized knowledge improve predictive performance on scientific tasks?
• RQ3 Can LLMs provide meaningful explanations for ML predictions in scientific contexts?
• RQ4 What framework best supports using LLMs for synthesis, inference, and explanation to accelerate scientific discovery.

Key findings

• LLMs can augment knowledge for scientific inference from datasets and literature.
• Augmented systems with LLM-derived synthesis outperform current state-of-the-art on molecular property prediction benchmarks.
• The approach provides explanations for the ML predictions, enhancing interpretability.