[Paper Review] Bridging AI and Science: Implications from a Large-Scale Literature Analysis of AI4Science

Yutong Xie, Yijun Pan+2 more|arXiv (Cornell University)|Nov 27, 2024

Big Data and Business IntelligenceBusiness, Management and Accounting3 citations

TL;DR

This paper presents a large-scale, data-driven analysis of AI4Science literature using LLMs to extract scientific problems and AI methods from 159,295 publications (2014–2024). It reveals significant imbalances in AI–science integration, identifies key gaps in cross-disciplinary linkages, and demonstrates through link prediction that RAG-augmented LLMs can effectively recommend AI methods for scientific problems, enabling more targeted interdisciplinary collaboration.

ABSTRACT

Artificial Intelligence has proven to be a transformative tool for advancing scientific research across a wide range of disciplines. However, a significant gap still exists between AI and scientific communities, limiting the full potential of AI methods in driving broad scientific discovery. Existing efforts in identifying and bridging this gap have often relied on qualitative examination of small samples of literature, offering a limited perspective on the broader AI4Science landscape. In this work, we present a large-scale analysis of the AI4Science literature, starting by using large language models to identify scientific problems and AI methods in publications from top science and AI venues. Leveraging this new dataset, we quantitatively highlight key disparities between AI methods and scientific problems, revealing substantial opportunities for deeper AI integration across scientific disciplines. Furthermore, we explore the potential and challenges of facilitating collaboration between AI and scientific communities through the lens of link prediction. Our findings and tools aim to promote more impactful interdisciplinary collaborations and accelerate scientific discovery through deeper and broader AI integration. Our code and dataset are available at: https://github.com/charles-pyj/Bridging-AI-and-Science.

Motivation & Objective

• To address the persistent gap between AI and scientific communities that limits the full potential of AI in scientific discovery.
• To provide a comprehensive, data-driven overview of the AI4Science landscape beyond qualitative reviews of small literature samples.
• To identify structural imbalances in the co-occurrence of scientific problems and AI methods across disciplines.
• To evaluate the feasibility and effectiveness of LLM-based link prediction in suggesting novel AI–science collaborations.
• To develop a novel, balanced AI4Science dataset for future research and tool development in interdisciplinary AI integration.

Proposed method

• Employed large language models (LLMs) to extract scientific problems, AI methods, and AI usage patterns from publications in top science and AI venues.
• Constructed a large-scale, balanced AI4Science dataset containing 159,295 publications, 123,773 scientific problems, and 35,522 AI methods.
• Generated 2D projections of problem and method embeddings using dimensionality reduction to visualize distributional disparities.
• Built a bipartite graph of scientific problem clusters and AI method clusters to analyze connectivity imbalances and identify 'hub' nodes.
• Applied retrieval-augmented generation (RAG) with LLMs to predict missing links between scientific problems and AI methods.
• Evaluated LLM link generation using ROUGE, BLEURT, and cosine similarity metrics across multiple models and few-shot settings.

Figure 1 . Illustration of the AI4Science dataset: (1) LLM-based extraction of scientific problem $p_{i}$ , AI method $m_{i}$ , and AI usage $u_{i}$ from an example publication. (2) 2D projections of the extracted scientific problems $\{p_{i}\}$ and AI methods $\{m_{i}\}$ . Each dot represents a pub

Experimental results

Research questions

• RQ1 What are the dominant scientific problems and AI methods in the AI4Science literature across the past decade?
• RQ2 How are AI methods distributed across scientific domains, and what disparities exist in their integration with scientific problems?
• RQ3 To what extent can LLMs with RAG effectively predict meaningful AI–science linkages that are not yet observed in the literature?
• RQ4 How do different LLM architectures and prompting strategies (e.g., direct, imitation, RAG) compare in generating accurate and semantically relevant AI method recommendations?
• RQ5 What structural imbalances exist in the network of AI–science collaborations, and how do they reflect under-explored research opportunities?

Key findings

• The AI4Science literature shows a pronounced imbalance in the distribution of AI methods across scientific problems, with a few AI methods (e.g., deep learning, transformers) dominating across multiple domains.
• Scientific problem nodes exhibit a highly heavy-tailed degree distribution, indicating that only a small number of problems attract the majority of AI applications, while many remain underexplored.
• RAG-augmented LLMs outperformed direct prompting and imitation in link prediction, achieving a ROUGE-1-F score of 0.314 (@10) and a cosine similarity of 0.900 for Sci→AI link generation using gpt-3.5-turbo.
• The model performance was significantly improved by including RAG examples, with ROUGE-1-F increasing from 0.263 (direct) to 0.290 (RAG n=5) for Sci→AI generation.
• Link prediction performance was higher on under-investigated scientific problems (ROUGE-1-F: 0.301 @1) than on well-investigated ones (0.284 @1), suggesting potential for discovering novel AI applications.
• The bipartite graph revealed strong hubs in both problem and method spaces, indicating that certain AI methods (e.g., GNNs, diffusion models) are frequently applied to a narrow set of problems, while many scientific domains remain underserved.

Figure 2 . The AI-Science bipartite graph and node distributions. (a) The bipartite graph with scientific problem clusters and AI method clusters as nodes. The size of each node corresponds to its unweighted degree, representing the number of AI methods applied to a scientific problem, or the number