[Paper Review] A Bibliometric Review of Large Language Models Research from 2017 to 2023

Lizhou Fan, Lingyao Li+4 more|arXiv (Cornell University)|Apr 3, 2023

Topic Modeling49 citations

TL;DR

A bibliometric and discourse analysis of LLM research from 2017–2023, synthesizing over 5,000 publications to map trends, paradigms, and applications.

ABSTRACT

Large language models (LLMs) are a class of language models that have demonstrated outstanding performance across a range of natural language processing (NLP) tasks and have become a highly sought-after research area, because of their ability to generate human-like language and their potential to revolutionize science and technology. In this study, we conduct bibliometric and discourse analyses of scholarly literature on LLMs. Synthesizing over 5,000 publications, this paper serves as a roadmap for researchers, practitioners, and policymakers to navigate the current landscape of LLMs research. We present the research trends from 2017 to early 2023, identifying patterns in research paradigms and collaborations. We start with analyzing the core algorithm developments and NLP tasks that are fundamental in LLMs research. We then investigate the applications of LLMs in various fields and domains including medicine, engineering, social science, and humanities. Our review also reveals the dynamic, fast-paced evolution of LLMs research. Overall, this paper offers valuable insights into the current state, impact, and potential of LLMs research and its applications.

Motivation & Objective

• Characterize the evolution and landscape of large language models (LLMs) research from 2017 to early 2023.
• Identify core algorithmic developments and fundamental NLP tasks driving LLMs research.
• Examine applications of LLMs across medicine, engineering, social sciences, and humanities.
• Map collaborations, research paradigms, and discourse patterns to inform researchers, practitioners, and policymakers.

Proposed method

• Perform bibliometric analysis on a corpus of over 5,000 publications on LLMs.
• Conduct discourse analyses to reveal patterns in terminology, topics, and framing.
• Synthesize trends in core algorithms, NLP tasks, and application domains.
• Aggregate insights to provide a roadmap of the current state, impact, and potential of LLMs research.

Experimental results

Research questions

• RQ1 What are the dominant research paradigms and algorithmic developments in LLMs from 2017 to early 2023?
• RQ2 How have LLMs been applied across different domains such as medicine, engineering, social sciences, and humanities?
• RQ3 What are the patterns of collaboration and discourse in the LLMs literature?
• RQ4 What is the trajectory and future potential of LLMs research according to the bibliometric and discourse analyses?

Key findings

• LLMs research shows a dynamic, fast-paced evolution across 2017–early 2023.
• Core algorithm developments and fundamental NLP tasks remain central to LLMs research.
• LLMs are applied across diverse fields including medicine, engineering, social sciences, and humanities.
• The study highlights patterns of collaboration and discourse shaping the LLMs landscape.
• The work provides a roadmap for researchers, practitioners, and policymakers.