[Paper Review] Natural Language Processing in the Legal Domain

Daniel Katz, Dirk Hartung+3 more|arXiv (Cornell University)|Feb 23, 2023

Artificial Intelligence in Law4 citations

TL;DR

This paper presents a comprehensive, data-driven analysis of the evolution of Natural Language Processing in law (Legal NLP) over the past decade, based on a curated corpus of 600+ peer-reviewed papers. It reveals increasing methodological sophistication, growing multilingual coverage, and rising adherence to reproducibility standards, signaling the field’s maturation toward professional and technical parity with mainstream NLP.

ABSTRACT

In this paper, we summarize the current state of the field of NLP & Law with a specific focus on recent technical and substantive developments. To support our analysis, we construct and analyze a nearly complete corpus of more than six hundred NLP & Law related papers published over the past decade. Our analysis highlights several major trends. Namely, we document an increasing number of papers written, tasks undertaken, and languages covered over the course of the past decade. We observe an increase in the sophistication of the methods which researchers deployed in this applied context. Slowly but surely, Legal NLP is beginning to match not only the methodological sophistication of general NLP but also the professional standards of data availability and code reproducibility observed within the broader scientific community. We believe all of these trends bode well for the future of the field, but many questions in both the academic and commercial sphere still remain open.

Motivation & Objective

• To map the current state of Legal NLP by analyzing a comprehensive corpus of 600+ research papers published over the past decade.
• To identify key trends in publication volume, research tasks, languages covered, and methodological sophistication in Legal NLP.
• To assess the field’s progress in adopting scientific standards such as data availability and code reproducibility.
• To evaluate the role of large language models (LLMs) and domain-specific adaptation in advancing legal text understanding.
• To establish a living, community-maintained survey infrastructure to support ongoing research and collaboration in Legal NLP.

Proposed method

• Construction of a nearly complete corpus of 600+ Legal NLP papers from major NLP conferences (e.g., ACL, NAACL, EMNLP) and specialized legal informatics venues.
• Systematic classification of papers using a taxonomy covering tasks, methods, languages, and data sources to enable longitudinal and comparative analysis.
• Application of network analysis to extract and visualize citation and reference graphs of the collected publications.
• Use of public repositories and community-driven contributions via a web-based infrastructure to maintain and update the corpus dynamically.
• Integration of metadata (e.g., publication year, language, task type) to enable filtering and temporal trend analysis.
• Emphasis on reproducibility by linking each paper to its associated code and data, where available, and promoting open science practices.

Experimental results

Research questions

• RQ1 How has the volume and diversity of Legal NLP research evolved over the past decade in terms of publications, tasks, and languages?
• RQ2 To what extent has Legal NLP adopted the methodological and reproducibility standards seen in mainstream NLP research?
• RQ3 How do general-purpose large language models compare to domain-adapted models in performance on legal NLP tasks?
• RQ4 What role do generative models play in current and future Legal NLP research, particularly in scientific literature review and knowledge synthesis?
• RQ5 How can community-driven, open infrastructure enhance the sustainability and accessibility of Legal NLP research?

Key findings

• The number of Legal NLP papers has grown significantly over the past decade, reflecting increasing academic and commercial interest in the field.
• There has been a marked increase in the diversity of languages covered, indicating a broader, more inclusive research scope.
• Methodological sophistication has advanced, with researchers increasingly employing state-of-the-art NLP techniques, including transformer-based models and fine-tuning strategies.
• Legal NLP is approaching the standards of data availability and code reproducibility observed in the broader NLP community, signaling improved scientific rigor.
• Despite the rise of general-purpose LLMs, domain-specific fine-tuning and prompt engineering continue to yield superior performance on complex legal tasks.
• The field shows signs of maturation, with growing institutional and commercial interest, particularly from legal publishers, law firms, and courts, driven by the potential of neural models to scale legal services.