[Paper Review] State of the Art in Artificial Intelligence applied to the Legal Domain

João Dias, Pedro A. Santos+5 more|arXiv (Cornell University)|Mar 10, 2022

Artificial Intelligence in Law5 citations

TL;DR

This paper reviews the state of the art in applying artificial intelligence—particularly deep learning and semantic role labeling—to legal text analysis, focusing on overcoming the 'natural language barrier' in legal information retrieval. It proposes using advanced NLP techniques to extract structured legal norms from unstructured text, achieving 76.33 F1-score for obligation norms and 85.15 for power norms, enabling more intuitive and accurate legal search for non-experts.

ABSTRACT

While Artificial Intelligence applied to the legal domain is a topic with origins in the last century, recent advances in Artificial Intelligence are posed to revolutionize it. This work presents an overview and contextualizes the main advances on the field of Natural Language Processing and how these advances have been used to further the state of the art in legal text analysis.

Motivation & Objective

• To analyze recent advancements in NLP and AI for improving access to legal texts.
• To address the 'natural language barrier' that hinders non-experts from understanding and retrieving legal norms.
• To evaluate the effectiveness of semantic role labeling and norm extraction techniques on real legislative texts.
• To identify scalable, low-cost methods for training legal NLP systems using weakly-supervised techniques.
• To propose a proactive legal information retrieval system that enhances relevance and timeliness of legal results.

Proposed method

• Employing semantic role labeling (SRL) to identify predicates, arguments, and conditions in legal texts.
• Using a hybrid approach combining logic-based formalization and data-centric NLP techniques for norm representation.
• Applying weakly-supervised learning to generate question-answer pairs for training legal information retrieval models.
• Extracting legal norms using predefined templates based on Eunomos and Legal-URN Hohfeldian models.
• Mapping natural language expressions in legislation to structured norm types: permission, obligation, power, etc.
• Validating the system on real legislative corpora, including unseen legal bodies, to assess generalization.

Experimental results

Research questions

• RQ1 How can recent NLP advances be leveraged to improve legal information retrieval for non-expert users?
• RQ2 To what extent can semantic role labeling accurately extract legal norm components from unstructured legal text?
• RQ3 What is the performance of norm extraction systems on unseen legislative texts, particularly for passive roles and complex conditions?
• RQ4 Can weakly-supervised techniques reduce annotation costs while maintaining high accuracy in legal NLP tasks?
• RQ5 How effective is the integration of semantic representation with logic-based reasoning in legal text understanding?

Key findings

• The system achieved an F1-score of 76.33 in detecting obligation-type norms and 85.15 for power-type norms on a test set of legal texts.
• While 78.5% of legal definitions had fully correct SRL arguments, only 52% of norm examples achieved full argument correctness.
• The system performed well in extracting norm type and active role, but struggled with passive roles and conditional clauses.
• Semantic information extraction enabled deeper understanding beyond keyword matching, improving relevance in legal text retrieval.
• Weakly-supervised techniques showed promise in reducing annotation costs and scaling to large legal corpora.
• The integration of deep learning with semantic representation models offers a viable path toward proactive, user-friendly legal information systems.