[논문 리뷰] Corpus for Automatic Structuring of Legal Documents

Prathamesh Kalamkar, Aman Tiwari외 5명|arXiv (Cornell University)|2022. 01. 31.

Artificial Intelligence in Law인용 수 39

한 줄 요약

요약: 12가지 수사적 역할로 주석된 새로운 영어 인도 법원 판결 코퍼스를 소개하고, 기초 RR 예측 모델을 학습시키며, RR-기반 요약 및 판결 예측을 시연한다.

ABSTRACT

In populous countries, pending legal cases have been growing exponentially. There is a need for developing techniques for processing and organizing legal documents. In this paper, we introduce a new corpus for structuring legal documents. In particular, we introduce a corpus of legal judgment documents in English that are segmented into topical and coherent parts. Each of these parts is annotated with a label coming from a list of pre-defined Rhetorical Roles. We develop baseline models for automatically predicting rhetorical roles in a legal document based on the annotated corpus. Further, we show the application of rhetorical roles to improve performance on the tasks of summarization and legal judgment prediction. We release the corpus and baseline model code along with the paper.

연구 동기 및 목표

• Create a large annotated corpus of Indian legal judgments labeled with topically coherent rhetorical roles (RRs).
• Develop transformer-based baselines for automatic RR labeling at the sentence level.
• Show applications of rhetorical roles to extractive and abstractive summarization of court judgments.
• Demonstrate how RR predictions can improve legal judgment prediction.
• Release the corpus and baseline code to foster Legal NLP research.

제안 방법

• Annotate 354 Indian legal judgments in English with 12 RR labels plus NONE at the sentence level.
• Use a crowdsourcing data annotation pipeline with calibration, MOOC, ground-truth checks, and majority adjudication.
• Evaluate baseline RR prediction with transformer-based architectures (BERT_CRF, BERT_only, SciBERT-HSLN).
• Incorporate context by Bi-LSTM and attention within SciBERT-HSLN for sentence representation and CRF for labeling.
• Apply RR predictions to extractive summarization (BERTSUM RR) and abstractive summarization (Legal Pegasus RR).
• Test RR-based filtering for court judgment prediction using XLNet on last-512 tokens (and on ANALYSIS-only sentences).

실험 결과

연구 질문

• RQ1 Can a large corpus of English Indian legal judgments annotated with Rhetorical Roles be created and crowdsourced with acceptable quality?
• RQ2 Do transformer-based models can accurately predict sentence-level Rhetorical Roles in long legal documents?
• RQ3 Do Rhetorical Roles improve downstream tasks like extractive/abstractive summarization of judgments and judgment prediction?
• RQ4 How well do RR-based methods generalize to out-domain legal domains?
• RQ5 What is the impact of selecting specific RR-based inputs on judgment prediction performance?

주요 결과

• SciBERT-HSLN achieves the best in-domain RR prediction with a weighted F1 of 0.78.
• BERT-only Baseline yields 0.67 weighted F1 and BERT_CRF yields 0.23 weighted F1 on in-domain test.
• Out-domain RR prediction overall F1 is 0.79, with varying per-label performance.
• For extractive summarization, BERTSUM RR improves ROUGE-1/2/L to 0.62/0.46/0.61 versus BERTSUM at 0.60/0.42/0.59.
• For abstractive summarization, Legal Pegasus RR yields ROUGE-1/2/L of 0.56/0.36/0.48 versus Legal Pegasus at 0.55/0.34/0.47.
• Judgment prediction using XLNet_last512 achieves 0.59 F1, while XLNet_last512_Analysis achieves 0.62 F1.