[Paper Review] Learning the grammar of prescription: recurrent neural network grammars for medication information extraction in clinical texts

Ivan Lerner, Jordan Jouffroy+2 more|arXiv (Cornell University)|Apr 24, 2020

Natural Language Processing Techniques30 references4 citations

TL;DR

This study evaluates the Recurrent Neural Network Grammar (RNNG), a top-down, transition-based neural parser, for extracting medication information—drug names, dosage, frequency, duration, route, and condition—from French clinical texts. RNNG outperforms BiLSTM-based baselines in relation extraction (88.5% F1 vs. 84.6%) due to its architectural inductive bias and joint entity-relation modeling, though it lags slightly in entity detection (82.4% vs. 84.1%).

ABSTRACT

In this study, we evaluated the RNNG, a neural top-down transition based parser, for medication information extraction in clinical texts. We evaluated this model on a French clinical corpus. The task was to extract the name of a drug (or class of drug), as well as fields informing its administration: frequency, dosage, duration, condition and route of administration. We compared the RNNG model that jointly identify entities and their relations with separate BiLSTMs models for entities and relations as baselines. We call seq-BiLSTMs the baseline models for relations extraction that takes as extra-input the output of the BiLSTMs for entities. RNNG outperforms seq-BiLSTM for identifying relations, with on average 88.5% [87.2-89.8] versus 84.6 [83.1-86.1] F-measure. However, RNNG is weaker than the baseline BiLSTM on detecting entities, with on average 82.4 [80.8-83.8] versus 84.1 [82.7-85.6] % F- measure. RNNG trained only for detecting relations is weaker than RNNG with the joint modelling objective, 87.4 [85.8-88.8] versus 88.5% [87.2-89.8]. The performance of RNNG on relations can be explained both by the model architecture, which provides shortcut between distant parts of the sentence, and the joint modelling objective which allow the RNNG to learn richer representations. RNNG is efficient for modeling relations between entities in medical texts and its performances are close to those of a BiLSTM for entity detection.

Motivation & Objective

• To improve medication information extraction in clinical texts by modeling syntactic and relational structure using neural grammars.
• To evaluate whether a recurrent neural network grammar (RNNG) can jointly detect entities and their relations more effectively than separate models.
• To assess the impact of joint modeling and architectural design on performance in low-resource clinical NLP settings.
• To compare RNNG's performance against BiLSTM-based baselines for both entity and relation extraction in French clinical notes.

Proposed method

• The RNNG model uses a top-down, transition-based parsing approach to jointly predict entities and their relations in clinical text.
• The model employs a recurrent neural network to maintain hidden states that guide parsing decisions and represent structural dependencies.
• Joint training of entity and relation prediction allows the RNNG to learn richer, contextually informed representations through shared hidden states.
• The RNNG is compared against a baseline seq-BiLSTM model that first detects entities and then uses their outputs as input for relation extraction.
• The RNNG is also evaluated in a variant trained only for relation extraction to isolate the contribution of joint modeling.
• Model performance is evaluated using F1 scores on a French clinical corpus annotated for drug name, dosage, frequency, duration, route, and condition.

Experimental results

Research questions

• RQ1 Can a neural grammar model like RNNG outperform standard BiLSTM models in extracting relations between medication entities in clinical texts?
• RQ2 Does joint modeling of entities and relations in RNNG lead to better performance than separate entity and relation modeling?
• RQ3 How does the architectural inductive bias of RNNG, particularly its ability to connect distant constituents, affect relation extraction?
• RQ4 What is the contribution of joint training to the model’s performance compared to training only on relations?
• RQ5 To what extent does RNNG’s performance in entity detection compare to that of a dedicated BiLSTM model?

Key findings

• RNNG achieves a mean F1 score of 88.5% [87.2–89.8] for relation extraction, significantly outperforming the seq-BiLSTM baseline at 84.6% [83.1–86.1].
• The RNNG model performs worse than the BiLSTM baseline in entity detection, with an F1 of 82.4% [80.8–83.8] versus 84.1% [82.7–85.6].
• Joint modeling in RNNG improves relation extraction performance, as training only on relations yields a lower F1 of 87.4% [85.8–88.8] compared to 88.5% with joint training.
• The architectural inductive bias of RNNG, enabling shortcuts between distant sentence parts, contributes to its superior relation extraction capability.
• The RNNG’s performance on relations is close to that of BiLSTM models for entity detection, indicating strong potential for integration in clinical NLP pipelines.
• The results suggest that RNNG’s strength lies in modeling relational structure in clinical text, particularly when syntactic and semantic dependencies span longer distances.