[Paper Review] LATTE: Latent Type Modeling for Biomedical Entity Linking

Ming Zhu, Busra Celikkaya+2 more|arXiv (Cornell University)|Nov 21, 2019

Topic Modeling26 references4 citations

TL;DR

LATTE proposes a novel neural model for biomedical entity linking that jointly learns latent fine-grained types of mentions and entities to improve disambiguation, using cross-attention and auxiliary supervision from coarse types. It achieves state-of-the-art performance on MedMentions and 3DNotes, with a 5.8% absolute gain in P@1 on MedMentions over strong baselines by modeling implicit semantic types.

ABSTRACT

Entity linking is the task of linking mentions of named entities in natural language text, to entities in a curated knowledge-base. This is of significant importance in the biomedical domain, where it could be used to semantically annotate a large volume of clinical records and biomedical literature, to standardized concepts described in an ontology such as Unified Medical Language System (UMLS). We observe that with precise type information, entity disambiguation becomes a straightforward task. However, fine-grained type information is usually not available in biomedical domain. Thus, we propose LATTE, a LATent Type Entity Linking model, that improves entity linking by modeling the latent fine-grained type information about mentions and entities. Unlike previous methods that perform entity linking directly between the mentions and the entities, LATTE jointly does entity disambiguation, and latent fine-grained type learning, without direct supervision. We evaluate our model on two biomedical datasets: MedMentions, a large scale public dataset annotated with UMLS concepts, and a de-identified corpus of dictated doctor's notes that has been annotated with ICD concepts. Extensive experimental evaluation shows our model achieves significant performance improvements over several state-of-the-art techniques.

Motivation & Objective

• To address the challenge of ambiguous biomedical entity linking due to similar-sounding terms and lack of fine-grained type information.
• To improve entity disambiguation in clinical and biomedical text where surface-level features are insufficient.
• To jointly learn latent fine-grained types of mentions and entities without direct supervision, using coarse type supervision as auxiliary guidance.
• To overcome OOV and abbreviation challenges in biomedical NLP by leveraging character-level encodings and type-based matching.
• To develop a unified framework that enhances entity linking through hierarchical type modeling and attention-based similarity scoring.

Proposed method

• LATTE uses a dual-encoder architecture with shared parameters for mention and candidate representations, incorporating word and character embeddings.
• It introduces a latent type encoder that learns implicit, fine-grained types for mentions and entities using cross-attention mechanisms.
• The model employs a distribution similarity measure between latent type distributions of mentions and candidates to guide disambiguation.
• Auxiliary known type classifiers are trained on coarse-grained UMLS or ICD types to provide supervision for latent type learning.
• A cross-attention mechanism computes context-aware similarity between mentions and candidate entities, enabling better semantic matching.
• The final entity score is computed via a feed-forward network applied to the dot product of mention and candidate representations, with type-aware attention.

Experimental results

Research questions

• RQ1 Can modeling latent fine-grained types improve biomedical entity linking performance when explicit type annotations are unavailable?
• RQ2 How effective is cross-attention in capturing semantic similarity between mentions and candidate entities in the biomedical domain?
• RQ3 Does incorporating coarse-grained known types as auxiliary supervision enhance the learning of latent types and entity linking accuracy?
• RQ4 To what extent can latent type modeling resolve challenges from OOV words and abbreviations in biomedical text?
• RQ5 How does hierarchical type modeling (latent + known types) compare to models using only surface-level features or known types alone?

Key findings

• LATTE achieves a P@1 of 88.46 on MedMentions and 87.98 on 3DNotes, outperforming all baselines and state-of-the-art models.
• Adding latent type modeling (LATTE_base+LT) improves P@1 by 6.07% on MedMentions and 2.32% on 3DNotes over the base model.
• Incorporating known type supervision (LATTE_base+KT) yields a 7.71% P@1 gain on MedMentions and 3.72% on 3DNotes, showing its strong positive effect.
• The full LATTE model, combining both latent and known type supervision, achieves the highest performance, with a 5.8% absolute improvement in P@1 on MedMentions over the strongest baseline.
• Qualitative analysis confirms that LATTE effectively handles OOV words and abbreviations by leveraging type information and character-level encodings.
• The cross-attention mechanism enables accurate disambiguation even when candidates share the same coarse type but differ in latent semantic types.