[Paper Review] Biomedical Entity Linking with Contrastive Context Matching

Shogo Ujiie, Hayate Iso+1 more|arXiv (Cornell University)|Jun 14, 2021

Biomedical Text Mining and Ontologies23 references4 citations

TL;DR

BioCoM is a contrastive learning framework for biomedical entity linking that uses only a small dictionary and raw PubMed articles to train a context-aware model. By constructing training instances from dictionary-matched sentences and leveraging contrastive learning on contextual representations, BioCoM achieves state-of-the-art performance, especially in low-resource settings, outperforming prior methods by significant margins on three benchmark datasets.

ABSTRACT

We introduce BioCoM, a contrastive learning framework for biomedical entity linking that uses only two resources: a small-sized dictionary and a large number of raw biomedical articles. Specifically, we build the training instances from raw PubMed articles by dictionary matching and use them to train a context-aware entity linking model with contrastive learning. We predict the normalized biomedical entity at inference time through a nearest-neighbor search. Results found that BioCoM substantially outperforms state-of-the-art models, especially in low-resource settings, by effectively using the context of the entities.

Motivation & Objective

• To develop a biomedical entity normalization method that functions effectively under low-resource conditions, particularly when manually annotated data is unavailable.
• To address the challenge of linking disease mentions to unified concepts in biomedical knowledge graphs despite limited synonym coverage in small dictionaries.
• To improve entity linking performance by leveraging contextual cues from surrounding text, even when direct synonym matches are absent.
• To eliminate reliance on large annotated datasets or extensive knowledge graphs, making the method practical for real-world medical NLP applications.
• To demonstrate that context-aware representations learned from raw text can significantly enhance entity linking accuracy, especially when resources are constrained.

Proposed method

• Construct training instances by matching entity mentions from raw PubMed articles to a small biomedical dictionary, forming a corpus of entity-linked sentences.
• Use contrastive learning to train a context-aware model that learns to distinguish correct entity-concept pairs from negative pairs using contextual representations.
• Encode entity mentions and their surrounding sentences using PubMedBERT, extracting contextual representations from the [ENT] token embedding.
• At inference, normalize input entities via nearest-neighbor search in the embedding space using the learned contextual representations.
• Leverage the longest match algorithm to identify entity mentions in sentences, ensuring alignment between input mentions and dictionary entries.
• Optimize hyperparameters such as the number of nearest neighbors (k=15) using development set performance.

Experimental results

Research questions

• RQ1 Can a biomedical entity linking model achieve state-of-the-art performance using only a small dictionary and raw PubMed articles, without any manually annotated data?
• RQ2 How does the inclusion of contextual information from surrounding sentences improve entity linking accuracy, especially when synonyms are missing from the dictionary?
• RQ3 Does contrastive learning on automatically constructed, dictionary-matched sentences lead to better contextual representations than existing self-supervised or supervised baselines?
• RQ4 How does model performance degrade or remain stable when the number of synonyms in the dictionary is reduced, simulating low-resource scenarios?
• RQ5 To what extent can context-matching via nearest-neighbor search in the embedding space outperform models relying solely on synonym matching or fixed embeddings?

Key findings

• BioCoM achieved 60.3% accuracy on NCBID, 69.0% on BC5CDR-d, and 71.4% on MedMentions-d, significantly outperforming all baselines, including SapBert and PubMedBERT.
• On MedMentions-d, BioCoM achieved 71.4% accuracy, a 6.3-point improvement over SapBert (65.1%) and a 26.2-point improvement over PubMedBERT (45.2%).
• When the dictionary size was reduced to 39,959 synonyms, BioCoM maintained strong performance, demonstrating superior robustness in low-resource settings compared to other models.
• Qualitative analysis showed that BioCoM correctly linked 'B-ALL' to 'B-cell acute lymphoblastic leukemia' by leveraging contextual cues like 'genetic translocations' in the sentence, while SapBert failed due to lack of context awareness.
• An error in one prediction was traced to a mismatch caused by the longest-match algorithm, where 'carcinoma, non-small-cell lung' was incorrectly matched instead of 'renal cell carcinomas', indicating a limitation in mention recognition.
• The results confirm that context-aware representations learned through contrastive learning from raw text are highly effective for biomedical entity linking, even without large-scale annotations or extensive knowledge graphs.