[Paper Review] Pangloss: Fast Entity Linking in Noisy Text Environments

Michael Conover, Matthew Hayes+3 more|arXiv (Cornell University)|Jul 16, 2018

Topic Modeling39 references4 citations

TL;DR

Pangloss is a production-ready entity linking system that achieves state-of-the-art F1 performance (>5% improvement) on noisy text by combining probabilistic key phrase identification with context-dependent document embeddings. It leverages a tiered, embedded database architecture to enable low-latency, on-device disambiguation with 6% of Stanford CoreNLP's memory usage and twice the throughput.

ABSTRACT

Entity linking is the task of mapping potentially ambiguous terms in text to their constituent entities in a knowledge base like Wikipedia. This is useful for organizing content, extracting structured data from textual documents, and in machine learning relevance applications like semantic search, knowledge graph construction, and question answering. Traditionally, this work has focused on text that has been well-formed, like news articles, but in common real world datasets such as messaging, resumes, or short-form social media, non-grammatical, loosely-structured text adds a new dimension to this problem. This paper presents Pangloss, a production system for entity disambiguation on noisy text. Pangloss combines a probabilistic linear-time key phrase identification algorithm with a semantic similarity engine based on context-dependent document embeddings to achieve better than state-of-the-art results (>5% in F1) compared to other research or commercially available systems. In addition, Pangloss leverages a local embedded database with a tiered architecture to house its statistics and metadata, which allows rapid disambiguation in streaming contexts and on-device disambiguation in low-memory environments such as mobile phones.

Motivation & Objective

• To address the challenge of entity linking in real-world, noisy text such as instant messages, resumes, and social media, where traditional systems fail due to poor grammar and loose structure.
• To develop a scalable, low-latency entity linking system suitable for production deployment across diverse domains like workplace messaging, source code commits, and collaborative documentation.
• To reduce memory pressure and improve inference efficiency in resource-constrained environments like mobile devices without sacrificing performance.
• To bridge the gap between academic research and industrial deployment by integrating algorithmic innovation with practical, memory-efficient system architecture.

Proposed method

• Uses a probabilistic linear-time tokenization algorithm to identify key phrases in noisy, unstructured text, improving robustness over traditional tokenizers in low-quality inputs.
• Employs context-dependent document embeddings to compute multiple semantic representations per document, capturing distinct themes in multi-topic texts.
• Applies a semantic similarity engine based on these embeddings to disambiguate named entities by matching them to knowledge base entries.
• Stores metadata and statistics in a local embedded database with a tiered caching architecture to ensure data locality and reduce memory pressure at inference time.
• Optimizes for low-latency inference by minimizing resident memory usage through selective caching and efficient metadata access patterns.

Experimental results

Research questions

• RQ1 How can entity linking performance be significantly improved in noisy, low-resource text environments such as workplace messaging and social media?
• RQ2 To what extent can a system achieve high F1 performance while maintaining low memory and latency in production-scale deployments?
• RQ3 Can a tiered, embedded database architecture effectively reduce memory pressure in entity linking systems without degrading performance?
• RQ4 How does probabilistic tokenization outperform traditional tokenization in disambiguating named entities in poorly formatted text?

Key findings

• Pangloss achieves over 5% higher F1 score than other research and commercial entity linking systems on standard benchmarks, with at least 15% better performance on a manually annotated dataset of workplace messaging.
• On the Wikipedia pages benchmark, Pangloss achieves 3.9 entity links per second, outperforming Stanford CoreNLP’s 2.0 links per second, representing nearly double the throughput.
• Pangloss uses only 6% of the resident memory (0.345 GB vs 6.02 GB) and 31% of the virtual memory (3.07 GB vs 10.1 GB) of Stanford CoreNLP, demonstrating significant memory efficiency.
• The system maintains high performance in low-memory environments, enabling on-device entity linking on mobile platforms through its embedded, tiered metadata storage design.
• The combination of probabilistic tokenization and context-aware embeddings leads to more accurate disambiguation, particularly in short, ambiguous, or loosely structured text.