[Paper Review] ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

Omar Khattab, Matei Zaharia|arXiv (Cornell University)|Apr 27, 2020

Topic Modeling37 references189 citations

TL;DR

ColBERT introduces a late interaction architecture that encodes queries and documents with BERT separately and uses a cheap MaxSim-based interaction to score relevance, enabling offline document indexing and end-to-end retrieval with strong accuracy and large speedups over prior BERT-based rankers.

ABSTRACT

Recent progress in Natural Language Understanding (NLU) is driving fast-paced advances in Information Retrieval (IR), largely owed to fine-tuning deep language models (LMs) for document ranking. While remarkably effective, the ranking models based on these LMs increase computational cost by orders of magnitude over prior approaches, particularly as they must feed each query-document pair through a massive neural network to compute a single relevance score. To tackle this, we present ColBERT, a novel ranking model that adapts deep LMs (in particular, BERT) for efficient retrieval. ColBERT introduces a late interaction architecture that independently encodes the query and the document using BERT and then employs a cheap yet powerful interaction step that models their fine-grained similarity. By delaying and yet retaining this fine-granular interaction, ColBERT can leverage the expressiveness of deep LMs while simultaneously gaining the ability to pre-compute document representations offline, considerably speeding up query processing. Beyond reducing the cost of re-ranking the documents retrieved by a traditional model, ColBERT's pruning-friendly interaction mechanism enables leveraging vector-similarity indexes for end-to-end retrieval directly from a large document collection. We extensively evaluate ColBERT using two recent passage search datasets. Results show that ColBERT's effectiveness is competitive with existing BERT-based models (and outperforms every non-BERT baseline), while executing two orders-of-magnitude faster and requiring four orders-of-magnitude fewer FLOPs per query.

Motivation & Objective

• Address the efficiency bottleneck of BERT-based ranking for passage retrieval.
• Propose a late interaction mechanism that separates query and document encoding to enable offline indexing.
• Leverage MaxSim-based interactions to retain fine-grained matching while enabling vector-indexed retrieval.
• Demonstrate end-to-end retrieval capabilities from large collections without sacrificing effectiveness.

Proposed method

• Encode queries and documents with shared BERT-based encoders to produce contextualized bags of embeddings.
• Prepend special tokens to distinguish queries [Q] and documents [D] and apply a linear projection to obtain fixed-size embeddings.
• Apply a late interaction score as a sum of maximum similarities (MaxSim) between query and document embeddings, using cosine similarity (via normalization) or squared L2 distance.
• Train end-to-end with a pairwise loss over (query, positive doc, negative doc) using Adam optimization.
• Index document embeddings offline to enable fast retrieval and support vector-indexed end-to-end retrieval with FAISS.
• For end-to-end retrieval, use a two-stage pruning approach with FAISS IVFPQ to filter candidates before exact ColBERT scoring.

Experimental results

Research questions

• RQ1 How close can ColBERT come to the effectiveness of full BERT-based rankers while reducing latency significantly?
• RQ2 Can ColBERT support end-to-end retrieval from large collections using vector similarity indexes?
• RQ3 What is the contribution of late interaction and query augmentation to ColBERT's performance?
• RQ4 What are the offline indexing costs and memory footprints for ColBERT in practical settings?

Key findings

• ColBERT achieves competitive effectiveness with non-BERT baselines and improves over many existing neural rankers while offering substantial speedups.
• In re-ranking setups, ColBERT delivers up to about 170x speedup and orders-of-magnitude fewer FLOPs compared to standard BERT-based rankers (ColBERT (re-rank) vs. BERT baselines).
• End-to-end retrieval with ColBERT (L2) achieves 36.0 MRR@10 on MS MARCO with an end-to-end latency of 458 ms, and Recall@50/200/1000 are 82.9/92.3/96.8 respectively, demonstrating practical end-to-end performance.
• Document indexing to compute embeddings is feasible on a single server with four GPUs (MS MARCO scale) within a few hours, with modest space footprint (tens of GiBs).
• A thorough ablation study shows that late interaction, MaxSim, and encoder design choices are essential for ColBERT’s effectiveness.