[Paper Review] A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques

Jimmy Lin, Xueguang Ma|arXiv (Cornell University)|Jun 28, 2021

Information Retrieval and Search Behavior22 references68 citations

TL;DR

The paper presents a two-dimensional framework for IR techniques: sparse vs. dense representations and unsupervised vs. learned representations, analyzes DeepImpact, COIL, and uniCOIL within this framework, and reports experiments on MS MARCO showing uniCOIL achieving state-of-the-art in sparse retrieval.

ABSTRACT

Recent developments in representational learning for information retrieval can be organized in a conceptual framework that establishes two pairs of contrasts: sparse vs. dense representations and unsupervised vs. learned representations. Sparse learned representations can further be decomposed into expansion and term weighting components. This framework allows us to understand the relationship between recently proposed techniques such as DPR, ANCE, DeepCT, DeepImpact, and COIL, and furthermore, gaps revealed by our analysis point to "low hanging fruit" in terms of techniques that have yet to be explored. We present a novel technique dubbed "uniCOIL", a simple extension of COIL that achieves to our knowledge the current state-of-the-art in sparse retrieval on the popular MS MARCO passage ranking dataset. Our implementation using the Anserini IR toolkit is built on the Lucene search library and thus fully compatible with standard inverted indexes.

Motivation & Objective

• Propose a conceptual framework to organize recent IR techniques along sparse/dense and unsupervised/learned axes.
• Analyze how DeepCT, DeepImpact, and COIL fit into this framework and identify gaps.
• Introduce uniCOIL and evaluate its performance on MS MARCO passage ranking.
• Show how document expansion and term weighting components contribute to sparse retrieval performance.
• Discuss implications for design choices, indexes, and future research directions.

Proposed method

• Organize existing IR techniques into a 2x2 framework (dense vs sparse; supervised vs unsupervised).
• Decompose learned sparse methods into expansion (document expansion) and term-weighting components.
• Experiment with variants of COIL and uniCOIL using Anserini/Lucene based inverted indexes.
• Quantify performance on MS MARCO passage ranking and compare with dense baselines and hybrids.
• Perform ablation-like comparisons by separating expansion and weighting components (e.g., doc2query–T5, DeepCT, DeepImpact, COIL variants).
• Demonstrate that uniCOIL with doc2query–T5 achieves competitive to state-of-the-art sparse retrieval results.

Experimental results

Research questions

• RQ1 How can we conceptually categorize recent IR techniques (dense vs sparse, supervised vs unsupervised) and what insights emerge from this categorization?
• RQ2 What are the contributions of document expansion vs. term weighting in learned sparse retrieval?
• RQ3 Can a simple extension of COIL (uniCOIL) achieve state-of-the-art results in sparse retrieval on MS MARCO?
• RQ4 What are the tradeoffs between dense, sparse, and hybrid retrieval approaches in terms of effectiveness, index size, and compatibility with inverted indexes?

Key findings

• A two-axis framework helps relate DPR, ANCE, DeepCT, DeepImpact, and COIL.
• Expansion (doc2query–T5) is critical for sparse representations to address vocabulary mismatch.
• uniCOIL (with appropriate constraints and doc2query–T5) achieves state-of-the-art sparse retrieval among learned-impact methods on MS MARCO.
• COIL-tok with 32-dim tokens shows strong results but uniCOIL can match or exceed with expansions.
• Dense methods (e.g., RocketQA) still lead overall, but dense–sparse hybrids can outperform either alone.
• Inverted-index-friendly variants (uniCOIL) demonstrate practical advantages in index size and compatibility.