[Paper Review] Lightning-fast adaptive immune receptor similarity search by symmetric deletion lookup

Touchchai Chotisorayuth, Andreas Tiffeau-Mayer|arXiv (Cornell University)|Mar 14, 2024

Artificial Immune Systems Applications4 citations

TL;DR

This paper introduces XTNeighbor, a GPU-parallelized algorithm for lightning-fast adaptive immune receptor (AIR) similarity search using symmetric deletion lookup (SymDel), which accelerates Levenshtein distance-based neighbor identification by orders of magnitude. For one million sequences, XTNeighbor identifies all pairs within two edits in seconds on commodity hardware, vastly outperforming existing methods and enabling scalable cohort-level immunosequencing analysis.

ABSTRACT

An individual's adaptive immune receptor (AIR) repertoire records immune history due to the exquisite antigen specificity of AIRs. Reading this record requires computational approaches for inferring receptor function from sequence, as the diversity of possible receptor-antigen pairs vastly outstrips experimental knowledge. Identification of AIRs with similar sequence and thus putatively similar function is a common performance bottleneck in these approaches. Here, we benchmark the time complexity of five different algorithmic approaches to radius-based search for Levenshtein neighbors. We show that a symmetric deletion lookup approach, originally proposed for spell-checking, is particularly scalable. We then introduce XTNeighbor, a variant of this algorithm that can be massively parallelized on GPUs. For one million input sequences, XTNeighbor identifies all sequence neighbors that differ by up to two edits in seconds on commodity hardware, orders of magnitude faster than existing approaches. We also demonstrate how symmetric deletion lookup can speed up search with more complex sequence-similarity metrics such as TCRdist. Our contribution is poised to greatly speed up existing analysis pipelines and enable processing of large-scale immunosequencing data without downsampling.

Motivation & Objective

• To address the performance bottleneck in AIR similarity search, a common step in repertoire analysis pipelines.
• To develop a scalable, high-performance algorithm for identifying AIR sequences within a given Levenshtein distance threshold.
• To enable efficient processing of large-scale immunosequencing datasets without downsampling.
• To extend the applicability of symmetric deletion lookup—originally used in spell-checking—to the domain of adaptive immune receptor analysis.
• To demonstrate that SymDel-based search can be efficiently parallelized on GPUs for real-world immunological data workloads.

Proposed method

• Adapts symmetric deletion lookup (SymDel), a technique from spell-checking, to identify AIR sequence neighbors within a specified edit distance by generating and comparing deletion variants of both query and reference sequences.
• Uses exact hash-based matching on deletion variants to efficiently identify candidate neighbors, avoiding exhaustive pairwise comparisons.
• Introduces XTNeighbor, a GPU-optimized variant of SymDel that enables massive parallelization across thousands of threads.
• Employs a two-stage process: first, SymDel generates candidate neighbors via deletion variants; second, post-processing removes duplicates and filters pairs exceeding the edit distance threshold.
• Extends the SymDel paradigm to support complex similarity metrics like TCRdist by using SymDel as a fast prefilter.
• Implements the algorithm in both CPU and GPU versions, with the GPU version enabling sub-second search on one million sequences.

Figure 1: Symmetric deletion lookup by examples. (A) Reference sequences with insertions are identified by matches between their deletion variants and the queries. (B) References sequences with substitutions are identified by matches among deletion variants of both reference and query sequences. (C)

Experimental results

Research questions

• RQ1 Can symmetric deletion lookup be effectively adapted to accelerate AIR similarity search with high edit distance thresholds?
• RQ2 How does SymDel-based search scale compared to combinatorial lookup and other exact search methods on realistic immunosequencing datasets?
• RQ3 To what extent can SymDel be parallelized on GPUs to achieve interactive-scale performance for cohort-level AIR analysis?
• RQ4 Can SymDel be used as a fast prefilter for more complex similarity metrics such as TCRdist without sacrificing accuracy?
• RQ5 What is the performance gain of SymDel over existing tools when applied to real-world AIR datasets with varying sequence diversity and similarity thresholds?

Key findings

• XTNeighbor identifies all AIR sequence neighbors within a Levenshtein distance of two in under ten seconds for one million input sequences on commodity hardware, representing a 100-fold speed-up over baseline methods.
• The SymDel lookup algorithm achieves a ~100-fold speed-up for Levenshtein distance threshold d=2 and ~20-fold speed-up for d=3 compared to alternative approaches.
• Despite aggressive prefiltering, over 99% of sequence pairs within TCRdist ~20 and ~30 were retained at d=2 and d=3, respectively, indicating high recall and minimal information loss.
• The SymDel approach enables efficient filtering of candidate neighbors, reducing the number of pairwise comparisons by orders of magnitude while preserving accuracy.
• The GPU-parallelized XTNeighbor implementation scales effectively, making exact similarity search feasible at cohort-scale data sizes.
• The integration of SymDel as a prefilter for TCRdist-based searches maintains high sensitivity and drastically reduces downstream computation time.

Figure 2: Benchmark of AIR sequence similarity search algorithms on CPU. Time measured in seconds for near-sequence neighbor search as a function of input size for various algorithms at Levenshtein distance threshold (A) $d=1$ and (B) $d=2$ . Lines are linear fits on log-log scale. (C) Side-by-side