[Paper Review] AliBI: An Alignment-Based Index for Genomic Datasets

Héctor Ferrada, Travis Gagie+2 more|arXiv (Cornell University)|Jul 24, 2013

Algorithms and Data Compression23 references3 citations

TL;DR

AliBI proposes a novel alignment-based index (AliBI) for genomic datasets that leverages pre-aligned human genomes to improve space efficiency and random access performance over traditional FM-indexes. By integrating alignment information directly into the index structure and replacing LZ77 parsing with alignment-based filtering, AliBI reduces preprocessing overhead and enables faster approximate pattern matching while maintaining compact storage.

ABSTRACT

With current hardware and software, a standard computer can now hold in RAM an index for approximate pattern matching on about half a dozen human genomes. Sequencing technologies have improved so quickly, however, that scientists will soon demand indexes for thousands of genomes. Whereas most researchers who have addressed this problem have proposed completely new kinds of indexes, we recently described a simple technique that scales standard indexes to work on more genomes. Our main idea was to filter the dataset with LZ77, build a standard index for the filtered file, and then create a hybrid of that standard index and an LZ77-based index. In this paper we describe how to our technique to use alignments instead of LZ77, in order to simplify and speed up both preprocessing and random access.

Motivation & Objective

• Address the growing challenge of indexing thousands of human genomes efficiently as sequencing becomes cheaper and more widespread.
• Overcome the space inefficiency of standard FM-indexes, which scale nearly linearly with the number of genomes due to limited exploitation of genomic repetitiveness.
• Develop a hybrid indexing approach that retains compatibility with existing FM-index infrastructure while scaling to large-scale genomic datasets.
• Reduce preprocessing time and improve random access performance by replacing LZ77 parsing with alignment-based filtering.
• Integrate alignment metadata directly into the index to exploit structural redundancy in aligned genomes without storing alignments separately.

Proposed method

• Use pre-aligned genomic sequences (aligned to a reference) as input, assuming a known upper bound on pattern length (M) and edit distance (K).
• Mark all characters within distance M+K−1 of mismatches (SNPs or indels) in aligned genomes and mark all reference genome characters.
• Extend unmarked regions by M+K−1 characters on each side to ensure secondary occurrences are fully contained in unmarked zones.
• Concatenate all marked substrings (including the reference) with K+1 '#' characters as separators, storing each distinct substring only once with associated pointers.
• Build a standard FM-index on the concatenated string to support fast approximate pattern matching and random access.
• Use a 2-sided range reporting data structure on a grid (bounded by reference genome size) to map primary matches in the reference to secondary matches in aligned genomes, leveraging positional consistency across genomes.

Experimental results

Research questions

• RQ1 Can alignment information be used to replace LZ77 parsing in hybrid genomic indexes to reduce preprocessing cost and improve performance?
• RQ2 Does integrating alignment metadata into the index structure lead to significant space savings compared to storing alignments separately?
• RQ3 Can the use of alignment-based filtering simplify and accelerate random access operations in approximate pattern matching?
• RQ4 How does the performance of an alignment-based index compare to LZ77-based or standard FM-index approaches in terms of index size, preprocessing time, and query speed?
• RQ5 To what extent can the structural similarity of aligned genomes be exploited to compress the range reporting data structure used for secondary match recovery?

Key findings

• AliBI replaces LZ77 parsing with alignment-based filtering, eliminating the need to compute a full LZ77 parse and reducing preprocessing time.
• The index integrates alignment information directly, avoiding separate storage of alignments and reducing space usage through structural exploitation.
• Random access performance is expected to improve due to the elimination of LZ77 parsing and the use of alignment metadata for faster match resolution.
• The range reporting data structure is bounded by the size of the reference genome rather than the full dataset, enabling more efficient space usage and faster queries.
• Secondary matches can be efficiently recovered from primary matches using a compressed 2-sided range reporting structure, leveraging positional consistency across aligned genomes.
• Preliminary expectations suggest AliBI will be significantly smaller and faster than standard FM-indexes when scaling to thousands of genomes.