[Paper Review] DNA Lossless Differential Compression Algorithm based on Similarity of Genomic Sequence Database

Heba M. Afify, Muhammad Najam-ul-Islam+1 more|arXiv (Cornell University)|Sep 1, 2011

Algorithms and Data Compression3 citations

TL;DR

This paper proposes a lossless differential compression algorithm for genomic sequence databases that exploits sequence similarity by storing a reference sequence and the differences (deltas) from it, rather than individual sequences. Using an op-code table to encode differences, the method achieves up to 195-fold compression (99.4% space reduction) across three genomic datasets without requiring prior statistical knowledge of the sequence set.

ABSTRACT

Modern biological science produces vast amounts of genomic sequence data. This is fuelling the need for efficient algorithms for sequence compression and analysis. Data compression and the associated techniques coming from information theory are often perceived as being of interest for data communication and storage. In recent years, a substantial effort has been made for the application of textual data compression techniques to various computational biology tasks, ranging from storage and indexing of large datasets to comparison of genomic databases. This paper presents a differential compression algorithm that is based on production of difference sequences according to op-code table in order to optimize the compression of homologous sequences in dataset. Therefore, the stored data are composed of reference sequence, the set of differences, and differences locations, instead of storing each sequence individually. This algorithm does not require a priori knowledge about the statistics of the sequence set. The algorithm was applied to three different datasets of genomic sequences, it achieved up to 195-fold compression rate corresponding to 99.4% space saving.

Motivation & Objective

• To address the growing challenge of storing massive genomic sequence datasets efficiently.
• To reduce storage overhead in genomic databases by leveraging high sequence similarity among homologous genomes.
• To develop a compression algorithm that does not require prior knowledge of sequence statistics.
• To optimize storage by storing a reference sequence and only the differences from it.
• To achieve high compression ratios while enabling lossless reconstruction of original sequences.

Proposed method

• The algorithm selects a reference sequence from the dataset based on sequence similarity.
• It computes difference sequences (deltas) between the reference and all other sequences using an op-code table to encode variations.
• The differences are stored along with their positions in the reference sequence to enable reconstruction.
• The op-code table maps nucleotide changes (e.g., A→G) to compact codes, reducing storage for repeated patterns.
• The method avoids reliance on statistical models of the data, making it adaptable to diverse genomic datasets.
• The compressed data structure consists of the reference sequence, the list of differences with locations, and the op-code table.

Experimental results

Research questions

• RQ1 Can differential compression based on sequence similarity significantly reduce genomic data storage requirements?
• RQ2 How effective is a similarity-driven differential compression approach without prior statistical assumptions about the data?
• RQ3 What compression ratios can be achieved on real genomic sequence datasets using this method?
• RQ4 Can the algorithm maintain lossless reconstruction while achieving high compression on homologous sequences?
• RQ5 How does the op-code table design impact compression efficiency in genomic data?

Key findings

• The algorithm achieved a maximum compression ratio of 195-fold on tested genomic datasets.
• The method resulted in up to 99.4% reduction in storage space compared to storing sequences individually.
• The approach does not require prior knowledge of sequence statistics, enhancing its general applicability.
• The use of an op-code table for encoding differences significantly improved compression efficiency.
• The method was validated on three distinct genomic sequence datasets, demonstrating consistent high performance.
• Lossless reconstruction of original sequences is fully supported through reference and delta sequence recovery.