[Paper Review] Employing Subsequence Matching in Audio Data Processing

Petr Volný, David Novák+1 more|arXiv (Cornell University)|Apr 11, 2012

Music and Audio Processing38 references3 citations

TL;DR

This paper proposes a Subsequence Matching Framework (SMF) built on MESSIF to accelerate similarity search in audio data processing, particularly for unsupervised spoken term detection. By leveraging metric indexes like M-index with edit-distance-based functions (e.g., ERD, ERP), the framework overcomes the computational limitations of DTW, enabling faster, scalable query-by-example speech retrieval while supporting flexible algorithm prototyping without recompilation.

ABSTRACT

We overview current problems of audio retrieval and time-series subsequence matching. We discuss the usage of subsequence matching approaches in audio data processing, especially in automatic speech recognition (ASR) area and we aim at improving performance of the retrieval process. To overcome the problems known from the time-series area like the occurrence of implementation bias and data bias we present a Subsequence Matching Framework as a tool for fast prototyping, building, and testing similarity search subsequence matching applications. The framework is build on top of MESSIF (Metric Similarity Search Implementation Framework) and thus the subsequence matching algorithms can exploit advanced similarity indexes in order to significantly increase their query processing performance. To prove our concept we provide a design of query-by-example spoken term detection type of application with the usage of phonetic posteriograms and subsequence matching approach.

Motivation & Objective

• To address performance bottlenecks in time-series and audio similarity search, especially the high computational cost of DTW in unsupervised speech analysis.
• To reduce reliance on expensive, large-scale transcribed data by enabling efficient, prototype-friendly subsequence matching in low-resource or out-of-vocabulary speech scenarios.
• To develop a modular, extensible framework that supports rapid experimentation with different subsequence matching algorithms and distance functions.
• To enable efficient similarity search in audio data by integrating subsequence matching with advanced metric indexes like M-index.
• To demonstrate the feasibility and performance gains of the framework in a real-world query-by-example spoken term detection application.

Proposed method

• The Subsequence Matching Framework (SMF) is built on top of the MESSIF metric similarity search framework, enabling integration with existing metric indexes such as M-index.
• The framework supports modular design, allowing runtime configuration of components like distance functions, feature extractors, and indexing strategies without recompilation.
• It employs phonetic posteriograms—time-series representations of phoneme posterior probabilities—as input features for subsequence matching.
• The framework replaces traditional DTW with edit-distance-based metrics (e.g., ERD, ERP) that are compatible with metric space indexing, improving query performance.
• It enables range queries and KNN queries through MESSIF’s built-in query interface, supporting distributed and multi-layered indexing for large-scale audio data.
• The system is implemented in Java using reflection and RMI, and supports web-based UI integration via MESSIF-UI for demo and evaluation purposes.

Experimental results

Research questions

• RQ1 Can subsequence matching with metric-compatible distance functions improve the performance of spoken term detection compared to DTW-based approaches?
• RQ2 How effectively can a modular, extensible framework accelerate prototyping and testing of subsequence matching algorithms in audio retrieval?
• RQ3 To what extent can metric indexes like M-index be leveraged to speed up similarity search in audio data when using non-DTW distance functions?
• RQ4 Can the framework support efficient query-by-example spoken term detection without requiring large-scale transcribed training data?
• RQ5 What are the performance and accuracy trade-offs when using edit-distance-based metrics (e.g., ERD, ERP) instead of DTW in subsequence matching for speech?

Key findings

• The Subsequence Matching Framework successfully enables fast, scalable similarity search in audio data by replacing DTW with metric-compatible distance functions such as ERD and ERP.
• The framework supports rapid prototyping and testing of different algorithmic combinations through modular, configuration-driven design without recompilation.
• By leveraging MESSIF’s M-index, the framework achieves significant performance improvements over DTW-based approaches, which are not natively indexable in metric spaces.
• The use of phonetic posteriograms as feature representation allows effective subsequence matching for spoken term detection, even in low-resource language scenarios.
• Preliminary evaluation on TIMIT and NIST STD datasets confirms that the framework achieves comparable detection quality to DTW-based methods while enabling faster query processing through indexing.
• The framework demonstrates feasibility for approximate search in spoken term detection, opening avenues for further optimization in large-scale audio retrieval.