[Paper Review] Piecewise Linear Approximation in Data Streaming: Algorithmic Implementations and Experimental Analysis

Romaric Duvignau, Vincenzo Gulisano+2 more|arXiv (Cornell University)|Aug 27, 2018

Time Series Analysis and Forecasting16 references4 citations

TL;DR

This paper proposes a streaming-aware framework for piecewise linear approximation (PLA) in real-time data processing, introducing optimized compression protocols and a novel heuristic to reduce latency and error. It demonstrates that rethinking PLA output representation and protocol design leads to significant improvements in reconstruction delay and precision—especially under dynamic streaming conditions—without sacrificing compression efficiency.

ABSTRACT

Piecewise Linear Approximation (PLA) is a well-established tool to reduce the size of the representation of time series by approximating the series by a sequence of line segments while keeping the error introduced by the approximation within some predetermined threshold. With the recent rise of edge computing, PLA algorithms find a complete new set of applications with the emphasis on reducing the volume of streamed data. In this study, we identify two scenarios set in a data-stream processing context: data reduction in sensor transmissions and datacenter storage. In connection to those scenarios, we identify several streaming metrics and propose streaming protocols as algorithmic implementations of several state of the art PLA techniques. In an experimental evaluation, we measure the quality of the reviewed methods and protocols and evaluate their performance against those streaming statistics. All known methods have deficiencies when it comes to handling streaming-like data, e.g. inflation of the input stream, high latency or poor average error. Our experimental results highlight the challenges raised when transferring those classical methods into the stream processing world and present alternative techniques to overcome them and balance the related trade-offs.

Motivation & Objective

• Address the shortcomings of classical PLA methods in real-time streaming, where high latency, data inflation, and poor error control undermine practical use.
• Reframe PLA evaluation beyond compression ratio by introducing streaming-specific metrics: data reduction, reconstruction delay, and precision loss.
• Design and evaluate streaming protocols that optimize the transmission and storage of PLA-compressed data in edge and datacenter scenarios.
• Propose a new heuristic for PLA computation that prioritizes low individual errors while maintaining competitive compression rates.
• Demonstrate that small protocol and algorithmic changes can significantly improve trade-offs between compression, latency, and accuracy in streaming workloads.

Proposed method

• Define three streaming metrics: compression ratio, reconstruction delay (in number of segments), and average error (ℓ₂-norm) to evaluate PLA performance.
• Introduce 'compression protocols'—specific algorithmic implementations of PLA techniques that govern how and when line segments are output during streaming.
• Use a dynamic programming approach to minimize the 'size' of the PLA representation, counting joint knots as 2 bytes and disjoint knots as 3 bytes, moving beyond segment count as a metric.
• Apply a sign-trick (storing −x instead of x) to reduce storage overhead without extra bits during streaming.
• Extend existing PLA methods (e.g., from [10, 17, 18]) with new output protocols to reduce latency and improve error control.
• Measure actual byte consumption of compression records and evaluate performance on real-world timestamped data streams, not just synthetic or benchmark datasets like UCR.

Experimental results

Research questions

• RQ1 How do classical PLA methods perform in real-world streaming scenarios, particularly regarding reconstruction delay and error inflation?
• RQ2 To what extent can protocol-level design choices (e.g., when to output segments) improve the latency and precision of PLA-compressed streams?
• RQ3 Can a new heuristic for PLA computation reduce individual approximation errors without degrading compression efficiency?
• RQ4 Is there a meaningful trade-off between compression ratio, reconstruction delay, and error that can be optimized through protocol and algorithm design?
• RQ5 How do existing PLA methods behave under sudden data changes, and can adaptive strategies mitigate data inflation?

Key findings

• Classical PLA methods often fail in streaming contexts by inflating data volume, introducing high reconstruction delays (up to 4 segments behind), and producing higher-than-necessary individual errors.
• The proposed compression protocols significantly reduce reconstruction delay—achieving outputs within 2 to 4 segments of input processing—compared to baseline methods.
• The new heuristic method reduces average ℓ₂-error compared to standard approaches while maintaining competitive compression ratios.
• A small reduction in compression ratio can lead to substantial improvements in both delay and error, indicating that high compression should not be the sole optimization target.
• The dynamic programming-based size metric (counting knots as 2 or 3 bytes) provides a more accurate measure of storage cost than segment count alone.
• The experimental framework with real-world datasets shows that UCR benchmark data is not representative of streaming workloads due to its heterogeneity, and real-world data reveals more nuanced performance trade-offs.