[Paper Review] Unsupervised Online Anomaly Detection On Irregularly Sampled Or Missing Valued Time-Series Data Using LSTM Networks

Oğuzhan Karaahmetoğlu, Fatih İlhan+2 more|arXiv (Cornell University)|May 25, 2020

Anomaly Detection Techniques and Applications1 references4 citations

TL;DR

This paper proposes an end-to-end unsupervised online anomaly detection framework for irregularly sampled or missing-valued time series using a novel time-modulated LSTM architecture combined with Support Vector Data Description (SVDD). The method jointly optimizes feature extraction and one-class classification via a hybrid loss, significantly outperforming standard approaches on real datasets by effectively modeling temporal dependencies under irregular sampling and missing data through gated time modulation.

ABSTRACT

We study anomaly detection and introduce an algorithm that processes variable length, irregularly sampled sequences or sequences with missing values. Our algorithm is fully unsupervised, however, can be readily extended to supervised or semisupervised cases when the anomaly labels are present as remarked throughout the paper. Our approach uses the Long Short Term Memory (LSTM) networks in order to extract temporal features and find the most relevant feature vectors for anomaly detection. We incorporate the sampling time information to our model by modulating the standard LSTM model with time modulation gates. After obtaining the most relevant features from the LSTM, we label the sequences using a Support Vector Data Descriptor (SVDD) model. We introduce a loss function and then jointly optimize the feature extraction and sequence processing mechanisms in an end-to-end manner. Through this joint optimization, the LSTM extracts the most relevant features for anomaly detection later to be used in the SVDD, hence completely removes the need for feature selection by expert knowledge. Furthermore, we provide a training algorithm for the online setup, where we optimize our model parameters with individual sequences as the new data arrives. Finally, on real-life datasets, we show that our model significantly outperforms the standard approaches thanks to its combination of LSTM with SVDD and joint optimization.

Motivation & Objective

• To address the challenge of detecting anomalies in time series with irregular sampling or missing values, where labeled data is scarce or unavailable.
• To develop a fully unsupervised, online learning framework capable of processing variable-length sequences with temporal dependencies.
• To eliminate the need for expert-driven feature selection by jointly optimizing feature extraction and anomaly classification in an end-to-end manner.
• To incorporate sampling time information into the LSTM architecture to improve modeling of temporal patterns under non-uniform sampling.
• To enable real-time adaptation through an online training algorithm using sequential gradient updates.

Proposed method

• Introduces a modified LSTM with time modulation gates (M-LSTM) that scale hidden states based on inter-arrival time intervals, enabling effective handling of irregular sampling.
• Employs a reconstruction loss during training to preserve temporal structure in sequences with missing values, improving feature representation quality.
• Uses a Support Vector Data Descriptor (SVDD) as the one-class classifier to detect anomalies based on learned temporal features.
• Jointly optimizes the LSTM and SVDD components via a composite loss function combining reconstruction loss and SVDD margin loss, enabling end-to-end training.
• Designs an online learning algorithm that updates model parameters sequentially as new sequences arrive, supporting real-time deployment.
• Applies random sequence dropping during training to simulate missing data and improve robustness to irregular sampling.

Experimental results

Research questions

• RQ1 Can a modified LSTM architecture effectively model irregularly sampled or missing-valued time series for anomaly detection without requiring labeled data?
• RQ2 How does incorporating time modulation gates into the LSTM improve performance compared to standard or additive time-injected LSTM variants?
• RQ3 To what extent does end-to-end joint optimization of feature extraction and one-class classification outperform decoupled or heuristic feature selection methods?
• RQ4 How does the proposed method scale in performance under increasing levels of missing data or irregular sampling?
• RQ5 Can the model be effectively adapted to an online learning setup for real-time anomaly detection?

Key findings

• The proposed M-LSTM model with time modulation gates significantly outperforms A-LSTM and D-LSTM variants, especially as missing data rates increase, due to better modeling of long sampling intervals.
• On the activity recognition dataset with 0.7 random drop rate, the M-LSTM achieved a clear performance advantage, demonstrating robustness to high levels of missing data.
• The joint optimization of LSTM and SVDD via a combined loss function led to superior anomaly detection performance compared to models with separate training or fixed feature extraction.
• The online training algorithm enabled sequential parameter updates, making the system suitable for real-time deployment on streaming data.
• Even with variable-length sequences and high missing data rates, the M-LSTM maintained strong AUC scores, indicating effective temporal feature learning.
• The performance gap between M-LSTM and A-LSTM widened with increasing drop rates, confirming the superiority of modulation-based time integration over additive time injection.