[Paper Review] ODE - Augmented Training Improves Anomaly Detection in Sensor Data from Machines

Mohit Yadav, Pankaj Malhotra+3 more|arXiv (Cornell University)|May 5, 2016

Anomaly Detection Techniques and ApplicationsComputer Science8 references21 citations

TL;DR

This paper proposes ODE-augmented training to improve anomaly detection in sensor data from dynamical machines, using domain-knowledge ODEs to generate synthetic control input variations and augment real normal data. The method significantly boosts LSTM-AD performance, increasing F-score from 0.49 (on small real data) to 0.64 when augmented with ODE-generated data, enabling better detection of anomalies under novel control inputs.

ABSTRACT

Machines of all kinds from vehicles to industrial equipment are increasingly instrumented with hundreds of sensors. Using such data to detect anomalous behaviour is critical for safety and efficient maintenance. However, anomalies occur rarely and with great variety in such systems, so there is often insufficient anomalous data to build reliable detectors. A standard approach to mitigate this problem is to use one class methods relying only on data from normal behaviour. Unfortunately, even these approaches are more likely to fail in the scenario of a dynamical system with manual control input(s). Normal behaviour in response to novel control input(s) might look very different to the learned detector which may be incorrectly detected as anomalous. In this paper, we address this issue by modelling time-series via Ordinary Differential Equations (ODE) and utilising such an ODE model to simulate the behaviour of dynamical systems under varying control inputs. The available data is then augmented with data generated from the ODE, and the anomaly detector is retrained on this augmented dataset. Experiments demonstrate that ODE-augmented training data allows better coverage of possible control input(s) and results in learning more accurate distinctions between normal and anomalous behaviour in time-series.

Motivation & Objective

• To address the challenge of insufficient normal training data with limited variation in manual control inputs for anomaly detection in dynamical systems.
• To improve the robustness of anomaly detectors by simulating diverse control input scenarios not present in real data.
• To leverage domain-knowledge ODE models as generative tools to synthesize realistic normal time-series data for data augmentation.
• To evaluate whether ODE-augmented training improves the generalization and accuracy of LSTM-based anomaly detection models.
• To demonstrate that ODE-generated data enhances detection of anomalies that arise due to novel or unseen control inputs.

Proposed method

• Model the dependent sensor variables (e.g., coolant temperature) as a function of control inputs (e.g., accelerator pedal position) using an ODE with learnable parameters.
• Estimate ODE parameters using maximum likelihood estimation over time windows of real sensor data, leveraging domain knowledge on ODE structure.
• Generate synthetic time-series data by sampling novel control input values from learned statistical distributions and simulating the ODE response.
• Augment the original real normal training data with ODE-generated synthetic data to form a larger, more diverse training set.
• Retrain the LSTM-AD anomaly detection model on the augmented dataset, using likelihood scores from prediction errors and a threshold optimized via F-score maximization.
• Use a Gaussian likelihood model on prediction errors to compute anomaly scores, with thresholds selected to maximize F-score on validation data.

Experimental results

Research questions

• RQ1 Can ODE-based data generation improve anomaly detection performance when only limited real normal data is available?
• RQ2 Does augmenting training data with ODE-simulated control input variations lead to better detection of anomalies under novel control inputs?
• RQ3 How does ODE-augmented training compare to training on real data alone in terms of precision, recall, and F-score?
• RQ4 Can ODE models effectively simulate realistic normal behavior across diverse control input conditions not present in the original data?
• RQ5 Does the inclusion of ODE-generated data reduce false positives when the system encounters new but normal control inputs?

Key findings

• ODE-augmented training increased the F-score from 0.49 (on small real data) to 0.64, demonstrating significant improvement in anomaly detection performance.
• The precision improved from 0.34 to 0.52 when small real data was augmented with ODE-generated data, indicating fewer false positives.
• The model trained on ODE-augmented data successfully detected a type 1 anomaly (short zero-value spike) that was missed when trained only on small real data.
• The F-score curve in Figure 3 shows consistent improvement as ODE-generated data was progressively added to the small real dataset, confirming the benefit of data diversity.
• Larger real datasets (L(r)) also benefited from augmentation, with F-score increasing from 0.55 to 0.67 when ODE-generated data was added.
• The method effectively expanded coverage of normal behavior under varied control inputs, reducing the risk of misclassifying novel but normal control inputs as anomalies.