[Paper Review] Adversarial attacks against Bayesian forecasting dynamic models

Roi Naveiro|arXiv (Cornell University)|Oct 20, 2021

Adversarial Robustness in Machine Learning9 references4 citations

TL;DR

This paper proposes a decision analysis-based adversarial attack strategy targeting Bayesian forecasting dynamic models, where an attacker manipulates historical data to shift the defender’s optimal decision without triggering detection. By optimizing data perturbations to minimize cumulative Bayes factors while driving the defender’s decision away from the true optimum, the method achieves stealthy, effective manipulation—demonstrated by reducing optimal inventory from 136 to 116 units in a real-world example without raising alarms.

ABSTRACT

The last decade has seen the rise of Adversarial Machine Learning (AML). This discipline studies how to manipulate data to fool inference engines, and how to protect those systems against such manipulation attacks. Extensive work on attacks against regression and classification systems is available, while little attention has been paid to attacks against time series forecasting systems. In this paper, we propose a decision analysis based attacking strategy that could be utilized against Bayesian forecasting dynamic models.

Motivation & Objective

• To address the lack of adversarial attack research on time series forecasting systems, particularly Bayesian dynamic models.
• To develop a strategy that manipulates data to alter the defender’s final decision, rather than merely shifting predictions.
• To formalize 'subtle' attacks not by perturbation size but by minimizing cumulative Bayes factors, ensuring detection avoidance.
• To demonstrate the feasibility and stealth of such attacks in a realistic Bayesian dynamic model setting.

Proposed method

• Formulates the attacker’s goal as changing the defender’s optimal decision by manipulating historical observations from time $\alpha - h$ to $\alpha$.
• Uses decision analysis to model the defender’s utility function and posterior predictive distribution, defining the optimal decision as the one maximizing expected utility.
• Constructs an attack by solving an optimization problem that minimizes the distance between the defender’s predicted decision and the attacker’s target, subject to constraints on data perturbations.
• Employs the cumulative Bayes factor as a detection metric: attacks are considered stealthy if the minimum cumulative Bayes factor remains close to that of clean data.
• Applies the method to a conditionally Poisson dynamic model for demand forecasting in a supermarket inventory context.
• Compares two attack strategies: one minimizing perturbation size (L2 norm), and another minimizing Bayes factor deviation, to evaluate stealth and effectiveness.

Experimental results

Research questions

• RQ1 How can an adversary manipulate historical data to alter the optimal decision in a Bayesian forecasting dynamic model without being detected?
• RQ2 What is the most effective way to formalize 'subtle' attacks—by limiting perturbation size or by minimizing detection signals such as cumulative Bayes factors?
• RQ3 Can an attacker successfully reduce the optimal inventory decision in a real-world forecasting scenario while avoiding detection by model monitoring systems?
• RQ4 How do different attack objectives (e.g., minimizing perturbation vs. minimizing detection) affect the feasibility and stealth of adversarial manipulation?
• RQ5 What role does gradual data manipulation play in evading statistical monitoring mechanisms in time series models?

Key findings

• The attack successfully reduced the optimal inventory decision from 136 to 116 units, achieving the attacker’s goal of a 20% reduction.
• When optimizing for stealth via cumulative Bayes factors, the attack maintained a minimum cumulative Bayes factor indistinguishable from that of clean data, indicating no detection.
• In contrast, the L2-norm constrained attack triggered detection, as evidenced by a significant drop in the minimum cumulative Bayes factor.
• The stealthy attack required gradual reduction of demand in prior weekends, enabling the model to adapt naturally and avoid suspicion.
• The study demonstrates that decision-focused attacks are more effective and stealthy than prediction-focused ones in Bayesian dynamic models.
• The results validate that cumulative Bayes factors serve as a reliable metric for detecting adversarial data manipulation in sequential Bayesian models.