[Paper Review] Explaining RADAR features for detecting spoofing attacks in Connected Autonomous Vehicles

Nidhi Rastogi, Sara Rampazzi+4 more|arXiv (Cornell University)|Mar 1, 2022

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper proposes a Dempster-Shafer Theory-based framework to explain radar sensor data uncertainty for detecting spoofing attacks in connected autonomous vehicles (CAVs). By quantifying belief, plausibility, and uncertainty in radar features, the method improves ML model robustness, achieving 96% accuracy in classifying obstacle states and detecting all 11 spoofed packets as stationary with high confidence in mass functions.

ABSTRACT

Connected autonomous vehicles (CAVs) are anticipated to have built-in AI systems for defending against cyberattacks. Machine learning (ML) models form the basis of many such AI systems. These models are notorious for acting like black boxes, transforming inputs into solutions with great accuracy, but no explanations support their decisions. Explanations are needed to communicate model performance, make decisions transparent, and establish trust in the models with stakeholders. Explanations can also indicate when humans must take control, for instance, when the ML model makes low confidence decisions or offers multiple or ambiguous alternatives. Explanations also provide evidence for post-incident forensic analysis. Research on explainable ML to security problems is limited, and more so concerning CAVs. This paper surfaces a critical yet under-researched sensor data extit{uncertainty} problem for training ML attack detection models, especially in highly mobile and risk-averse platforms such as autonomous vehicles. We present a model that explains extit{certainty} and extit{uncertainty} in sensor input -- a missing characteristic in data collection. We hypothesize that model explanation is inaccurate for a given system without explainable input data quality. We estimate extit{uncertainty} and mass functions for features in radar sensor data and incorporate them into the training model through experimental evaluation. The mass function allows the classifier to categorize all spoofed inputs accurately with an incorrect class label.

Motivation & Objective

• To address the lack of explainable input data quality in ML-based attack detection for CAVs, particularly concerning sensor data uncertainty.
• To improve model trustworthiness and forensic analysis by incorporating uncertainty quantification into training data for better decision transparency.
• To enable security experts to distinguish between genuine sensor behavior and adversarial spoofing through evidence-based reasoning.
• To demonstrate that uncertainty-aware input data enhances the reliability of ML models in detecting spoofing attacks in radar sensor inputs.
• To provide a framework for integrating uncertain sensor data into ML models to support safer, more transparent autonomous vehicle decisions.

Proposed method

• Uses Dempster-Shafer Theory (DST) to model belief, plausibility, and uncertainty for each radar feature (e.g., velocity, reflection, distance) in the frame of discernment {stationary, moving, both}.
• Applies DST equations to compute belief, plausibility, and uncertainty values for each feature, enabling representation of ignorance and partial evidence.
• Assigns mass functions to each hypothesis (s, m, sm) to represent the degree of support for each class, with higher mass indicating stronger evidence.
• Empirically evaluates the model on a hybrid radar dataset combining simulated and real-world data, with features in CSV format and pre-assigned class labels.
• Injects 11 spoofed radar packets (labeling stationary objects as moving) to test detection capability under adversarial conditions.
• Uses the highest belief value to determine class prediction and leverages mass function shifts (e.g., increased m for moving) to detect spoofing, even when class labels are incorrect.

Experimental results

Research questions

• RQ1 Can uncertainty quantification in radar sensor data improve the explainability and reliability of ML-based attack detection in CAVs?
• RQ2 How does incorporating Dempster-Shafer Theory-based evidence functions enhance the detection of spoofing attacks in radar inputs?
• RQ3 To what extent can mass functions detect spoofed data when the ML model’s label assignment is incorrect due to adversarial manipulation?
• RQ4 Does integrating uncertainty in input features lead to more accurate and trustworthy classification of obstacle states (stationary vs. moving)?
• RQ5 Can the proposed method support forensic analysis by providing interpretable evidence of data anomalies and uncertainty in sensor readings?

Key findings

• The model achieved 96% accuracy when using combined features (velocity and reflection) for classifying obstacle states (stationary or moving).
• Accuracy dropped below 90% when distance was used as the sole feature for class membership, indicating feature-specific sensitivity to uncertainty.
• All 11 injected spoofed packets were correctly classified as stationary, with the mass function for the 'moving' class showing significantly lower values than expected.
• For spoofed packet #2, the mass for 'moving' was 0.3023, while the belief for 'stationary' was 0.6976, correctly indicating inconsistency and potential spoofing.
• The mass function for the combined class 's,m' (both stationary and moving) reached 0.36 for spoofed packet #1, signaling high uncertainty and potential data manipulation.
• The method successfully detected spoofing by identifying anomalous mass function shifts—especially when 'moving' mass was high despite the object being stationary—providing forensic evidence of adversarial interference.