[Paper Review] Fooling LiDAR Perception via Adversarial Trajectory Perturbation

Yiming Li, Congcong Wen+2 more|arXiv (Cornell University)|Mar 29, 2021

Adversarial Robustness in Machine Learning51 references4 citations

TL;DR

This paper proposes FLAT, a novel adversarial attack that fools LiDAR-based 3D object detectors by perturbing the vehicle's trajectory instead of directly modifying point clouds. By exploiting the differentiable motion compensation process, the method generates imperceptible trajectory perturbations—especially via polynomial smoothing—that drastically reduce detection performance, even blinding state-of-the-art detectors with minimal GPS spoofing.

ABSTRACT

LiDAR point clouds collected from a moving vehicle are functions of its trajectories, because the sensor motion needs to be compensated to avoid distortions. When autonomous vehicles are sending LiDAR point clouds to deep networks for perception and planning, could the motion compensation consequently become a wide-open backdoor in those networks, due to both the adversarial vulnerability of deep learning and GPS-based vehicle trajectory estimation that is susceptible to wireless spoofing? We demonstrate such possibilities for the first time: instead of directly attacking point cloud coordinates which requires tampering with the raw LiDAR readings, only adversarial spoofing of a self-driving car's trajectory with small perturbations is enough to make safety-critical objects undetectable or detected with incorrect positions. Moreover, polynomial trajectory perturbation is developed to achieve a temporally-smooth and highly-imperceptible attack. Extensive experiments on 3D object detection have shown that such attacks not only lower the performance of the state-of-the-art detectors effectively, but also transfer to other detectors, raising a red flag for the community. The code is available on https://ai4ce.github.io/FLAT/.

Motivation & Objective

• Investigate the vulnerability of LiDAR-based perception in autonomous vehicles to adversarial trajectory spoofing.
• Expose a new attack surface where small, imperceptible trajectory perturbations can severely degrade 3D object detection performance.
• Demonstrate that motion compensation, a standard preprocessing step, inadvertently creates a backdoor for adversarial attacks.
• Develop a differentiable, smooth, and highly imperceptible attack method using polynomial trajectory perturbations.
• Show transferability of the attack across different detectors and input representations, highlighting systemic risks in DNN-based LiDAR perception.

Proposed method

• Simulate motion distortion in LiDAR point clouds by applying adversarial perturbations to the vehicle’s trajectory, using real-world sweeps from datasets like KITTI and nuScenes.
• Model the LiDAR point cloud as a differentiable function of the vehicle’s trajectory, enabling gradient-based optimization of adversarial perturbations.
• Apply polynomial trajectory perturbation to ensure temporal smoothness and minimal perceptibility in the physical world.
• Use regularization to minimize changes in the resulting point cloud (measured via ℓp norm and Chamfer distance), enhancing stealthiness.
• Perform both white-box and black-box attacks on 3D detectors like PointRCNN and PointPillar++, evaluating transferability across architectures and object categories.
• Target specific components of the detector (e.g., classification in stage-2, regression in stage-1) to identify the most effective attack vectors.

Experimental results

Research questions

• RQ1 Can small, adversarial perturbations to the vehicle’s trajectory—specifically via GPS spoofing—cause significant degradation in LiDAR-based 3D object detection?
• RQ2 How does the differentiable nature of motion compensation enable gradient-based adversarial attacks on the trajectory instead of the point cloud directly?
• RQ3 To what extent does polynomial trajectory perturbation improve the imperceptibility and feasibility of the attack in real-world deployment?
• RQ4 How transferable is the attack across different 3D object detectors and input representations (e.g., voxel-based vs. point-based)?
• RQ5 Which component of the detection pipeline (e.g., classification, regression) is most vulnerable to trajectory-based adversarial attacks?

Key findings

• Attacking the full trajectory with only 10 cm perturbation in translation can reduce mean average precision (mAP) by up to 70%, effectively blinding the detector.
• When attacking the classification head in stage-2 of PointRCNN, mAP drops by 47.25 (99.6%) in easy scenarios, reducing performance to nearly zero.
• Polynomial trajectory perturbation maintains attack effectiveness while reducing point cloud changes by 47.6% (ℓp norm) and 51.7% (Chamfer distance), enhancing stealthiness.
• The attack shows strong transferability: when trained on car detection, it reduces mAP by 20.6 (58.2%) across ten categories and drops AP for pedestrians by 48.3 (72.2%).
• Attacking rotation leads to greater performance degradation than attacking translation, with AP drops of 45.09 (95.0%) in easy scenarios, indicating rotational spoofing is more effective.
• The attack remains effective even when targeting only the proposal generation stage (stage-1 regression), proving that fundamental detection components are vulnerable.