[Paper Review] IntentNet: Learning to Predict Intention from Raw Sensor Data

Sergio Casas, Wenjie Luo+1 more|arXiv (Cornell University)|Jan 20, 2021

Autonomous Vehicle Technology and Safety250 citations

TL;DR

IntentNet jointly detects vehicles and predicts continuous trajectories and discrete high-level intentions directly from raw LiDAR BEV data and dynamic maps in a single end-to-end network, improving detection, trajectory forecast, and intention prediction with shared computation.

ABSTRACT

In order to plan a safe maneuver, self-driving vehicles need to understand the intent of other traffic participants. We define intent as a combination of discrete high-level behaviors as well as continuous trajectories describing future motion. In this paper, we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. Our multi-task model achieves better accuracy than the respective separate modules while saving computation, which is critical to reducing reaction time in self-driving applications.

Motivation & Objective

• Motivate safe autonomous driving by accurately inferring other agents' high-level intentions and future motions.
• Develop a one-stage network that fuses LiDAR BEV and dynamic map information to predict discrete actions and continuous trajectories.
• jointly optimize detection, intention classification, and trajectory regression to improve accuracy while reducing computation time.

Proposed method

• Represent LiDAR point clouds in bird’s eye view (BEV) as a 3D tensor with height as channels and stack multiple past sweeps for temporal context.
• Rasterize dynamic HD maps into BEV binary masks to encode road topology, lanes, intersections, and traffic light states.
• Use a two-stream backbone to separately process LiDAR BEV and map inputs, then fuse features and pass to three task-specific heads (detection, discrete intention, trajectory regression).
• Predict for each anchor box: (i) vehicle/background detection scores, (ii) discrete high-level intention over 8 classes, and (iii) future trajectory through bounding boxes across time steps.
• Train end-to-end with a multi-task loss that includes detection loss, intention cross-entropy with temporal discounting, and a weighted smooth L1 regression loss for trajectories; include a temporal discount factor to account for future uncertainty.

Experimental results

Research questions

• RQ1 Can a single end-to-end network jointly detect vehicles, predict discrete high-level intentions, and forecast long-horizon trajectories from raw LiDAR and map data?
• RQ2 Does early/late fusion of LiDAR and map information plus temporal context improve detection and intention/trajectory prediction over prior FaF-based approaches?
• RQ3 What is the impact of including map priors and discrete intention losses on overall detection and motion forecasting performance?

Key findings

• IntentNet achieves higher detection mAP across IoU levels than real-time detectors (e.g., mAP@0.5 = 94.4, mAP@0.6 = 89.4, mAP@0.7 = 75.4, mAP@0.8 = 43.5, mAP@0.9 = 3.9).
• Trajectory regression improves over FaF and FaF’ in L1 along-track, L1 across-track, and heading errors (e.g., along 0s: 0.26 m; 1s: 0.46 m; across 0s: 0.15 m; heading 0s: 5.14 deg).
• IntentNet outperforms baselines in intention prediction, especially for less represented actions; mean accuracy across actions reaches 97.7% with map and discrete intention modeling.
• Including map priors and discrete intention loss yields best overall performance, while removing map or intention loss degrades results, per ablation study.
• Compared to FaF, IntentNet with full inputs (map + high-level actions) achieves improved detection and longer-horizon motion forecasting.