[Paper Review] Camera-based vehicle velocity estimation from monocular video

Moritz Kampelmühler, Michael G. Müller+1 more|arXiv (Cornell University)|Feb 20, 2018

Autonomous Vehicle Technology and Safety28 references21 citations

TL;DR

This paper presents a lightweight, real-time monocular vehicle velocity estimation method that outperforms deep learning-based depth and motion cues by relying solely on trajectory features extracted from a video tracker. Using a multilayer perceptron to regress velocity from these features, the approach achieves a test error of 1.12 m/s—remarkably close to a LiDAR-radar fusion system (0.71 m/s)—and won the CVPR2017 challenge on a single CPU.

ABSTRACT

This paper documents the winning entry at the CVPR2017 vehicle velocity estimation challenge. Velocity estimation is an emerging task in autonomous driving which has not yet been thoroughly explored. The goal is to estimate the relative velocity of a specific vehicle from a sequence of images. In this paper, we present a light-weight approach for directly regressing vehicle velocities from their trajectories using a multilayer perceptron. Another contribution is an explorative study of features for monocular vehicle velocity estimation. We find that light-weight trajectory based features outperform depth and motion cues extracted from deep ConvNets, especially for far-distance predictions where current disparity and optical flow estimators are challenged significantly. Our light-weight approach is real-time capable on a single CPU and outperforms all competing entries in the velocity estimation challenge. On the test set, we report an average error of 1.12 m/s which is comparable to a (ground-truth) system that combines LiDAR and radar techniques to achieve an error of around 0.71 m/s.

Motivation & Objective

• To address the emerging challenge of monocular vehicle velocity estimation in autonomous driving using only RGB video.
• To evaluate the effectiveness of various visual features—especially trajectory-based ones—compared to deep ConvNet-derived depth and optical flow cues.
• To develop a real-time, lightweight regression model capable of running on a single CPU.
• To determine whether trajectory-based features alone can outperform more complex motion and depth features in velocity estimation.
• To achieve state-of-the-art performance on the CVPR2017 vehicle velocity estimation challenge.

Proposed method

• A two-stage pipeline: first, object tracking extracts vehicle trajectories; then, dense depth and optical flow are estimated and aggregated at tracked locations over time.
• Features from tracking, depth, and optical flow are concatenated along the temporal dimension to form a spatiotemporal feature vector.
• A multilayer perceptron (MLP) is trained to directly regress vehicle velocity from the concatenated feature vector.
• The model is trained with early stopping based on mean squared error (MSE) on validation sets.
• Ablation studies compare performance across feature combinations and ranges (near, medium, far), with separate models trained per range.
• Inference is optimized for speed, with feature extraction as the main computational bottleneck.

Experimental results

Research questions

• RQ1 Do trajectory-based features outperform deep ConvNet-derived depth and optical flow features for monocular vehicle velocity estimation?
• RQ2 How does performance vary across different distance ranges (near, medium, far) when using different feature sets?
• RQ3 Can a lightweight, fully connected network achieve real-time inference on a single CPU while outperforming more complex models?
• RQ4 Is the performance of deep motion and depth estimators sufficient for accurate velocity regression in monocular video?
• RQ5 Can a simple, end-to-end trainable system be built that jointly optimizes tracking and velocity estimation?

Key findings

• Trajectory-based features alone outperform depth and motion cues from deep ConvNets, especially in far-distance scenarios where depth and flow estimates degrade.
• The model using only tracking features achieved a test error of 1.25 m/s (MSE 1.25 m²/s²), ranking first in the CVPR2017 challenge.
• The full model incorporating tracking, flow, and depth features achieved a test error of 1.30 m/s, slightly worse than the tracking-only model but still winning the challenge.
• For far-range predictions (>20m), depth and flow features degrade significantly, making trajectory features more robust.
• Inference with only trajectory features runs at 100 FPS on a single CPU, while full feature processing runs at ~2 FPS, highlighting the efficiency of the lightweight approach.
• The method’s 1.12 m/s average error is within 0.41 m/s of a LiDAR-radar ground-truth system, demonstrating strong performance with only monocular video.