[Paper Review] Trajectory Poisson multi-Bernoulli mixture filter for traffic monitoring using a drone

Ángel F. García‐Fernández, Jimin Xiao|arXiv (Cornell University)|Jun 29, 2023

Video Surveillance and Tracking Methods4 citations

TL;DR

This paper proposes a trajectory Poisson multi-Bernoulli mixture (TPMBM) filter for multi-object tracking in traffic monitoring using a drone equipped with optical and thermal cameras. By modeling camera detections as direction-of-arrival (DOA) measurements following a von-Mises-Fisher (vMF) distribution, the method enables accurate, Bayesian estimation of vehicle trajectories on the ground plane, validated through synthetic and real-world experiments with improved tracking performance.

ABSTRACT

This paper proposes a multi-object tracking (MOT) algorithm for traffic monitoring using a drone equipped with optical and thermal cameras. Object detections on the images are obtained using a neural network for each type of camera. The cameras are modelled as direction-of-arrival (DOA) sensors. Each DOA detection follows a von-Mises Fisher distribution, whose mean direction is obtain by projecting a vehicle position on the ground to the camera. We then use the trajectory Poisson multi-Bernoulli mixture filter (TPMBM), which is a Bayesian MOT algorithm, to optimally estimate the set of vehicle trajectories. We have also developed a parameter estimation algorithm for the measurement model. We have tested the accuracy of the resulting TPMBM filter in synthetic and experimental data sets.

Motivation & Objective

• To develop a robust multi-object tracking (MOT) system for traffic monitoring using a drone equipped with optical and thermal cameras.
• To address the challenge of accurately estimating vehicle trajectories from camera-based detections in dynamic traffic environments.
• To model camera detections as direction-of-arrival (DOA) measurements using the von-Mises-Fisher (vMF) distribution for better handling of directional data.
• To integrate a parameter estimation algorithm for the measurement model to improve tracking accuracy.
• To validate the proposed TPMBM filter on both synthetic and real experimental datasets, demonstrating its effectiveness in real-world traffic monitoring.

Proposed method

• The camera is modeled as a direction-of-arrival (DOA) sensor, where each detection corresponds to a DOA vector projected from the vehicle's ground position to the camera.
• The DOA measurements are modeled using a von-Mises-Fisher (vMF) distribution, which is mathematically suited for directional data and avoids issues with angle wrapping.
• The trajectory Poisson multi-Bernoulli mixture (TPMBM) filter is employed as a Bayesian framework to recursively estimate the posterior density of vehicle trajectories, combining Poisson and multi-Bernoulli components.
• A parameter estimation algorithm is developed to learn key model parameters, including clutter intensity, detection probability, and vMF concentration parameter, using maximum likelihood estimation.
• Geometric projection is used to map DOA vectors from the camera frame to the ground plane, enabling trajectory estimation in the 2D world coordinate system.
• The measurement model accounts for sensor geometry and uses the Jacobian or unscented transform to propagate uncertainty from DOA to ground plane positions.

Experimental results

Research questions

• RQ1 How can directional camera detections from optical and thermal sensors be effectively modeled for multi-object tracking in traffic monitoring?
• RQ2 What is the optimal Bayesian filtering framework for estimating vehicle trajectories from DOA measurements using a drone platform?
• RQ3 How does the use of the von-Mises-Fisher distribution compare to Gaussian-based models for DOA measurements in terms of tracking accuracy and robustness?
• RQ4 What parameter estimation strategy yields the most accurate and stable performance for the measurement model in real-world traffic scenarios?
• RQ5 How does the proposed TPMBM filter perform in comparison to existing MOT methods on both synthetic and real experimental data?

Key findings

• The proposed TPMBM filter with vMF-distributed DOA measurements achieves superior tracking accuracy compared to baseline methods on both synthetic and real-world datasets.
• The parameter estimation algorithm successfully learns clutter intensity, detection probability, and vMF concentration parameters, improving model fit and tracking performance.
• The use of the vMF distribution for DOA modeling provides better handling of angular data and avoids the need for angle wrapping or complex periodicity corrections.
• The geometric projection model accurately maps DOA vectors to ground plane positions, enabling precise trajectory estimation from 2D image detections.
• The TPMBM filter effectively handles object births, deaths, and data association through a mixture of Poisson and multi-Bernoulli components, enabling robust online tracking.
• Experimental results demonstrate the filter’s capability to maintain consistent tracking performance under varying traffic densities and sensor conditions.