[Paper Review] Detection and Tracking of General Movable Objects in Large 3D Maps

Nils Bore, Johan Ekekrantz+2 more|arXiv (Cornell University)|Dec 22, 2017

Robotics and Sensor-Based Localization29 references3 citations

TL;DR

This paper proposes a Rao-Blackwellized particle filter that decomposes object motion into local movements (modeled with Kalman filters) and rare global jumps (modeled probabilistically), enabling accurate multi-target tracking of visually similar, movable objects in large 3D maps despite intermittent observations. The method outperforms prior approaches on real-world data by leveraging probabilistic modeling and focused sampling of object posteriors.

ABSTRACT

This paper studies the problem of detection and tracking of general objects with long-term dynamics, observed by a mobile robot moving in a large environment. A key problem is that due to the environment scale, it can only observe a subset of the objects at any given time. Since some time passes between observations of objects in different places, the objects might be moved when the robot is not there. We propose a model for this movement in which the objects typically only move locally, but with some small probability they jump longer distances, through what we call global motion. For filtering, we decompose the posterior over local and global movements into two linked processes. The posterior over the global movements and measurement associations is sampled, while we track the local movement analytically using Kalman filters. This novel filter is evaluated on point cloud data gathered autonomously by a mobile robot over an extended period of time. We show that tracking jumping objects is feasible, and that the proposed probabilistic treatment outperforms previous methods when applied to real world data. The key to efficient probabilistic tracking in this scenario is focused sampling of the object posteriors.

Motivation & Objective

• To enable long-term tracking of multiple, visually similar movable objects in large-scale 3D environments where only subsets are observed at any time.
• To address the challenge of object movement during robot observation gaps, particularly rare long-distance jumps.
• To improve tracking accuracy beyond appearance-based methods by modeling spatial dynamics and movement probabilities.
• To develop a scalable, probabilistic framework that maintains realistic position posteriors even with incomplete observations.

Proposed method

• The system models object motion as a combination of local movements (small, frequent adjustments) and global jumps (rare, long-distance moves), with a probabilistic jump model.
• It uses a Rao-Blackwellized particle filter where the posterior over global movements and measurement associations is sampled, while local movements are tracked analytically using Kalman filters.
• Object detections are generated via temporal change detection in 3D point clouds, followed by instance segmentation and feature extraction using deep learning (e.g., CNNs).
• The tracker integrates visual features (from RGB images) and spatial measurements (from depth data) to improve robustness, especially for small objects.
• The method employs a closed-world assumption, assuming objects remain within the environment, and uses parameter tuning (e.g., p_jump, p_meas) to adapt to observed jump frequencies.
• The system estimates object-specific movement model variances (Q_k^s) to reflect different mobility patterns, such as chairs vs. monitors.

Experimental results

Research questions

• RQ1 How can a mobile robot maintain accurate, long-term tracking of multiple visually similar objects in large 3D environments with intermittent observations?
• RQ2 What is the optimal way to model object movement when objects may move locally or make rare, long-distance jumps?
• RQ3 Can a hybrid filtering approach combining particle sampling and Kalman filtering improve tracking performance over purely appearance-based or standard filtering methods?
• RQ4 How do parameters like jump probability (p_jump) and measurement probability (p_meas) affect tracking performance in real-world scenarios?

Key findings

• The proposed tracker significantly outperforms previous methods on real-world data, particularly in handling jumping objects and visually similar targets.
• The method achieves robust tracking even with noisy depth data by fusing RGB and depth features, especially improving detection of small objects like mugs.
• The MOTA score was highest when p_jump was set to 0.08 in experiment 1 (where actual jump frequency was ~0.18) and to 0.04 in experiment 2 (~0.05), indicating parameter tuning is critical.
• The system maintained accurate position posteriors across time, as shown in Figure 1, where blue-violet regions reflect refined belief distributions after repeated observations.
• The tracker successfully handled segmentation errors (e.g., chair oversegmentation) by relying on spatial and motion modeling rather than pure segmentation.
• Estimating the feature covariance R^f from data improved results significantly in experiment 1, suggesting adaptive tuning enhances performance.