[Paper Review] Never Mind the Bounding Boxes, Here's the SAND Filters

Zhiqiang Sui, Zhefan Ye+1 more|arXiv (Cornell University)|Aug 15, 2018

Robot Manipulation and Learning24 references3 citations

TL;DR

This paper proposes SAND Filters, a two-stage method combining CNN-based object detection with generative sampling-based local search to improve 6D object pose estimation in cluttered environments. By leveraging CNN confidence scores and iteratively refining hypotheses via particle filtering, SAND reduces false positives and negatives, achieving 95.2% pose estimation success and 88.1% task completion in categorical sorting.

ABSTRACT

Perception is the main bottleneck to perform autonomous mobile manipulation tasks, especially in cluttered and unstructured environment. In this paper, we propose a novel two-stage paradigm that leverage both CNN object prior and generative sampling to perform object detection and 6D pose estimation. Our two-stage approach builds upon both CNN and generative sampling-based local search method to achieve sampling the network density, or SAND filter. We show the quantitative results that SAND effectively improve object detection result by reducing false positive and false negative recognitions, and further produces accurate pose estimation. We also conduct extensive categorical object sorting experiments to show our method is able to produce accurate and reliable detections and object poses.

Motivation & Objective

• Address the challenge of accurate object detection and 6D pose estimation in cluttered, unstructured environments where traditional CNNs fail due to occlusion and viewpoint variation.
• Overcome limitations of end-to-end object detectors by decoupling detection from pose refinement using a two-stage framework.
• Improve robustness and reliability of perception in mobile manipulation by integrating deep learning priors with probabilistic sampling-based refinement.
• Enable accurate categorical object sorting by combining reliable detection with precise pose estimation for downstream grasping and placement.

Proposed method

• Use a CNN-based detector in the first stage to generate object bounding boxes with confidence scores, without filtering based on score thresholds.
• In the second stage, crop depth images corresponding to each bounding box and apply a particle filter (SAND filter) to refine object pose hypotheses.
• Each particle represents a 6D object pose, with weights updated using CNN confidence, rendered depth from a 3D model, and observed depth from the sensor.
• Iteratively resample particles based on likelihood, allowing convergence to the most probable pose despite occlusion and clutter.
• The method is modular and compatible with various CNN detectors and alternative second-stage methods (e.g., ICP, FPFH).
• The framework supports sequential observation extension and multiple object instance detection in future work.

Experimental results

Research questions

• RQ1 Can a two-stage approach combining CNN detection and generative sampling improve 6D object pose estimation in cluttered, occluded environments compared to end-to-end detectors?
• RQ2 To what extent does the SAND filter reduce false positive and false negative detections compared to baseline CNN-only methods?
• RQ3 How does the generative sampling-based refinement stage enhance pose estimation accuracy under challenging visual conditions?
• RQ4 Can the SAND filter enable reliable categorical object sorting in real-world robotic manipulation tasks?

Key findings

• The SAND filter achieved 95.2% success rate in pose estimation across 42 object instances in categorical sorting tasks, significantly outperforming the baseline method.
• With a tight 0.02-meter distance threshold, the SAND filter achieved over 60% pose estimation accuracy, demonstrating robustness to small pose errors.
• The second-stage generative sampling method outperformed ICP and FPFH when used with the same first-stage detection, confirming its superiority in pose refinement.
• Task completion success rate reached 88.1% (37 out of 42 sequences), indicating the system's practical viability for autonomous manipulation.
• Grasping failures were the primary bottleneck (3 of 5 failures), attributed to the use of an open-loop, hard gripper without tactile feedback.
• The method is modular and generalizable, as it can be adapted to other CNN detectors and second-stage components with minimal modification.