[Paper Review] AdaLAM: Revisiting Handcrafted Outlier Detection

Luca Cavalli, Viktor Larsson+3 more|arXiv (Cornell University)|Jun 7, 2020

Robotics and Sensor-Based Localization42 references7 citations

TL;DR

AdaLAM proposes a novel, highly parallelizable outlier filtering method for image matching that combines hierarchical spatial verification with adaptive thresholding for local affine transformations. It achieves state-of-the-art performance in both accuracy and speed—processing 8,000 matches per image pair in under 20ms on a modern GPU—demonstrating superior generalization across diverse outdoor and indoor scenes.

ABSTRACT

Local feature matching is a critical component of many computer vision pipelines, including among others Structure-from-Motion, SLAM, and Visual Localization. However, due to limitations in the descriptors, raw matches are often contaminated by a majority of outliers. As a result, outlier detection is a fundamental problem in computer vision, and a wide range of approaches have been proposed over the last decades. In this paper we revisit handcrafted approaches to outlier filtering. Based on best practices, we propose a hierarchical pipeline for effective outlier detection as well as integrate novel ideas which in sum lead to AdaLAM, an efficient and competitive approach to outlier rejection. AdaLAM is designed to effectively exploit modern parallel hardware, resulting in a very fast, yet very accurate, outlier filter. We validate AdaLAM on multiple large and diverse datasets, and we submit to the Image Matching Challenge (CVPR2020), obtaining competitive results with simple baseline descriptors. We show that AdaLAM is more than competitive to current state of the art, both in terms of efficiency and effectiveness.

Motivation & Objective

• To improve the robustness and efficiency of handcrafted outlier detection in image matching pipelines.
• To address the limitations of traditional filters like the ratio-test and mutual nearest neighbor by incorporating spatial consistency and geometric verification.
• To design a method that generalizes well across diverse scenes (outdoor and indoor) without relying on learned descriptors.
• To optimize for modern parallel hardware, enabling sub-20ms processing per image pair on a GPU.
• To introduce adaptive thresholding that dynamically relaxes geometric assumptions based on local region support, improving generalization.

Proposed method

• Employs a hierarchical pipeline: first identifying well-spread region correspondence hypotheses from putative matches.
• For each hypothesis, selects matches consistent with the same local affine transformation, using a sample-adaptive threshold to balance sensitivity and robustness.
• Applies local affine verification with RANSAC-like sampling to identify inliers that support a consistent transformation within each region.
• Uses adaptive thresholding that relaxes geometric constraints based on local support, improving generalization to varying scene conditions.
• Leverages modern GPU parallelism to process multiple region hypotheses and match sets simultaneously, enabling low-latency filtering.
• Integrates seamlessly with standard baseline descriptors (e.g., SIFT), demonstrating that performance gains come from the filtering pipeline, not descriptor quality.

Experimental results

Research questions

• RQ1 Can a handcrafted, geometry-based outlier filter outperform learned methods when combined with adaptive thresholding and parallelization?
• RQ2 How does adaptive relaxation of geometric assumptions improve generalization across diverse image pairs (e.g., indoor vs. outdoor, varying viewpoints)?
• RQ3 To what extent can a purely geometric, non-learning-based approach achieve state-of-the-art performance in outlier rejection?
• RQ4 What is the performance gain of using sample-adaptive thresholds compared to fixed thresholds in local affine verification?
• RQ5 How efficiently can such a pipeline be parallelized on modern GPU hardware without sacrificing accuracy?

Key findings

• AdaLAM achieves competitive performance on the Image Matching Challenge (CVPR 2020), outperforming state-of-the-art methods when using only standard baseline descriptors.
• On the YFCC100M dataset, AdaLAM achieves 88.8% mAP20 and 94.6% AUC20*, significantly outperforming OA-Net (84.7% mAP20, 90.9% AUC20*) and NGRANSAC (84.7% mAP20, 90.9% AUC20*).
• On the SUN3D dataset, AdaLAM achieves 58.4% mAP20 and 89.2% AUC20*, surpassing OA-Net (49.9% mAP20, 89.6% AUC20*) and NGRANSAC (46.5% mAP20, 90.9% AUC20*).
• AdaLAM processes 8,000 matches per image pair in under 20 milliseconds on a modern GPU, demonstrating high efficiency suitable for real-time applications.
• The adaptive thresholding mechanism improves generalization, especially in low-texture or challenging scenes, by dynamically adjusting to local support levels.
• Refitting the essential matrix using LO-RANSAC on AdaLAM’s inlier set further enhances performance, confirming the robustness of the filtered matches.