[Paper Review] Efficient Outlier Removal in Large Scale Global Structure-from-Motion

Fei Wen, Danping Zou+2 more|arXiv (Cornell University)|Aug 9, 2018

Advanced Vision and Imaging25 references4 citations

TL;DR

This paper proposes two efficient outlier removal methods for large-scale global structure-from-motion by leveraging dimension reduction in multiview geometry. The methods use convex relaxed ℓ₁ minimization (solved via linear programming) and iteratively reweighted ℓ₀ approximation, achieving over threefold speedup over the ℓ₁ method while preserving or improving consensus set quality on real and synthetic datasets.

ABSTRACT

This work addresses the outlier removal problem in large-scale global structure-from-motion. In such applications, global outlier removal is very useful to mitigate the deterioration caused by mismatches in the feature point matching step. Unlike existing outlier removal methods, we exploit the structure in multiview geometry problems to propose a dimension reduced formulation, based on which two methods have been developed. The first method considers a convex relaxed $\ell_1$ minimization and is solved by a single linear programming (LP), whilst the second one approximately solves the ideal $\ell_0$ minimization by an iteratively reweighted method. The dimension reduction results in a significant speedup of the new algorithms. Further, the iteratively reweighted method can significantly reduce the possibility of removing true inliers. Realistic multiview reconstruction experiments demonstrated that, compared with state-of-the-art algorithms, the new algorithms are much more efficient and meanwhile can give improved solution. Matlab code for reproducing the results is available at extit{https://github.com/FWen/OUTLR.git}.

Motivation & Objective

• To address the challenge of outlier removal in large-scale global structure-from-motion, where mismatches in feature matching degrade 3D reconstruction quality.
• To develop faster and more robust algorithms than existing methods like ℓ₁ minimization, which are computationally prohibitive for large-scale problems.
• To reduce computational complexity through dimension reduction while maintaining high accuracy in identifying inliers and removing outliers.
• To enable practical deployment of robust outlier removal in real-world large-scale 3D reconstruction pipelines.

Proposed method

• Proposes a dimension-reduced formulation that reduces the number of variables in the optimization problem by exploiting the structure of multiview geometry.
• Introduces a convex relaxed ℓ₁ minimization problem solved via a single linear program (LP), enabling fast computation.
• Develops an iteratively reweighted algorithm to approximately solve the ideal ℓ₀ minimization, improving inlier preservation.
• Uses a nonnegative slack variable s to model outlier residuals, reformulating the maximum consensus problem into a sparse optimization framework.
• Applies a depth constraint (κ=6) in experiments to improve stability and reduce overfitting in real-world datasets.
• Employs a reweighting scheme with parameter q∈{0.1,0.2,0.5} to iteratively enhance sparsity and convergence in the ℓ₀ approximation.

Experimental results

Research questions

• RQ1 Can dimension reduction in multiview geometry significantly reduce computational complexity in global SfM outlier removal?
• RQ2 How does the proposed ℓ₁-based method compare to the state-of-the-art ℓ₁ method in terms of speed and consensus set quality?
• RQ3 Can the iteratively reweighted ℓ₀ approximation method preserve more true inliers while maintaining high efficiency?
• RQ4 To what extent do the proposed methods outperform existing deterministic and randomized algorithms in large-scale real-world SfM datasets?

Key findings

• The proposed ℓ₁-based method (Algorithm 1) is more than three times faster than the state-of-the-art ℓ₁ method, with a significant reduction in the number of variables (e.g., from 327,201 to 96,976 in the cathedral dataset).
• Algorithm 1 achieves a comparable root-mean-squared error (RMSE) in reprojection error (e.g., 1.28 pixels for the cathedral dataset) while removing fewer outliers than the ℓ₁ method.
• The iteratively reweighted method (Algorithm 2) with K=2 iterations achieves a consensus set size comparable to the ℓ₁ method but runs much faster, with a 3x speedup over the ℓ₁ method in runtime.
• On the house dataset, Algorithm 1 reduces the reprojection RMSE from 4.76 to 1.28 pixels, demonstrating effective outlier removal.
• Algorithm 2 removes significantly fewer outliers than other methods but maintains high inlier preservation, reducing the risk of removing true inliers.
• The algorithms converge within 10 iterations in synthetic experiments, with Algorithm 2 achieving good performance in just 2–5 iterations.