[Paper Review] Robust approximate linear regression without correspondence

Amin Nejatbakhsh, Erdem Varol|arXiv (Cornell University)|Jun 1, 2019

Sparse and Compressive Sensing TechniquesEngineering32 references3 citations

TL;DR

This paper introduces robust regression without correspondence (rRWOC), a framework for estimating linear regression coefficients when both source and target point sets contain outliers and missing correspondences. It proposes randomized algorithms using RANSAC and Hungarian algorithm-based assignment to achieve exact recovery in noiseless one-dimensional cases and approximate recovery in noisy or higher-dimensional settings, with theoretical guarantees and validation on C. elegans neuron matching.

ABSTRACT

We propose methods for estimating correspondence between two point sets under the presence of outliers in both the source and target sets. The proposed algorithms expand upon the theory of the regression without correspondence problem to estimate transformation coefficients using unordered multisets of covariates and responses. Previous theoretical analysis of the problem has been done in a setting where the responses are a complete permutation of the regressed covariates. This paper expands the problem setting by analyzing the cases where only a subset of the responses is a permutation of the regressed covariates in addition to some covariates being outliers. We term this problem extit{robust regression without correspondence} and provide several algorithms based on random sample consensus for exact and approximate recovery in a noiseless and noisy one-dimensional setting as well as an approximation algorithm for multiple dimensions. The theoretical guarantees of the algorithms are verified in simulated data. We demonstrate an important computational neuroscience application of the proposed framework by demonstrating its effectiveness in a extit{Caenorhabditis elegans} neuron matching problem where the presence of outliers in both the source and target nematodes is a natural tendency.

Motivation & Objective

• Address the challenge of linear regression without correspondence when both covariates and responses contain outliers and missing data, a common issue in biological imaging.
• Extend prior work on regression without correspondence by modeling incomplete and corrupted data, not just complete permutations.
• Develop efficient, provably correct algorithms for exact and approximate recovery in noisy and high-dimensional settings.
• Demonstrate the method's effectiveness on a real-world computational neuroscience problem: matching neurons in C. elegans across different individuals.

Proposed method

• Introduce the rRWOC problem formulation that models missing correspondences and outliers explicitly, going beyond prior work assuming complete permutations.
• Propose a polynomial-time algorithm for exact recovery in one-dimensional noiseless rRWOC by testing all possible inlier pairs and using the Hungarian algorithm to verify assignments.
• Use RANSAC-inspired sampling to test candidate regression coefficients derived from pairs of points, selecting those that yield the largest inlier set under assignment cost.
• Apply the Hungarian algorithm to compute optimal assignment between regressed covariates and responses, using a thresholded inlier count to identify the correct coefficient.
• Define a margin ν based on inter-point distances to distinguish correct from incorrect coefficient estimates in the noisy regime.
• Use statistical bounds on noise variance to show that correct coefficient estimates yield significantly more inliers than incorrect ones with high probability.

Experimental results

Research questions

• RQ1 Can we achieve exact recovery of regression coefficients in one-dimensional rRWOC when both sets contain outliers and missing data?
• RQ2 How can we design an efficient, randomized algorithm for approximate recovery in noisy or higher-dimensional rRWOC?
• RQ3 What theoretical guarantees can be provided for coefficient recovery under noise and incomplete correspondence?
• RQ4 Can the proposed method outperform existing approaches in real-world applications with high outlier rates, such as C. elegans neuron matching?

Key findings

• The proposed algorithm achieves exact recovery of the true regression coefficient in the one-dimensional noiseless rRWOC setting under mild assumptions on data separation.
• In the noisy regime, the algorithm recovers the true coefficient with high probability when the signal-to-noise ratio is sufficiently high, as shown by variance and inlier count bounds.
• The method uses a margin ν defined as half the minimum absolute difference between distinct data points scaled by the true coefficient, enabling reliable distinction between correct and incorrect coefficient estimates.
• Theoretical analysis shows that correct coefficient estimates yield at least m−k inliers (where k is the number of outliers), while incorrect ones yield fewer, enabling identification via inlier cardinality.
• The algorithm is computationally efficient compared to brute-force methods, especially in higher dimensions, due to randomized sampling and early termination based on inlier count.
• Empirical validation on simulated data confirms the theoretical guarantees, and the method successfully matches neurons in C. elegans with high accuracy despite high outlier rates.