[Paper Review] Learning how to be robust: Deep polynomial regression

Juan-Manuel Pérez-Rúa, Tomás Crivelli+2 more|arXiv (Cornell University)|Apr 17, 2018

Advanced Vision and Imaging40 references3 citations

TL;DR

This paper proposes a deep learning framework for robust polynomial regression using a differentiable, hard-wired polynomial decoder to bypass the need for supervised annotations. By training with synthetic, outlier-contaminated data and optimizing via loss functions on the decoded output, the model learns to generalize to real-world data, outperforming traditional robust regression methods in video stabilization tasks.

ABSTRACT

Polynomial regression is a recurrent problem with a large number of applications. In computer vision it often appears in motion analysis. Whatever the application, standard methods for regression of polynomial models tend to deliver biased results when the input data is heavily contaminated by outliers. Moreover, the problem is even harder when outliers have strong structure. Departing from problem-tailored heuristics for robust estimation of parametric models, we explore deep convolutional neural networks. Our work aims to find a generic approach for training deep regression models without the explicit need of supervised annotation. We bypass the need for a tailored loss function on the regression parameters by attaching to our model a differentiable hard-wired decoder corresponding to the polynomial operation at hand. We demonstrate the value of our findings by comparing with standard robust regression methods. Furthermore, we demonstrate how to use such models for a real computer vision problem, i.e., video stabilization. The qualitative and quantitative experiments show that neural networks are able to learn robustness for general polynomial regression, with results that well overpass scores of traditional robust estimation methods.

Motivation & Objective

• To develop a generic, unsupervised deep learning method for polynomial regression that does not require labeled ground-truth coefficients during training.
• To address the challenge of structured outliers in polynomial regression, which bias classical methods like least squares and RANSAC.
• To investigate whether deep neural networks can learn robustness to contamination without prior assumptions about outlier distribution or ratio.
• To compare the effectiveness of loss functions applied on the decoded output versus the estimated polynomial coefficients in improving generalization.
• To demonstrate the method's applicability to real-world computer vision tasks, such as parametric motion estimation and video stabilization.

Proposed method

• Uses a stacked hourglass architecture with 1D or 2D convolutions depending on input dimensionality (scalar or vector field regression), enabling multi-scale spatial reasoning.
• Employs a model-based autoencoder with 1D/2D convolutions and batch normalization to extract features and regress polynomial coefficients.
• Introduces a differentiable, hard-wired polynomial decoder that maps predicted coefficients back to the input data space, enabling end-to-end training without explicit supervision on coefficients.
• Trains the network using synthetic datasets (Data 1 and Data 2) with controlled contamination: Gaussian noise and structured polygonal outliers.
• Applies both standard L2 loss and robust loss functions (e.g., Huber loss) on the decoded output to improve generalization under contamination.
• Uses online data generation to simulate diverse outlier patterns, avoiding reliance on real annotated datasets.

Experimental results

Research questions

• RQ1 Can a deep neural network trained on purely synthetic, contaminated data learn to perform robust polynomial regression without any ground-truth coefficient supervision?
• RQ2 Does optimizing the loss on the decoded output (i.e., the reconstructed signal) yield better generalization than optimizing directly on the estimated polynomial coefficients?
• RQ3 How does the network’s performance compare to classical robust regression methods like RANSAC and M-estimators in the presence of structured outliers?
• RQ4 To what extent can a model trained on synthetic data generalize to real-world video stabilization tasks with complex, real-world noise and occlusions?
• RQ5 What architectural components (e.g., multi-scale processing, spatial awareness) are essential for achieving robustness in polynomial regression?

Key findings

• The proposed model achieves superior performance compared to standard robust regression methods like RANSAC and M-estimators in polynomial regression tasks with structured outliers.
• Training with a loss function applied on the decoded output (i.e., the reconstructed signal) leads to better generalization than optimizing directly on the polynomial coefficients.
• The model trained on synthetic data with 30% outlier ratio (Data 2) generalizes effectively to real video stabilization tasks, producing high-quality results without additional fine-tuning.
• The use of a differentiable, hard-wired polynomial decoder enables end-to-end training without requiring annotated coefficients, making the method scalable and unsupervised.
• Stacked hourglass networks with multi-scale and spatially-aware convolutions effectively learn to suppress structured outliers and preserve polynomial structure.
• The method demonstrates state-of-the-art performance in global motion estimation and video stabilization, outperforming classical approaches in both qualitative and quantitative evaluations.