[Paper Review] SLURP: Side Learning Uncertainty for Regression Problems

Xuanlong Yu, Gianni Franchi+1 more|arXiv (Cornell University)|Oct 21, 2021

Anomaly Detection Techniques and Applications50 references4 citations

TL;DR

SLURP introduces a generic, efficient side learning framework for uncertainty estimation in regression tasks by leveraging features from a pre-trained main model and its predictions. It achieves state-of-the-art uncertainty calibration with minimal computational overhead, outperforming existing methods on monocular depth and optical flow benchmarks while remaining plug-and-play with no retraining of the main model.

ABSTRACT

It has become critical for deep learning algorithms to quantify their output uncertainties to satisfy reliability constraints and provide accurate results. Uncertainty estimation for regression has received less attention than classification due to the more straightforward standardized output of the latter class of tasks and their high importance. However, regression problems are encountered in a wide range of applications in computer vision. We propose SLURP, a generic approach for regression uncertainty estimation via a side learner that exploits the output and the intermediate representations generated by the main task model. We test SLURP on two critical regression tasks in computer vision: monocular depth and optical flow estimation. In addition, we conduct exhaustive benchmarks comprising transfer to different datasets and the addition of aleatoric noise. The results show that our proposal is generic and readily applicable to various regression problems and has a low computational cost with respect to existing solutions.

Motivation & Objective

• To address the lack of reliable uncertainty estimation in deep learning regression models, especially in vision tasks like depth and optical flow estimation.
• To develop a generic, non-intrusive method that does not require retraining or modifying the main prediction model.
• To improve uncertainty estimation by recovering semantic context missing in prediction outputs using a side learner.
• To ensure low computational cost while maintaining high uncertainty calibration performance across diverse datasets and noise conditions.

Proposed method

• A side learner network is trained to predict uncertainty maps using the original input and the main model's predictions as inputs.
• The side learner fuses features from the main model's intermediate layers and the prediction output to reconstruct the prediction error.
• It leverages frozen feature extractors (e.g., ResNet) from the main model to preserve semantic context and improve uncertainty estimation.
• The method uses a transposable architecture that allows plug-in deployment without modifying the main model’s training or inference pipeline.
• The side learner is trained end-to-end to minimize the difference between predicted uncertainty and ground-truth error (e.g., end-point error in optical flow).
• Two variants are proposed: Ours (full) and Ours-light (lightweight), both designed for efficiency and compatibility with real-time applications.

Experimental results

Research questions

• RQ1 Can a side learning approach effectively estimate uncertainty in pixel-wise regression tasks without modifying the main model?
• RQ2 How well does SLURP perform compared to existing uncertainty estimation methods in terms of calibration and robustness across diverse datasets?
• RQ3 To what extent does SLURP recover semantic context lost in the main model’s predictions to improve uncertainty estimation?
• RQ4 What is the computational efficiency of SLURP relative to ensemble-based and joint training methods?

Key findings

• SLURP achieves state-of-the-art performance on both monocular depth and optical flow tasks, with AUSE-EPE of 1.20 on FlyingChairs and 4.69 on KITTI, outperforming most baselines.
• On the Sintel dataset, SLURP achieves AUROC of 0.906, significantly outperforming MHP (0.878) and Confid (0.728), demonstrating strong generalization.
• The lightweight version (Ours-light) reduces inference time to 65ms on optical flow and 88ms on depth, outperforming MC (79ms) and EE (144ms) in speed while maintaining competitive accuracy.
• SLURP maintains high performance across transfer settings and noisy inputs, showing robustness without fine-tuning on new datasets.
• The side learner reduces uncertainty estimation error by effectively recovering semantic context from the main model’s features, especially in edge and occlusion regions.
• The method is computationally efficient, with only 87.2M parameters for the full model and 57.0M for Ours-light, significantly lower than ensemble-based methods.