[Paper Review] High Quality Monocular Depth Estimation via Transfer Learning

Ibraheem Alhashim, Peter Wonka|arXiv (Cornell University)|Dec 31, 2018

Advanced Vision and Imaging37 references190 citations

TL;DR

The authors propose a simple encoder–decoder network that uses a pre-trained DenseNet-169 encoder and a lightweight decoder to produce high-resolution monocular depth maps with improved fidelity, trained with a multi-term loss and targeted data augmentations.

ABSTRACT

Accurate depth estimation from images is a fundamental task in many applications including scene understanding and reconstruction. Existing solutions for depth estimation often produce blurry approximations of low resolution. This paper presents a convolutional neural network for computing a high-resolution depth map given a single RGB image with the help of transfer learning. Following a standard encoder-decoder architecture, we leverage features extracted using high performing pre-trained networks when initializing our encoder along with augmentation and training strategies that lead to more accurate results. We show how, even for a very simple decoder, our method is able to achieve detailed high-resolution depth maps. Our network, with fewer parameters and training iterations, outperforms state-of-the-art on two datasets and also produces qualitatively better results that capture object boundaries more faithfully. Code and corresponding pre-trained weights are made publicly available.

Motivation & Objective

• Demonstrate that transfer learning from image classification encoders can yield higher-quality monocular depth maps.
• Show that a simple decoder can achieve state-of-the-art performance with fewer parameters and iterations.
• Propose a loss function that preserves depth accuracy while maintaining boundary details.
• Define an augmentation strategy that improves learning efficiency and generalization.
• Provide a new synthetic indoor depth dataset to evaluate generalization (Unreal-1k).

Proposed method

• Encoder: pre-trained DenseNet-169 (ImageNet) truncated for depth estimation.
• Decoder: lightweight upsampling blocks with skip connections, no Batch Normalization.
• Loss: L = lambda * L_depth + L_grad + L_SSIM with lambda = 0.1 and L_depth as L1 in depth space; depth values are transformed via reciprocal scaling to stabilize training.
• Data augmentation: horizontal flipping with probability 0.5; color channel permutations with probability 0.25.
• Training strategy: end-to-end training with ADAM, 1M iterations on NYU Depth v2, 300K iterations on KITTI; uses mean squared/relative error metrics for evaluation.
• Evaluation: qualitative and quantitative assessment on NYU Depth v2, KITTI, and Unreal-1k datasets.

Experimental results

Research questions

• RQ1 Does transfer learning from image classification encoders improve monocular depth estimation quality?
• RQ2 Can a simple decoder outperform more complex architectures when paired with a strong encoder?
• RQ3 How do loss design and augmentation strategies affect depth accuracy and edge preservation?
• RQ4 How well does the proposed model generalize to synthetic indoor datasets (Unreal-1k) and cross-domain data?

Key findings

• Achieves state-of-the-art performance on NYU Depth v2 in most metrics with fewer parameters (~42.6M) and fewer training iterations (1M) than some prior methods.
• Produces depth maps with sharper object boundaries and fewer artifacts than previous methods.
• Outperforms on Unreal-1k synthetic indoor dataset in both quantitative (lower error) and qualitative measures.
• Second-best overall on KITTI dataset across standard metrics, with performance limited by very sparse ground-truth data but qualitative depth maps remain high quality.
• Ablation studies show color-channel augmentation significantly improves generalization and that a deeper encoder provides diminishing returns given higher parameter costs.