[Paper Review] Hierarchical Normalization for Robust Monocular Depth Estimation

Chi Zhang, Wei Yin+4 more|arXiv (Cornell University)|Oct 18, 2022

Advanced Vision and Imaging8 citations

TL;DR

This paper proposes Hierarchical Depth Normalization (HDN), a multi-scale normalization strategy that improves monocular depth estimation by preserving fine-grained depth details through spatial and depth-domain context grouping. By normalizing depth representations at multiple hierarchical levels, HDN outperforms instance-level normalization, achieving new state-of-the-art performance on five zero-shot transfer benchmarks with significant gains in accuracy and edge sharpness.

ABSTRACT

In this paper, we address monocular depth estimation with deep neural networks. To enable training of deep monocular estimation models with various sources of datasets, state-of-the-art methods adopt image-level normalization strategies to generate affine-invariant depth representations. However, learning with image-level normalization mainly emphasizes the relations of pixel representations with the global statistic in the images, such as the structure of the scene, while the fine-grained depth difference may be overlooked. In this paper, we propose a novel multi-scale depth normalization method that hierarchically normalizes the depth representations based on spatial information and depth distributions. Compared with previous normalization strategies applied only at the holistic image level, the proposed hierarchical normalization can effectively preserve the fine-grained details and improve accuracy. We present two strategies that define the hierarchical normalization contexts in the depth domain and the spatial domain, respectively. Our extensive experiments show that the proposed normalization strategy remarkably outperforms previous normalization methods, and we set new state-of-the-art on five zero-shot transfer benchmark datasets.

Motivation & Objective

• To address the limitation of instance-level normalization in monocular depth estimation, which emphasizes global scene structure but overlooks fine-grained depth differences.
• To design a normalization strategy that balances global scene structure and local depth detail preservation during training.
• To enable effective mix-dataset training across diverse depth annotations with scale and shift inconsistencies.
• To improve zero-shot generalization by enhancing model robustness to in-the-wild image variations.
• To develop a flexible normalization mechanism that operates across spatial and depth domains with configurable context levels.

Proposed method

• Proposes Hierarchical Depth Normalization (HDN), which normalizes depth representations using multi-scale contexts in both spatial and depth domains.
• In the spatial domain, the image is divided into grids, and normalization is performed per cell to emphasize local depth differences.
• In the depth domain, pixels are grouped by ground-truth depth values into bins, enabling context-aware normalization based on depth magnitude.
• Multiple normalization levels are combined during training to balance local detail and global structure.
• The method uses a learnable normalization loss that preserves relative depth differences across different context scales.
• The approach is integrated into a standard monocular depth estimation network, with no architectural changes required.

Experimental results

Research questions

• RQ1 Can hierarchical normalization improve fine-grained depth detail preservation compared to instance-level normalization?
• RQ2 Does multi-scale normalization in spatial and depth domains enhance zero-shot generalization on diverse benchmarks?
• RQ3 How does the number of hierarchical context levels affect model performance and robustness?
• RQ4 Can HDN effectively handle domain shifts in mix-dataset training with inconsistent depth scales?
• RQ5 Does the proposed method outperform state-of-the-art models under zero-shot transfer settings?

Key findings

• HDN-S (spatial domain) reduces AbsRel to 8.58 on the KITTI benchmark, outperforming instance-level normalization (9.81).
• HDN-DR (depth and spatial domain) achieves 6.9 AbsRel on NYU, a 24.3% improvement over the previous SOTA (9.0).
• On ETH3D, HDN-DR achieves 8.0 AbsRel and 83.3% δ1, setting a new SOTA in both metrics.
• The method consistently outperforms baselines across five zero-shot transfer benchmarks, including DIODE, KITTI, NYU, ScanNet, and ETH3D.
• Ablation studies show that using multiple context levels improves performance, while relying only on the lowest level harms accuracy.
• Qualitative results demonstrate smoother surfaces, sharper edges, and more detailed predictions compared to instance-level normalization.