[Paper Review] GCNDepth: Self-supervised Monocular Depth Estimation based on Graph Convolutional Network

Armin Masoumian, Hatem A. Rashwan+3 more|arXiv (Cornell University)|Dec 13, 2021

Advanced Vision and Imaging6 citations

TL;DR

This paper proposes GCNDepth, a self-supervised monocular depth estimation model that leverages graph convolutional networks (GCNs) to improve depth map accuracy by modeling topological relationships between pixels. By integrating multi-scale GCNs in a U-Net-like decoder and combining photometric, reprojection, and smoothness losses, the method achieves state-of-the-art performance with 89% accuracy on KITTI and Make3D, while reducing parameters by 40% compared to prior methods.

ABSTRACT

Depth estimation is a challenging task of 3D reconstruction to enhance the accuracy sensing of environment awareness. This work brings a new solution with a set of improvements, which increase the quantitative and qualitative understanding of depth maps compared to existing methods. Recently, a convolutional neural network (CNN) has demonstrated its extraordinary ability in estimating depth maps from monocular videos. However, traditional CNN does not support topological structure and they can work only on regular image regions with determined size and weights. On the other hand, graph convolutional networks (GCN) can handle the convolution on non-Euclidean data and it can be applied to irregular image regions within a topological structure. Therefore, in this work in order to preserve object geometric appearances and distributions, we aim at exploiting GCN for a self-supervised depth estimation model. Our model consists of two parallel auto-encoder networks: the first is an auto-encoder that will depend on ResNet-50 and extract the feature from the input image and on multi-scale GCN to estimate the depth map. In turn, the second network will be used to estimate the ego-motion vector (i.e., 3D pose) between two consecutive frames based on ResNet-18. Both the estimated 3D pose and depth map will be used for constructing a target image. A combination of loss functions related to photometric, projection, and smoothness is used to cope with bad depth prediction and preserve the discontinuities of the objects. In particular, our method provided comparable and promising results with a high prediction accuracy of 89% on the publicly KITTI and Make3D datasets along with a reduction of 40% in the number of trainable parameters compared to the state of the art solutions. The source code is publicly available at https://github.com/ArminMasoumian/GCNDepth.git

Motivation & Objective

• To improve monocular depth estimation accuracy by modeling non-Euclidean, topological relationships between image pixels, overcoming limitations of traditional CNNs.
• To reduce the number of trainable parameters in depth estimation networks without sacrificing performance.
• To enhance depth map quality by preserving object boundaries and discontinuities through geometric-aware feature learning.
• To develop a self-supervised framework that eliminates the need for expensive ground-truth depth annotations.
• To enable robust depth prediction in low-texture and challenging lighting conditions.

Proposed method

• The model employs a dual-branch architecture: one branch uses ResNet-50 with multi-scale GCNs for depth estimation, and another uses ResNet-18 for ego-motion prediction between consecutive frames.
• GCNs are applied in the decoder to propagate features across pixels based on learned topological structures, capturing long-range dependencies and preserving geometric consistency.
• A multi-scale GCN design is implemented across decoder layers, using different neighborhood receptive fields to model spatial correlations at varying levels of granularity.
• The training objective combines photometric loss (between original and reconstructed images), reprojection loss (to handle occlusions), and smoothness loss (to preserve edges and reduce texture bias).
• The model uses a differentiable image reconstruction pipeline: predicted depth and estimated pose are used to warp the source frame to reconstruct the target frame.
• The loss function is optimized end-to-end using backpropagation, with the GCN layers enabling structured feature propagation beyond regular grid convolutions.

Experimental results

Research questions

• RQ1 Can graph convolutional networks improve depth estimation accuracy by modeling pixel relationships beyond regular grid structures?
• RQ2 How does multi-scale GCN integration in the decoder enhance feature representation and depth map quality?
• RQ3 To what extent can a self-supervised GCN-based model reduce parameter count while maintaining or improving performance compared to CNN-based SOTA methods?
• RQ4 Can GCN-based feature learning better preserve object boundaries and handle low-texture regions in depth estimation?
• RQ5 How does the combination of photometric, reprojection, and smoothness losses affect reconstruction fidelity and depth map discontinuity preservation?

Key findings

• GCNDepth achieved a test error of 0.424 on the KITTI dataset in terms of Abs-Rel, outperforming other self-supervised methods and approaching supervised baselines.
• On the Make3D dataset, GCNDepth achieved an RMSE of 6.757, a Sq-Rel of 3.075, and an RMSE-log of 0.107, demonstrating strong generalization without fine-tuning.
• The model reduced the number of trainable parameters by 40% compared to state-of-the-art self-supervised methods, indicating improved parameter efficiency.
• Qualitative results show that GCNDepth better preserves object boundaries and depth discontinuities, especially in low-texture and complex illumination regions.
• The model outperformed Monodepth2 in Sq-Rel and RMSE-log on Make3D, despite Monodepth2 achieving the best Abs-Rel among self-supervised methods.
• The use of GCNs enabled better reconstruction of depth in challenging regions such as columns, greenery, and buildings, with clearer depth boundaries than baseline models.