[論文レビュー] GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose

Zhichao Yin, Jianping Shi|arXiv (Cornell University)|Mar 6, 2018

Advanced Vision and Imaging参考文献 39被引用数 122

ひとこと要約

GeoNet は動画から監督なし・幾何学駆動のフレームワークで、2段階カスケードと適応幾何学的一貫性を用いて動的領域と遮蔽を処理し、単眼深度、光学流、ego-motion を同時に学習する。KITTI における監督なし法の中で最先端の結果を達成し、監督付きアプローチと同等程度に匹敵する。

ABSTRACT

We propose GeoNet, a jointly unsupervised learning framework for monocular depth, optical flow and ego-motion estimation from videos. The three components are coupled by the nature of 3D scene geometry, jointly learned by our framework in an end-to-end manner. Specifically, geometric relationships are extracted over the predictions of individual modules and then combined as an image reconstruction loss, reasoning about static and dynamic scene parts separately. Furthermore, we propose an adaptive geometric consistency loss to increase robustness towards outliers and non-Lambertian regions, which resolves occlusions and texture ambiguities effectively. Experimentation on the KITTI driving dataset reveals that our scheme achieves state-of-the-art results in all of the three tasks, performing better than previously unsupervised methods and comparably with supervised ones.

研究の動機と目的

• Motivate learning 3D scene geometry from monocular video without ground-truth labels.
• Decompose scene motion into rigid (depth+ego-motion) and non-rigid (object motion) components.
• Enforce geometric consistency to robustly handle occlusions and non-Lambertian regions.
• Propose an end-to-end two-stage architecture to improve unsupervised depth, flow, and pose estimation.
• Evaluate on KITTI showing competitive performance against supervised methods.

提案手法

• Two-stage cascaded architecture: first stage rigid structure reconstructor (DepthNet + PoseNet) to estimate depth and camera motion, producing rigid flow; second stage ResFlowNet to learn residual non-rigid flow and refine full flow.
• End-to-end differentiable view synthesis losses that compare synthesized views to target frames (photometric loss combining SSIM and L1).
• Edge-aware depth smoothness loss to preserve details while encouraging smooth depth in textureless regions.
• Adaptive geometric consistency loss that mimics forward-backward consistency, selectively enforcing coherence only in non-occluded regions to mitigate occlusions and non-Lambertian effects.
• Joint loss L = sum over scales and frame pairs of: L_rw + lambda_ds*L_ds + L_fw + lambda_fs*L_fs + lambda_gc*L_gc.

実験結果

リサーチクエスチョン

• RQ1 Can monocular depth, optical flow, and ego-motion be learned jointly in an unsupervised manner from video?
• RQ2 How can static and dynamic scene components be separated and leveraged to improve robustness to occlusions and non-Lambertian regions?
• RQ3 Does enforcing adaptive geometric consistency improve prediction quality for depth, flow, and pose in unsupervised learning?
• RQ4 What are the trade-offs and performance gains of a two-stage architecture (rigid structure vs. non-rigid motion) on KITTI?

主な発見

• GeoNet achieves state-of-the-art results among unsupervised methods on KITTI for depth, flow, and camera pose estimation.
• The two-stage architecture (DepthNet + PoseNet followed by ResFlowNet) effectively decomposes rigid and non-rigid motion, improving overall predictions.
• Adaptive geometric consistency loss improves robustness to occlusions and texture ambiguities by selectively enforcing prediction coherence in non-occluded regions.
• On KITTI, GeoNet’s unsupervised depth estimation and pose estimation are competitive with supervised approaches, and its flow results surpass prior unsupervised baselines.
• Ablation studies show the importance of geometric consistency and residual flow in handling dynamic objects and challenging regions (occlusions, lighting).