[Paper Review] GroundNet: Segmentation-Aware Monocular Ground Plane Estimation with Geometric Consistency.

Yunze Man, Xinshuo Weng+1 more|arXiv (Cornell University)|Nov 17, 2018

Robotics and Sensor-Based Localization57 references6 citations

TL;DR

GroundNet proposes a multi-task learning framework for monocular ground plane estimation that jointly predicts surface normals, 2D ground segmentation, and depth by leveraging two separate streams for normal estimation and enforcing geometric consistency via a consistency loss. The model achieves state-of-the-art performance on ground plane normal estimation and horizon line detection on KITTI and ApolloScape datasets.

ABSTRACT

We focus on the problem of estimating the orientation of the ground plane with respect to a mobile monocular camera platform (e.g., ground robot, wearable camera, assistive robotic platform). To address this problem, we formulate the ground plane estimation problem as an inter-mingled multi-task prediction problem by jointly optimizing for point-wise surface normal direction, 2D ground segmentation, and depth estimates. Our proposed model -- GroundNet -- estimates the ground normal in two streams separately and then a consistency loss is applied on top of the two streams to enforce geometric consistency. A semantic segmentation stream is used to isolate the ground regions and are used to selectively back-propagate parameter updates only through the ground regions in the image. Our experiments on KITTI and ApolloScape datasets verify that the GroundNet is able to predict consistent depth and normal within the ground region. It also achieves top performance on ground plane normal estimation and horizon line detection.

Motivation & Objective

• To improve ground plane orientation estimation for mobile monocular camera platforms such as ground robots and wearable devices.
• To address the challenge of inconsistent depth and normal predictions in non-ground regions by enforcing geometric consistency.
• To enhance estimation accuracy by selectively back-propagating gradients only through ground-identified regions using a semantic segmentation stream.
• To achieve state-of-the-art performance in ground plane normal estimation and horizon line detection.
• To ensure robustness and consistency in depth and normal predictions across the ground plane using a multi-task learning framework.

Proposed method

• GroundNet formulates ground plane estimation as a multi-task learning problem combining surface normal prediction, 2D ground segmentation, and depth estimation.
• It uses two separate streams to predict surface normals, which are then aligned via a geometric consistency loss to ensure spatial coherence.
• A semantic segmentation stream identifies ground regions and restricts back-propagation to these areas, improving training efficiency and localization accuracy.
• The consistency loss enforces alignment between the two normal prediction streams, ensuring geometric plausibility across the ground plane.
• The model jointly optimizes for all three tasks—normal, depth, and segmentation—through end-to-end training on monocular RGB images.
• Training is conducted on KITTI and ApolloScape datasets, with supervision from ground-truth normals, depth, and semantic labels.

Experimental results

Research questions

• RQ1 Can a multi-task learning framework improve the consistency and accuracy of monocular ground plane normal estimation?
• RQ2 How does enforcing geometric consistency between dual normal prediction streams affect estimation quality?
• RQ3 To what extent does selective back-propagation through ground-identified regions improve model performance?
• RQ4 Can joint prediction of depth and segmentation enhance normal estimation on the ground plane?
• RQ5 How does GroundNet compare to state-of-the-art methods in ground plane normal estimation and horizon line detection?

Key findings

• GroundNet achieves state-of-the-art performance on ground plane normal estimation on both KITTI and ApolloScape datasets.
• The model produces consistent depth and normal predictions within the ground region, as validated by geometric consistency loss.
• The use of a semantic segmentation stream enables selective gradient updates, improving training efficiency and localization precision on ground areas.
• The geometric consistency loss significantly improves the spatial coherence of predicted normals across the ground plane.
• GroundNet demonstrates strong performance in horizon line detection, indicating accurate estimation of ground plane orientation.
• The joint optimization of normal, depth, and segmentation leads to more robust and reliable predictions compared to single-task baselines.