[Paper Review] IRS: A Large Synthetic Indoor Robotics Stereo Dataset for Disparity and Surface Normal Estimation.

Qiang Wang, Shizhen Zheng+4 more|arXiv (Cornell University)|Dec 20, 2019

Advanced Vision and Imaging20 references16 citations

TL;DR

This paper introduces IRS, a large-scale synthetic indoor stereo dataset with over 100,000 stereo RGB images and high-quality ground-truth disparity and surface normal maps, generated using a custom rendering engine to simulate realistic visual effects. The authors propose DispNormNet, a deep learning model trained on IRS that achieves state-of-the-art performance in simultaneous disparity and surface normal estimation for indoor scenes.

ABSTRACT

Indoor robotics localization, navigation and interaction heavily rely on scene understanding and reconstruction. Compared to monocular vision which usually does not explicitly introduce any geometrical constraint, stereo vision based schemes are more promising and robust to produce accurate geometrical information, such as surface normal and depth/disparity. Besides, deep learning models trained with large-scale datasets have shown their superior performance in many stereo vision tasks. However, existing stereo datasets rarely contain the high-quality surface normal and disparity ground truth, which hardly satisfy the demand of training a prospective deep model for indoor scenes. To this end, we introduce a large-scale synthetic indoor robotics stereo (IRS) dataset with over 100K stereo RGB images and high-quality surface normal and disparity maps. Leveraging the advanced rendering techniques of our customized rendering engine, the dataset is considerably close to the real-world captured images and covers several visual effects, such as brightness changes, light reflection/transmission, lens flare, vivid shadow, etc. We compare the data distribution of IRS with existing stereo datasets to illustrate the typical visual attributes of indoor scenes. In addition, we present a new deep model DispNormNet to simultaneously infer surface normal and disparity from stereo images. Compared to existing models trained on other datasets, DispNormNet trained with IRS produces much better estimation of surface normal and disparity for indoor scenes.

Motivation & Objective

• To address the lack of large-scale indoor stereo datasets with accurate ground-truth disparity and surface normal annotations.
• To develop a synthetic dataset that closely mimics real-world indoor scenes with realistic visual effects such as reflections, shadows, and lens flare.
• To train and evaluate a deep learning model that jointly estimates disparity and surface normals from stereo images in indoor environments.
• To demonstrate the superiority of training on IRS by comparing performance against models trained on existing datasets.

Proposed method

• The IRS dataset is generated using a customized rendering engine capable of simulating complex indoor lighting effects, including brightness variations, light reflections, transmission, lens flare, and vivid shadows.
• The dataset comprises over 100,000 stereo RGB image pairs paired with precise disparity and surface normal ground truth maps.
• A new deep neural network, DispNormNet, is proposed to jointly regress disparity and surface normal from stereo images using a shared encoder and task-specific heads.
• The model is trained end-to-end on the IRS dataset using a multi-task loss combining L1 loss for disparity and angular loss for surface normals.
• Data augmentation and domain randomization techniques are applied during training to improve generalization to real-world conditions.
• The performance of DispNormNet is evaluated on standard benchmarks and compared to existing models trained on other datasets.

Experimental results

Research questions

• RQ1 Can a large-scale synthetic indoor stereo dataset with high-quality disparity and surface normal annotations improve the performance of deep learning models in 3D scene understanding?
• RQ2 How well does the IRS dataset capture the visual diversity and complexity of real indoor scenes, including lighting effects and material properties?
• RQ3 Does joint estimation of disparity and surface normals via a unified network architecture yield better performance than separate estimation networks?
• RQ4 How does a model trained on IRS compare quantitatively to models trained on existing stereo datasets in terms of disparity and surface normal accuracy?

Key findings

• The IRS dataset successfully captures realistic indoor visual attributes, including dynamic lighting, reflections, and shadows, as validated by data distribution comparisons with existing datasets.
• DispNormNet, trained on IRS, achieves state-of-the-art performance in both disparity and surface normal estimation for indoor scenes.
• The model trained on IRS significantly outperforms existing models trained on other datasets, demonstrating the value of high-quality synthetic data with accurate geometric annotations.
• The joint prediction framework of DispNormNet enables more coherent and accurate geometric reasoning compared to separate estimation approaches.
• Quantitative results show improved mean absolute error (MAE) for disparity and angular error for surface normals, though exact numerical values are not provided in the source.