[Paper Review] Real-Time Stereo Vision for Road Surface 3-D Reconstruction

Rui Fan, Yanan Liu+4 more|arXiv (Cornell University)|Jul 18, 2018

Advanced Vision and Imaging18 references4 citations

TL;DR

This paper presents a real-time stereo vision system for 3-D road surface reconstruction using perspective transformation and fast bilateral stereo (FBS) on an NVIDIA GTX 1080 GPU. By transforming the target image to align with the reference view, the method improves disparity accuracy and processing speed, achieving subpixel disparity maps with 3 mm reconstruction accuracy at 25 fps.

ABSTRACT

Stereo vision techniques have been widely used in civil engineering to acquire 3-D road data. The two important factors of stereo vision are accuracy and speed. However, it is very challenging to achieve both of them simultaneously and therefore the main aim of developing a stereo vision system is to improve the trade-off between these two factors. In this paper, we present a real-time stereo vision system used for road surface 3-D reconstruction. The proposed system is developed from our previously published 3-D reconstruction algorithm where the perspective view of the target image is first transformed into the reference view, which not only increases the disparity accuracy but also improves the processing speed. Then, the correlation cost between each pair of blocks is computed and stored in two 3-D cost volumes. To adaptively aggregate the matching costs from neighbourhood systems, bilateral filtering is performed on the cost volumes. This greatly reduces the ambiguities during stereo matching and further improves the precision of the estimated disparities. Finally, the subpixel resolution is achieved by conducting a parabola interpolation and the subpixel disparity map is used to reconstruct the 3-D road surface. The proposed algorithm is implemented on an NVIDIA GTX 1080 GPU for the real-time purpose. The experimental results illustrate that the reconstruction accuracy is around 3 mm.

Motivation & Objective

• To develop a real-time stereo vision system for 3-D road surface reconstruction that balances accuracy and speed.
• To improve disparity estimation accuracy by aligning the perspective view of the target image with the reference view before matching.
• To accelerate processing and reduce ambiguity in stereo matching using adaptive bilateral filtering on cost volumes.
• To achieve subpixel disparity resolution via parabolic interpolation for high-precision 3-D reconstruction.
• To implement the algorithm on a GPU for real-time performance suitable for field applications.

Proposed method

• The target image is transformed via perspective transformation to align with the reference image, enhancing similarity and improving matching accuracy.
• Normalized Cross-Correlation (NCC) is used to compute the initial correlation cost between image blocks, forming two 3-D cost volumes.
• Bilateral filtering is applied to the cost volumes to adaptively aggregate matching costs, reducing ambiguity and improving disparity precision.
• Subpixel disparity resolution is achieved through parabolic interpolation on the cost-optimized disparity map.
• The final disparity map is used to reconstruct the 3-D road surface using triangulation from calibrated stereo geometry.
• The algorithm is implemented on an NVIDIA GTX 1080 GPU to enable real-time processing at 25 fps.

Experimental results

Research questions

• RQ1 Can perspective transformation between stereo images improve disparity estimation accuracy and processing speed in road surface reconstruction?
• RQ2 How does bilateral filtering on cost volumes enhance the precision of stereo matching in real-time systems?
• RQ3 What is the achievable trade-off between reconstruction accuracy and processing speed in a GPU-accelerated stereo vision pipeline?
• RQ4 Can subpixel disparity estimation significantly improve 3-D surface reconstruction accuracy for road infrastructure assessment?
• RQ5 How does the system perform in real-world outdoor conditions with fixed camera baseline and limited depth resolution?

Key findings

• The proposed system achieves a reconstruction accuracy of approximately 3 mm, validated using 3D-printed sample models.
• The algorithm processes stereo images at 25 frames per second on an NVIDIA GTX 1080 GPU, enabling real-time performance.
• The implementation achieves a processing speed of 566.37 Mde/s when parameters ϱ=3 and ρ=4 are used.
• Runtime increases with higher values of ϱ and ρ, with a 2× increase in processing time when ρ increases from 2 to 4.
• The perspective transformation significantly improves matching quality by increasing image similarity between reference and target views.
• The use of bilateral filtering on cost volumes effectively reduces matching ambiguities and enhances disparity map precision.