[Paper Review] A Comparison of Stereo-Matching Cost between Convolutional Neural Network and Census for Satellite Images

Bihe Chen, Rongjun Qin+3 more|arXiv (Cornell University)|May 22, 2019

Advanced Vision and Imaging6 references4 citations

TL;DR

This paper compares stereo-matching cost computation using a fast Convolutional Neural Network (fst-CNN) and the Census transform on satellite images. Using semi-global matching, fst-CNN achieves higher absolute matching accuracy, better error distribution, and improved completeness than Census, though both methods perform in the same order of magnitude.

ABSTRACT

Stereo dense image matching can be categorized to low-level feature based matching and deep feature based matching according to their matching cost metrics. Census has been proofed to be one of the most efficient low-level feature based matching methods, while fast Convolutional Neural Network (fst-CNN), as a deep feature based method, has small computing time and is robust for satellite images. Thus, a comparison between fst-CNN and census is critical for further studies in stereo dense image matching. This paper used cost function of fst-CNN and census to do stereo matching, then utilized semi-global matching method to obtain optimized disparity images. Those images are used to produce digital surface model to compare with ground truth points. It addresses that fstCNN performs better than census in the aspect of absolute matching accuracy, histogram of error distribution and matching completeness, but these two algorithms still performs in the same order of magnitude.

Motivation & Objective

• To evaluate and compare the performance of deep learning-based (fst-CNN) and low-level feature-based (Census) stereo-matching cost metrics on satellite images.
• To assess the impact of different cost functions on disparity map quality and digital surface model (DSM) accuracy.
• To determine whether the computational efficiency of fst-CNN translates into superior matching performance compared to Census in remote sensing applications.
• To analyze matching completeness and error distribution as key performance indicators for stereo matching in challenging satellite imagery.

Proposed method

• Applied the fst-CNN-based cost function to compute stereo-matching costs using deep features extracted from convolutional layers.
• Used the Census transform to compute matching costs based on local intensity patterns in a sliding window.
• Employed semi-global matching (SGM) to optimize disparity maps from both cost functions.
• Generated digital surface models (DSMs) from the optimized disparity maps for comparison with ground truth points.
• Evaluated performance using absolute matching accuracy, error distribution histograms, and matching completeness metrics.
• Conducted experiments on satellite image pairs to ensure relevance to real-world remote sensing scenarios.

Experimental results

Research questions

• RQ1 How does the stereo-matching accuracy of fst-CNN compare to Census in satellite imagery?
• RQ2 What is the difference in error distribution between fst-CNN and Census-based stereo matching?
• RQ3 How do fst-CNN and Census perform in terms of matching completeness on satellite images?
• RQ4 Do the computational advantages of fst-CNN translate into measurable performance gains over Census?
• RQ5 Are the performance differences between fst-CNN and Census statistically significant in the context of digital surface model generation?

Key findings

• fst-CNN achieves higher absolute matching accuracy than the Census transform on satellite images.
• The error distribution of fst-CNN is more tightly clustered around zero compared to Census, indicating fewer large errors.
• fst-CNN demonstrates superior matching completeness, particularly in texture-poor or low-contrast regions common in satellite imagery.
• Despite performance differences, both fst-CNN and Census perform in the same order of magnitude, indicating that Census remains a strong baseline.
• The disparity maps generated from fst-CNN produce more accurate digital surface models when validated against ground truth points.
• The results confirm that deep feature-based methods like fst-CNN outperform traditional low-level feature methods such as Census in stereo matching for satellite images.