[Paper Review] A Reflectance Based Method For Shadow Detection and Removal

Sri Kalyan Yarlagadda, Fengqing Zhu|arXiv (Cornell University)|Jul 11, 2018

Video Surveillance and Tracking Methods10 references4 citations

TL;DR

This paper proposes a non-training-based shadow detection method using reflectance, texture, and luminance characteristics to identify shadow-non-shadow segment pairs in color images. By leveraging median color differences, angle thresholds, and iterative refinement, the method achieves high shadow detection accuracy (90.6% on UIUC, 92.0% on UCF) with fewer parameters than deep learning or CRF-based approaches.

ABSTRACT

Shadows are common aspect of images and when left undetected can hinder scene understanding and visual processing. We propose a simple yet effective approach based on reflectance to detect shadows from single image. An image is first segmented and based on the reflectance, illumination and texture characteristics, segments pairs are identified as shadow and non-shadow pairs. The proposed method is tested on two publicly available and widely used datasets. Our method achieves higher accuracy in detecting shadows compared to previous reported methods despite requiring fewer parameters. We also show results of shadow-free images by relighting the pixels in the detected shadow regions.

Motivation & Objective

• To address the challenge of accurate shadow detection in single-color images, especially when shadows and shading are visually ambiguous.
• To develop a method that requires fewer parameters than deep learning or CRF-based approaches while maintaining high detection accuracy.
• To enable effective shadow removal by accurately identifying shadow regions through reflectance and illumination consistency.
• To improve robustness in complex scenes by using segment-level comparisons based on reflectance, texture, and luminance.

Proposed method

• The method begins with image segmentation using the Quickshift algorithm with a Gaussian kernel of size 9 to group pixels with similar reflectance and illumination.
• A reflectance classifier computes the angle between the median color of the non-shadow segment and the difference vector (I_NS - I_S), applying a 10° threshold to identify same-material pairs.
• A luminance classifier ensures shadow segments are darker by enforcing a gray-scale threshold via Otsu’s method, limiting shadow regions to low-luminance segments.
• A texture classifier compares local texture features (using local binary patterns) between segment pairs to reinforce shadow detection confidence.
• A connection graph is built from initial segment pairs and refined iteratively using geometric and intensity consistency to improve mask accuracy.
• Refinement involves re-segmenting with a smaller kernel (size 3) and labeling segments as shadow or non-shadow based on whether >70% of pixels are below the Otsu threshold.

Experimental results

Research questions

• RQ1 Can a non-training-based method achieve high shadow detection accuracy using only reflectance and illumination consistency?
• RQ2 How effective is the angle criterion between median color vectors in distinguishing shadow from shading under varying lighting conditions?
• RQ3 Can segment-level comparisons of reflectance, texture, and luminance outperform learning-based methods with fewer parameters?
• RQ4 To what extent does iterative refinement of segment connections improve shadow mask accuracy?

Key findings

• On the UIUC dataset, the method achieved a shadow detection accuracy of 90.6%, outperforming unary + pairwise (71.6%) and ConvNet (84.7%) methods.
• The method recorded a Balanced Error Rate (BER) of 0.119 on UIUC, slightly higher than ConvNet’s 0.099 but still competitive with fewer parameters.
• On the more complex UCF dataset, the method achieved 92.0% shadow detection accuracy, surpassing BDT-BCRF (63.9%), unary + pairwise (73.3%), and ConvNet (78.0%) methods.
• The BER on UCF was 0.1265, which is better than all compared methods, indicating strong performance on imbalanced data.
• The method successfully removed shadows in test images using relighting techniques, demonstrating practical utility in image enhancement.
• Refinement using a smaller kernel and Otsu threshold improved boundary accuracy, reducing false positives from very bright segments.