[Paper Review] A Conditional Random Field Model for Context Aware Cloud Detection in Sky Images

Vijai Jayadevan, Jeffrey J. Rodrı́guez+1 more|arXiv (Cornell University)|Jun 18, 2019

Remote Sensing in Agriculture28 references4 citations

TL;DR

This paper proposes a conditional random field (CRF) model that improves cloud detection in ground-based sky images by integrating a discriminative classifier (logistic regression) with higher-order clique potentials to model spatial context. By segmenting images into homogeneous regions via mean shift clustering and applying iterative inference using the ICM algorithm, the method achieves significantly higher accuracy than adaptive or fixed thresholding techniques, particularly in regions with overlapping cloud-sky features.

ABSTRACT

A conditional random field (CRF) model for cloud detection in ground based sky images is presented. We show that very high cloud detection accuracy can be achieved by combining a discriminative classifier and a higher order clique potential in a CRF framework. The image is first divided into homogeneous regions using a mean shift clustering algorithm and then a CRF model is defined over these regions. The various parameters involved are estimated using training data and the inference is performed using Iterated Conditional Modes (ICM) algorithm. We demonstrate how taking spatial context into account can boost the accuracy. We present qualitative and quantitative results to prove the superior performance of this framework in comparison with other state of the art methods applied for cloud detection.

Motivation & Objective

• To address the limitations of fixed and adaptive thresholding methods in cloud detection due to feature space overlap and high intra-class variance.
• To improve segmentation accuracy by incorporating spatial context through a CRF framework that models relationships between image regions.
• To develop a flexible, context-aware cloud detection system suitable for ground-based sky imaging in solar power forecasting applications.
• To demonstrate that combining discriminative classification with higher-order clique potentials enhances performance over pixel-wise classification.

Proposed method

• The input sky image is segmented into homogeneous regions using the mean shift clustering algorithm to reduce noise and improve computational efficiency.
• A conditional random field (CRF) model is defined over these regions, with the association potential derived from logistic regression using NSV and NBR ratios as features.
• The interaction potential is modeled using a higher-order clique potential that captures spatial context and enforces smoothness across neighboring regions.
• Model parameters are estimated using training data, and inference is performed via the Iterated Conditional Modes (ICM) algorithm to maximize the posterior probability.
• The CRF framework allows for flexible integration of additional features into the association or interaction potentials, enhancing adaptability to different imaging setups.
• Performance is evaluated using standard metrics: accuracy, precision, and recall, computed from confusion matrices on test data.

Experimental results

Research questions

• RQ1 Can a CRF-based approach outperform traditional adaptive and fixed thresholding methods in cloud detection from ground-based sky images?
• RQ2 How does incorporating spatial context through higher-order clique potentials improve segmentation accuracy in the presence of overlapping cloud and sky features?
• RQ3 To what extent does region-based classification using mean shift segmentation reduce noise and improve robustness compared to pixel-wise classification?
• RQ4 Does the fusion of discriminative classification (logistic regression) with CRF modeling lead to better performance than standalone thresholding or classification?

Key findings

• The CRF-based method achieved significantly higher accuracy than both adaptive thresholding (using NSV or NBR ratios) and fixed thresholding, with non-overlapping 98% confidence intervals.
• The CRF model improved accuracy by 12.5 percentage points over the best-performing adaptive thresholding method (NSV-based) on Set C of the test database.
• Incorporating the interaction potential (spatial context) corrected misclassifications made by the association potential alone, especially in regions with histogram overlap.
• Region-based classification using mean shift segmentation reduced noise and produced more coherent segmentation results than pixel-wise logistic regression.
• The CRF model demonstrated robustness to variations in sky brightness and cloud morphology, outperforming methods that rely solely on feature thresholds.
• The method is highly extensible, allowing for easy integration of additional features into the association or interaction potentials for different imaging systems.