[Paper Review] Local Binary Pattern(LBP) Optimization for Feature Extraction

Zeinab Sedaghatjoo, Hossein Hosseinzadeh+1 more|arXiv (Cornell University)|Jul 26, 2024

Advanced Algorithms and Applications4 citations

TL;DR

This paper proposes a novel SVD-based optimization of Local Binary Pattern (LBP) features by decomposing the LBP operator into three fixed matrices, two of which are optimized via singular value decomposition to enhance classification performance. The method yields significantly improved accuracy in face detection and facial expression recognition using far fewer features than standard LBP.

ABSTRACT

The rapid growth of image data has led to the development of advanced image processing and computer vision techniques, which are crucial in various applications such as image classification, image segmentation, and pattern recognition. Texture is an important feature that has been widely used in many image processing tasks. Therefore, analyzing and understanding texture plays a pivotal role in image analysis and understanding.Local binary pattern (LBP) is a powerful operator that describes the local texture features of images. This paper provides a novel mathematical representation of the LBP by separating the operator into three matrices, two of which are always fixed and do not depend on the input data. These fixed matrices are analyzed in depth, and a new algorithm is proposed to optimize them for improved classification performance. The optimization process is based on the singular value decomposition (SVD) algorithm. As a result, the authors present optimal LBPs that effectively describe the texture of human face images. Several experiment results presented in this paper convincingly verify the efficiency and superiority of the optimized LBPs for face detection and facial expression recognition tasks.

Motivation & Objective

• To address the suboptimal performance of standard LBP in texture classification by identifying optimal LBP patterns.
• To mathematically reformulate the LBP process using matrix decomposition for deeper analysis and optimization.
• To reduce the number of required LBP features while maintaining or improving classification accuracy.
• To develop a systematic method for selecting optimal LBP patterns based on singular value decomposition of fixed matrices.
• To validate the proposed method on real-world datasets for face detection and facial expression recognition.

Proposed method

• Represents the LBP operator as a product of three matrices: LBP matrix (E), tiling matrix (L), and histogram matrix (H), enabling mathematical analysis.
• Identifies two of the three matrices (L and H) as fixed and independent of input data, allowing for pre-optimization.
• Applies singular value decomposition (SVD) to the fixed matrices L and H to compute optimal values that maximize classification performance.
• Derives optimal 8-bit binary LBP patterns that are predominantly uniform (with at most two transitions between 0 and 1).
• Proposes a new algorithm (Algorithm 2) to extract only the most discriminative LBP features based on SVD-optimized matrices.
• Uses linear SVM for classification with cross-validation to evaluate performance on face and expression recognition tasks.

Figure 1: LBP values computation process. Each $3\times 3$ pixel block in the image is encoded to a LBP value.

Experimental results

Research questions

• RQ1 Can the LBP feature extraction process be mathematically decomposed into fixed and variable components to enable optimization?
• RQ2 How can singular value decomposition (SVD) be applied to the fixed matrices of the LBP operator to derive optimal LBP patterns?
• RQ3 Does the optimized LBP feature set achieve higher classification accuracy than standard LBP with fewer features?
• RQ4 To what extent do the optimized LBP patterns improve performance in face detection and facial expression recognition tasks?
• RQ5 Can the proposed SVD-based optimization be generalized to other texture descriptors or larger pixel neighborhoods?

Key findings

• The proposed SVD-optimized LBP features achieve over 98% accuracy in face vs. clutter classification using only 16 features, compared to 256 features required by standard LBP.
• In facial expression recognition on the CK+ dataset, the optimized LBP features achieve an average accuracy of approximately 90%, outperforming the standard LBP’s 87% despite using fewer features.
• The optimal LBP patterns identified are predominantly uniform patterns, with at most two transitions between 0 and 1, confirming their relevance for texture analysis.
• The confusion matrix for the optimized LBP shows significantly fewer misclassifications across all emotion classes compared to standard LBP, especially for rare classes like contempt and disgust.
• The method reduces the number of required LBP features by more than half while improving or maintaining classification accuracy, demonstrating high efficiency.
• The optimal LBP values are consistent across both face detection and facial expression recognition tasks, indicating robustness and generalizability of the optimized patterns.

Figure 2: LBP feature extraction process. The image is divided into 16 sub-regions and LBP values are extracted from each region. Then the histogram is applied on the LBP values to extract $8$ features for each sub-region. Then, finally, $128=16\times 8$ features are exploited from the image as LBP