[Paper Review] An Algorithm and Heuristic based on Normalized Mutual Information for Dimensionality Reduction and Classification of Hyperspectral images

Elkebir Sarhrouni, Ahmed Hammouch+1 more|arXiv (Cornell University)|Oct 22, 2022

Remote-Sensing Image Classification3 citations

TL;DR

This paper proposes a two-stage heuristic algorithm using normalized mutual information (NMI) to select non-reduundant, informative bands from hyperspectral images (HSI) for improved classification. It first selects bands with high mutual information to the ground truth (relevance), then applies NMI to eliminate redundant bands, achieving 84.24% classification accuracy with only 42 bands from the AVIRIS 92AV3C dataset, demonstrating effective dimensionality reduction without data transformation.

ABSTRACT

In the feature classification domain, the choice of data affects widely the results. The Hyperspectral image (HSI), is a set of more than a hundred bidirectional measures (called bands), of the same region (called ground truth map: GT). The HSI is modelized at a set of N vectors. So we have N features (or attributes) expressing N vectors of measures for C substances (called classes). The problematic is that it's pratically impossible to investgate all possible subsets. So we must find K vectors among N, such as relevant and no redundant ones; in order to classify substances. Here we introduce an algorithm based on Normalized Mutual Information to select relevant and no redundant bands, necessary to increase classification accuracy of HSI. Keywords: Feature Selection, Normalized Mutual information, Hyperspectral images, Classification, Redundancy.

Motivation & Objective

• To address the challenge of high-dimensional feature space in hyperspectral image (HSI) classification, where redundant and irrelevant bands degrade performance.
• To develop a filter-based feature selection method that separates relevance and redundancy assessment for improved efficiency and control.
• To reduce dimensionality without transforming the original data, preserving interpretability and enabling real-time application.
• To achieve high classification accuracy with minimal bands by leveraging normalized mutual information as a redundancy metric.
• To provide a tunable threshold-based heuristic for practical deployment in commercial and real-time systems.

Proposed method

• The method uses mutual information (MI) between each band and the ground truth map (GT) to rank band relevance, selecting those with high MI values.
• A two-stage process is employed: first, bands are filtered based on a relevance threshold (IM) to retain only informative ones.
• Second, normalized mutual information (NMI) is computed between all pairs of candidate bands to detect and remove redundant bands.
• Two forms of NMI are evaluated: one based on symmetric matrix computation and another optimized form to reduce computational complexity.
• The algorithm applies a redundancy threshold (TH) to eliminate bands with high NMI values, ensuring diversity in the final subset.
• The final band subset is used to train a classifier, with performance validated on the AVIRIS 92AV3C dataset.

Experimental results

Research questions

• RQ1 Can normalized mutual information effectively measure redundancy between hyperspectral bands in a way that supports efficient feature selection?
• RQ2 How does separating relevance and redundancy assessment improve classification performance compared to joint thresholding?
• RQ3 What is the optimal balance between relevance (high MI with GT) and redundancy (low NMI between bands) for maximizing classification accuracy?
• RQ4 Can the proposed heuristic achieve high accuracy with significantly fewer bands than the original 220, without data transformation?
• RQ5 How do different threshold combinations (IM and TH) affect the number of selected bands and resulting classification accuracy?

Key findings

• The proposed algorithm achieved a classification accuracy of 84.24% using only 42 bands from the AVIRIS 92AV3C dataset, demonstrating high efficiency in dimensionality reduction.
• With just 17 bands, the method reached 80% classification accuracy, highlighting its ability to preserve critical information in a minimal feature set.
• The second form of normalized mutual information produced nearly identical results (84.16% accuracy) with 42 bands, confirming robustness and consistency across formulations.
• The method effectively separates relevance and redundancy control, outperforming approaches that apply a single threshold to both criteria.
• Zone 4 in the threshold analysis (low TH, high TU) yielded the most effective results, showing that hard redundancy control combined with high relevance yields optimal performance.
• The algorithm generalizes well to unlabelled regions, successfully classifying pixels not present in the original ground truth map, indicating strong generalization capability.