[Paper Review] TreeSegNet: Adaptive Tree CNNs for Subdecimeter Aerial Image Segmentation

Kai Yue, Lei Yang+4 more|arXiv (Cornell University)|Apr 29, 2018

Advanced Neural Network Applications9 references3 citations

TL;DR

TreeSegNet proposes an adaptive Tree-CNN block within a DeepUNet framework to improve subdecimeter aerial image segmentation by learning class-specific feature representations through confusion matrix-driven architecture adaptation. The method achieves state-of-the-art performance on the ISPRS Potsdam dataset by fusing multiscale features via concatenating connections and optimizing ResNeXt-based nodes using a novel TreeCutting algorithm.

ABSTRACT

For the task of subdecimeter aerial imagery segmentation, fine-grained semantic segmentation results are usually difficult to obtain because of complex remote sensing content and optical conditions. Recently, convolutional neural networks (CNNs) have shown outstanding performance on this task. Although many deep neural network structures and techniques have been applied to improve the accuracy, few have paid attention to better differentiating the easily confused classes. In this paper, we propose TreeSegNet which adopts an adaptive network to increase the classification rate at the pixelwise level. Specifically, based on the infrastructure of DeepUNet, a Tree-CNN block in which each node represents a ResNeXt unit is constructed adaptively according to the confusion matrix and the proposed TreeCutting algorithm. By transporting feature maps through concatenating connections, the Tree-CNN block fuses multiscale features and learns best weights for the model. In experiments on the ISPRS 2D semantic labeling Potsdam dataset, the results obtained by TreeSegNet are better than those of other published state-of-the-art methods. Detailed comparison and analysis show that the improvement brought by the adaptive Tree-CNN block is significant.

Motivation & Objective

• To address the challenge of fine-grained semantic segmentation in complex subdecimeter aerial imagery with varying optical conditions.
• To improve classification accuracy for easily confused classes in remote sensing image segmentation.
• To develop an adaptive deep learning architecture that dynamically adjusts based on class confusion patterns.
• To enhance multiscale feature fusion in U-Net-like encoders for better contextual representation.

Proposed method

• The method builds on the DeepUNet architecture and introduces a Tree-CNN block where each node is a ResNeXt unit, enabling hierarchical feature learning.
• A confusion matrix is computed from model predictions to identify frequently misclassified class pairs.
• The TreeCutting algorithm adaptively prunes and restructures the Tree-CNN block based on confusion patterns to improve decision boundaries.
• Feature maps are transported through concatenating skip connections to fuse multiscale features and enhance representation learning.
• The adaptive Tree-CNN block is trained end-to-end with the rest of the network to optimize pixel-wise classification.
• The architecture is evaluated on the ISPRS 2D semantic labeling Potsdam dataset using standard segmentation metrics.

Experimental results

Research questions

• RQ1 Can an adaptive deep learning architecture improve pixel-wise classification accuracy in subdecimeter aerial image segmentation?
• RQ2 How does confusion matrix-guided network adaptation affect performance on hard-to-distinguish classes?
• RQ3 To what extent does multiscale feature fusion via concatenating connections enhance segmentation quality?
• RQ4 Does the TreeCutting algorithm effectively restructure the network to reduce misclassification between similar classes?

Key findings

• TreeSegNet achieves state-of-the-art performance on the ISPRS Potsdam dataset, outperforming other published methods in segmentation accuracy.
• The adaptive Tree-CNN block significantly improves classification rates for confusing classes such as impervious surfaces and trees.
• The confusion matrix-based adaptation leads to a measurable reduction in inter-class misclassification, especially between visually similar classes.
• Multiscale feature fusion through concatenating connections enhances the model’s ability to capture contextual information across scales.
• Ablation studies confirm that the TreeCutting algorithm and adaptive block design contribute substantially to performance gains.
• The method demonstrates robustness under challenging optical conditions and complex remote sensing content.