[Paper Review] Treelogy: A Novel Tree Classifier Utilizing Deep and Hand-crafted Representations

İlke Çuğu, Eren Şener+5 more|arXiv (Cornell University)|Jan 28, 2017

Smart Agriculture and AI20 references16 citations

TL;DR

Treelogy proposes a novel tree classifier that fuses deep convolutional neural network (CNN) features with hand-crafted image features from leaf images, achieving 90.5% top-1 accuracy on a 57-species dataset. The system uses CNN-based segmentation and stem removal, combines fc6-layer CNN features with hand-crafted descriptors, and trains a linear SVM, outperforming pure CNN or hand-crafted methods alone.

ABSTRACT

We propose a novel tree classification system called Treelogy, that fuses deep representations with hand-crafted features obtained from leaf images to perform leaf-based plant classification. Key to this system are segmentation of the leaf from an untextured background, using convolutional neural networks (CNNs) for learning deep representations, extracting hand-crafted features with a number of image processing techniques, training a linear SVM with feature vectors, merging SVM and CNN results, and identifying the species from a dataset of 57 trees. Our classification results show that fusion of deep representations with hand-crafted features leads to the highest accuracy. The proposed algorithm is embedded in a smart-phone application, which is publicly available. Furthermore, our novel dataset comprised of 5408 leaf images is also made public for use of other researchers.

Motivation & Objective

• To develop a high-accuracy tree classification system using both deep learning and traditional image processing features.
• To address limitations in existing methods that rely solely on hand-crafted features or pure CNNs, which may miss subtle species distinctions.
• To create and release a large, preprocessed leaf image dataset of 5,408 images across 57 tree species to support future research.
• To implement the system in a real-world mobile application for public use by ecologists, educators, and nature enthusiasts.
• To demonstrate that feature fusion improves classification performance over using either deep or hand-crafted features in isolation.

Proposed method

• Apply convolutional neural networks (CNNs) to segment leaves from untextured backgrounds and remove stems using end-to-end learning.
• Extract hand-crafted features using image processing techniques such as curvature descriptors, shape descriptors, and multi-scale distance matrices.
• Use a pre-trained CNN (fine-tuned on the dataset) to extract deep representations from the fc6 and fc7 layers.
• Concatenate the deep CNN features (fc6 or fc7) with hand-crafted feature vectors into a single 4152-dimensional homogeneous feature vector.
• Train a linear SVM (LSVM) on the fused feature vectors for final classification.
• Employ a weighted fusion strategy implicitly through LSVM, without assigning explicit weights to CNN or hand-crafted features.

Experimental results

Research questions

• RQ1 Can combining deep CNN features with hand-crafted features improve tree species classification accuracy beyond using either modality alone?
• RQ2 How does the performance of a hybrid model compare to pure CNN or hand-crafted feature models on a large, diverse leaf dataset?
• RQ3 To what extent does the inclusion of preprocessed, high-quality leaf images from diverse sources enhance model generalization and reproducibility?
• RQ4 Can a mobile application integrating such a hybrid model provide practical, real-time tree identification in the field?
• RQ5 Does feature fusion lead to better performance on challenging datasets with high intra-class variation and low inter-class distinction?

Key findings

• The fusion of hand-crafted features with fc6-layer CNN features achieved the highest top-1 accuracy of 90.5% on the 57-species dataset.
• The CNN-only model achieved 84.62% top-1 accuracy, while the hand-crafted features alone reached 89.62% accuracy with LSVM.
• The combination of fc6 and hand-crafted features outperformed both individual components, demonstrating the benefit of multimodal feature fusion.
• On the Flavia dataset (32 species, 1,907 images), the fused model achieved 99.68% accuracy, surpassing state-of-the-art methods including 98.44% for pure CNN.
• The system’s top-5 accuracy with CNN alone was 97.58%, indicating strong model confidence, but top-1 accuracy was maximized through fusion.
• The publicly released dataset of 5,408 preprocessed leaf images is expected to accelerate future research in leaf-based plant classification.