[Paper Review] Benchmarking tree species classification from proximally-sensed laser scanning data: introducing the FOR-species20K dataset

Rehush, Nataliia|arXiv (Cornell University)|Aug 12, 2024

Remote Sensing and LiDAR Applications5 citations

TL;DR

Introduces the FOR-species20K benchmark dataset with over 20k tree point clouds across 33 species, enabling benchmarking of DL models for species classification from TLS, MLS, and ULS data.

ABSTRACT

Proximally-sensed laser scanning offers significant potential for automated forest data capture, but challenges remain in automatically identifying tree species without additional ground data. Deep learning (DL) shows promise for automation, yet progress is slowed by the lack of large, diverse, openly available labeled datasets of single tree point clouds. This has impacted the robustness of DL models and the ability to establish best practices for species classification. To overcome these challenges, the FOR-species20K benchmark dataset was created, comprising over 20,000 tree point clouds from 33 species, captured using terrestrial (TLS), mobile (MLS), and drone laser scanning (ULS) across various European forests, with some data from other regions. This dataset enables the benchmarking of DL models for tree species classification, including both point cloud-based (PointNet++, MinkNet, MLP-Mixer, DGCNNs) and multi-view image-based methods (SimpleView, DetailView, YOLOv5). 2D image-based models generally performed better (average OA = 0.77) than 3D point cloud-based models (average OA = 0.72), with consistent results across different scanning platforms and sensors. The top model, DetailView, was particularly robust, handling data imbalances well and generalizing effectively across tree sizes. The FOR-species20K dataset, available at https://zenodo.org/records/13255198, is a key resource for developing and benchmarking DL models for tree species classification using laser scanning data, providing a foundation for future advancements in the field.

Motivation & Objective

• Motivate automated tree species classification from proximally-sensed laser scanning data without extensive ground-truth data.
• Provide a large, diverse, openly available labeled dataset of single-tree point clouds for robust DL benchmarking.
• Enable cross-platform and cross-sensor evaluation of DL models for species classification.
• Assess how 2D image-based and 3D point-cloud-based DL approaches perform on real-world TLS, MLS, and ULS data.

Proposed method

• Assemble and annotate over 20,000 tree point clouds from 33 species collected with terrestrial TLS, mobile MLS, and drone ULS across European forests.
• Evaluate DL models across two families: point-cloud-based (PointNet++, MinkNet, MLP-Mixer, DGCNNs) and multi-view image-based (SimpleView, DetailView, YOLOv5).
• Compare performance between 2D image-based and 3D point-cloud-based approaches using overall accuracy (OA) as a metric.
• Analyze robustness to data imbalance and generalization across tree sizes especially for the top-performing DetailView model.

Experimental results

Research questions

• RQ1 Can DL models trained on FOR-species20K reliably classify tree species from proximally-sensed laser scanning data across multiple sensors and forests?
• RQ2 How do 2D image-based methods compare to 3D point-cloud-based methods in accuracy and robustness for tree species classification?
• RQ3 Which factors (sensor type, species imbalance, tree size) influence model performance the most?
• RQ4 Is the DetailView model particularly robust to data imbalance and generalizes well across varying tree sizes?
• RQ5 What is the overall potential of FOR-species20K to standardize benchmarking for tree species classification tasks?

Key findings

• 2D image-based models achieved higher average accuracy (OA 0.77) than 3D point-cloud-based models (OA 0.72).
• The DetailView model emerged as the top performer and showed robustness to data imbalances.
• Performance was consistent across probing platforms and sensors, indicating cross-platform generalizability.
• The FOR-species20K dataset enables benchmarking of both point-cloud-based and multi-view image-based DL approaches.
• The dataset provides a foundation for future advances in tree species classification from proximally-sensed laser scanning data.