[Paper Review] Efficient Brood Cell Detection in Layer Trap Nests for Bees and Wasps: Balancing Labeling Effort and Species Coverage

Chenchang Liu, Felix Fornoff+4 more|OpenAgrar|Mar 17, 2026

Insect and Arachnid Ecology and Behavior0 citations

TL;DR

The paper presents a deep learning approach (YOLOv8) for detecting and classifying brood cells in layer trap nests (LTNs) and introduces Constrained False Positive Loss (CFPL) to handle unlabeled data, reducing labeling effort while improving detection across both common and rare species.

ABSTRACT

Monitoring cavity-nesting wild bees and wasps is vital for biodiversity research and conservation. Layer trap nests (LTNs) are emerging as a valuable tool to study the abundance and species richness of these insects, offering insights into their nesting activities and ecological needs. However, manually evaluating LTNs to detect and classify brood cells is labor-intensive and time-consuming. To address this, we propose a deep learning based approach for efficient brood cell detection and classification in LTNs. LTNs present additional challenges due to densely packed brood cells, leading to a high labeling effort per image. Moreover, we observe a significant imbalance in class distribution, with common species having notably more occurrences than rare species. Comprehensive labeling of common species is time-consuming and exacerbates data imbalance, while partial labeling introduces data incompleteness which degrades model performance. To reduce labeling effort and mitigate the impact of unlabeled data, we introduce a novel Constrained False Positive Loss (CFPL) strategy. CFPL dynamically masks predictions from unlabeled data, preventing them from interfering with the classification loss during training. We evaluate our approach on a dataset of 712 LTN images collected over one season, covering 28 fine-grained classes describing the taxonomy and status of brood cells. To minimize labeling effort, we limit the training set to a maximum of 300 labels per class. Experimental results demonstrate that deep learning can be effectively used to detect brood cells in LTNs. Our CFPL method further improves performance and balances model accuracy and labeling effort while also mitigating class imbalance.

Motivation & Objective

• Motivate automated monitoring of cavity-nesting bees and wasps using LTNs to assess abundance and species richness.
• Address labeling challenges in imbalanced ecological data, where common species are overrepresented and labeling all instances is impractical.
• Develop and evaluate a learning strategy (CFPL) to mitigate unlabeled data impact during training.
• Demonstrate feasibility and quantify performance gains on a real LTN image dataset with limited labels.

Proposed method

• Utilize YOLOv8 as the detection backbone for brood cells in LTNs.
• Introduce Constrained False Positive Loss (CFPL) to mask predictions from unlabeled areas during classification loss computation.
• Label up to 300 samples per class to balance labeling effort and data completeness.
• Partition data into training (with partial labeling), testing, and validation sets; use 712 LTN images with 28 classes.
• Compare baseline YOLOv8 with CFPL-enhanced YOLOv8 across majority and minority class groups.

Experimental results

Research questions

• RQ1 Can deep learning detect brood cells in LTNs with limited labeled data?
• RQ2 Does CFPL improve detection performance in the presence of unlabeled majority-class samples?
• RQ3 How does the CFPL approach affect precision (AP) and recall for majority versus minority classes?
• RQ4 What are the practical labeling-effort savings when using partial labeling strategies in ecological monitoring?

Key findings

• YOLOv8 baseline achieves AP of 63.11% and recall of 7.05% for majority classes; CFPL improves to 66.11% AP and 46.54% recall for majority classes.
• For minority classes, AP increases from 44.29% to 45.83% and recall from 31.46% to 35.57% with CFPL.
• CFPL reduces misclassification of unlabeled majority-class samples as background, lowering background misclassification from 0.56 to 0.33 on average.
• Overall, CFPL yields a substantial recall boost for majority classes (≈39.49 percentage points) and modest AP gains for both groups, demonstrating a better labeling-effort-to-performance trade-off.
• The dataset and CFPL enable effective detection with fewer than 300 labeled samples per class, highlighting practical applicability for ecological monitoring.