[Paper Review] WildlifeDatasets: An open-source toolkit for animal re-identification

Vojtěch Čermák, Lukáš Picek+2 more|arXiv (Cornell University)|Nov 15, 2023

Identification and Quantification in Food4 citations

TL;DR

This paper introduces WildlifeDatasets, an open-source Python toolkit for wildlife re-identification that standardizes access to 29 public datasets, enables benchmarking of local and deep learning methods, and introduces MegaDescriptor—a new foundation model based on Swin architectures trained on a curated dataset. MegaDescriptor achieves state-of-the-art performance across species, outperforming CLIP and DINOv2 significantly, with the largest variant (L-384) achieving 99.93% accuracy on AAUZebraFish and 78.41% on HyenaID2022.

ABSTRACT

In this paper, we present WildlifeDatasets (https://github.com/WildlifeDatasets/wildlife-datasets) - an open-source toolkit intended primarily for ecologists and computer-vision / machine-learning researchers. The WildlifeDatasets is written in Python, allows straightforward access to publicly available wildlife datasets, and provides a wide variety of methods for dataset pre-processing, performance analysis, and model fine-tuning. We showcase the toolkit in various scenarios and baseline experiments, including, to the best of our knowledge, the most comprehensive experimental comparison of datasets and methods for wildlife re-identification, including both local descriptors and deep learning approaches. Furthermore, we provide the first-ever foundation model for individual re-identification within a wide range of species - MegaDescriptor - that provides state-of-the-art performance on animal re-identification datasets and outperforms other pre-trained models such as CLIP and DINOv2 by a significant margin. To make the model available to the general public and to allow easy integration with any existing wildlife monitoring applications, we provide multiple MegaDescriptor flavors (i.e., Small, Medium, and Large) through the HuggingFace hub (https://huggingface.co/BVRA).

Motivation & Objective

• Address the lack of standardization in wildlife re-identification methods, datasets, and evaluation metrics across ecological and computer vision research.
• Provide a unified, reproducible framework for comparing local descriptors, deep learning models, and transfer learning approaches in animal re-identification.
• Enable researchers to benchmark models across diverse species and datasets with consistent pre-processing, evaluation, and fine-tuning pipelines.
• Develop and release a foundation model, MegaDescriptor, trained on a curated dataset to serve as a strong baseline for cross-species re-identification.
• Facilitate integration into real-world wildlife monitoring by providing pre-trained MegaDescriptor variants via the HuggingFace hub.

Proposed method

• Implement a modular Python toolkit that provides standardized access to 29 publicly available wildlife re-identification datasets, including metadata, splits, and preprocessing utilities.
• Support multiple re-identification paradigms: local feature descriptors (e.g., SIFT, ORB), deep metric learning, and fine-tuning of vision transformers.
• Introduce MegaDescriptor, a foundation model based on Swin transformer architectures (Swin-T, Swin-S, Swin-B, Swin-L), pre-trained on a large-scale, curated dataset of individual animals across 29 species.
• Train MegaDescriptor using contrastive learning with instance discrimination, optimizing for identity-aware representation learning across diverse animal patterns and markings.
• Provide multiple model variants (Small, Medium, Large) with different input resolutions (224px and 384px) for flexibility in accuracy vs. inference speed trade-offs.
• Host all pre-trained MegaDescriptor models on HuggingFace Hubs for easy deployment and integration into existing wildlife monitoring pipelines.

Experimental results

Research questions

• RQ1 How do local descriptors and deep learning models compare in performance across a broad range of wildlife re-identification datasets and species?
• RQ2 What is the impact of model size and input resolution on re-identification accuracy across diverse animal species?
• RQ3 Can a single foundation model generalize effectively across multiple species with varying visual patterns and data quality?
• RQ4 How does MegaDescriptor compare to established foundation models like CLIP and DINOv2 in zero-shot and fine-tuned settings?
• RQ5 To what extent do data leakage and poor curation in existing datasets inflate reported performance, and how can this be mitigated?

Key findings

• MegaDescriptor-L-384 achieves state-of-the-art performance, attaining 99.93% accuracy on AAUZebraFish, 98.17% on StripeSpotter, and 78.41% on HyenaID2022.
• The largest MegaDescriptor variant (L-384) outperforms smaller variants by a significant margin, with gains of up to 2.53% on FriesianCattle2017 and 0.48% on LeopardID2022.
• MegaDescriptor surpasses CLIP and DINOv2 by a substantial margin, particularly on challenging datasets like HumpbackWhaleID (77.81% vs. 51.83% for DINOv2) and HappyWhale (34.30% vs. 20.07% for CLIP).
• The ablation study confirms that model size and input resolution strongly correlate with performance, with MegaDescriptor-L-384 consistently outperforming smaller variants across all 29 datasets.
• The toolkit enables consistent evaluation across datasets, revealing that many existing benchmarks suffer from data leakage and inflated performance, which the toolkit helps mitigate through standardized splits.
• The largest MegaDescriptor variant (L-384) achieves 100% accuracy on AerialCattle2017, OpenCows2020, and SMALST, demonstrating strong generalization on high-quality, pattern-rich datasets.