[Paper Review] Counting Without Numbers \& Finding Without Words

Badri Narayana Patro|arXiv (Cornell University)|Mar 25, 2026

Animal Vocal Communication and Behavior0 citations

TL;DR

The paper introduces a multi-modal reunification framework that integrates visual, acoustic, and contextual cues to locate missing animals, demonstrating acoustic identity improves re-identification when visual data is ambiguous.

ABSTRACT

Every year, 10 million pets enter shelters, separated from their families. Despite desperate searches by both guardians and lost animals, 70% never reunite, not because matches do not exist, but because current systems look only at appearance, while animals recognize each other through sound. We ask, why does computer vision treat vocalizing species as silent visual objects? Drawing on five decades of cognitive science showing that animals perceive quantity approximately and communicate identity acoustically, we present the first multimodal reunification system integrating visual and acoustic biometrics. Our species-adaptive architecture processes vocalizations from 10Hz elephant rumbles to 4kHz puppy whines, paired with probabilistic visual matching that tolerates stress-induced appearance changes. This work demonstrates that AI grounded in biological communication principles can serve vulnerable populations that lack human language.

Motivation & Objective

• Motivate why animals and vulnerable populations rely on acoustic and multi-modal signals rather than symbolic human language.
• Formulate cross-modal re-identification to locate missing individuals using visual, acoustic, and contextual data.
• Propose a species-adaptive, multi-modal architecture that fuses visual, acoustic, and contextual features.
• Demonstrate that acoustic identity and soft matching improve identification when visual cues are degraded.
• Discuss practical deployment, limitations, and broader implications for AI grounded in biological communication principles.

Proposed method

• Propose a cross-modal re-identification framework that learns joint embeddings across visual, acoustic, and contextual features.
• Develop species-adaptive acoustic encoding to cover a wide frequency range from infrasound to ultrasound.
• Implement soft visual matching using approximate similarity with Gaussian embeddings to tolerate appearance changes.
• Model temporal degradation to capture how signal reliability decays over separation time.
• Provide controlled synthetic experiments with 60 identities to analyze component contributions and enable reproducibility.
• Pilot deployment in real shelters to assess practical feasibility.

Experimental results

Research questions

• RQ1 Can multi-modal fusion of visual, acoustic, and contextual cues improve missing-animal re-identification over visual-only systems?
• RQ2 How do species-specific acoustic encodings and soft perceptual matching affect Rank-1 accuracy and false negatives under appearance variability?
• RQ3 What is the impact of temporal dynamics on signal reliability in cross-modal re-identification?
• RQ4 Is there practical feasibility for deploying a multi-modal system in real shelters for ambiguous cases?

Key findings

• Acoustic features improve Rank-1 accuracy by 25.7% when visual appearance is ambiguous.
• Multi-modal fusion achieves a 30% relative reduction in false negatives through soft perceptual matching.
• Pilot deployment across two shelters achieved 61% success in 23 ambiguous cases where photo-only methods failed.