[Paper Review] Adaptive Evidence Weighting for Audio-Spatiotemporal Fusion

Oscar Ovanger, Levi Harris+1 more|arXiv (Cornell University)|Feb 3, 2026

Animal Vocal Communication and Behavior0 citations

TL;DR

FINCH adaptively weighs contextual spatiotemporal evidence per sample when fusing a frozen audio classifier with a spatiotemporal prior, improving accuracy and robustness over audio-only and fixed-weight fusion. It preserves an audio-only fallback and bounds contextual influence.

ABSTRACT

Many machine learning systems have access to multiple sources of evidence for the same prediction target, yet these sources often differ in reliability and informativeness across inputs. In bioacoustic classification, species identity may be inferred both from the acoustic signal and from spatiotemporal context such as location and season; while Bayesian inference motivates multiplicative evidence combination, in practice we typically only have access to discriminative predictors rather than calibrated generative models. We introduce extbf{F}usion under extbf{IN}dependent extbf{C}onditional extbf{H}ypotheses ( extbf{FINCH}), an adaptive log-linear evidence fusion framework that integrates a pre-trained audio classifier with a structured spatiotemporal predictor. FINCH learns a per-sample gating function that estimates the reliability of contextual information from uncertainty and informativeness statistics. The resulting fusion family \emph{contains} the audio-only classifier as a special case and explicitly bounds the influence of contextual evidence, yielding a risk-contained hypothesis class with an interpretable audio-only fallback. Across benchmarks, FINCH consistently outperforms fixed-weight fusion and audio-only baselines, improving robustness and error trade-offs even when contextual information is weak in isolation. We achieve state-of-the-art performance on CBI and competitive or improved performance on several subsets of BirdSet using a lightweight, interpretable, evidence-based approach. Code is available: exttt{\href{https://anonymous.4open.science/r/birdnoise-85CD/README.md}{anonymous-repository}}

Motivation & Objective

• Motivate robust fusion of heterogeneous, approximately independent evidences for the same target.
• Develop a per-sample gating mechanism to adaptively weight contextual (spatiotemporal) evidence without retraining base predictors.
• Provide a theoretically and empirically safe fusion framework with an explicit audio-only fallback.
• Demonstrate state-of-the-art or competitive performance on large-scale bioacoustic benchmarks using a lightweight, interpretable approach.

Proposed method

• Adopt log-linear (product-of-experts) fusion: log p(y|x,s) = log p_theta(y|x) + omega(x,s) * log p_psi(y|s).
• Learn a nonnegative, per-sample fusion weight omega(x,s) via a two-layer MLP gating network.
• Compute omega(x,s) from uncertainty and informativeness features of audio and context, plus metadata; constrain omega to [epsilon, omega_max].
• Freeze the audio encoder; train only the fusion/gating components and the contextual prior (AdaSTEM prior on CBI, metadata MLP on BirdSet).
• Include a variance-based regularizer to avoid gate collapse and ensure genuine adaptivity.
• Maintain an audio-only fallback (omega=0) and bound contextual influence to ensure robustness.

Experimental results

Research questions

• RQ1 Can per-sample adaptive weighting of contextual evidence improve bioacoustic classification beyond fixed-weight fusion and audio-only baselines?
• RQ2 Does the FINCH framework maintain an audio-only fallback while leveraging contextual priors when informative?
• RQ3 How does adaptive gating perform across heterogeneous and weak contextual signals on large-scale bird audio benchmarks?
• RQ4 What is the impact of using different spatiotemporal priors (AdaSTEM on CBI vs. learned metadata prior on BirdSet) within FINCH?

Key findings

• FINCH achieves higher test accuracy on CBI (0.826) compared to audio-only (0.806) under a linear-probe protocol.
• On BirdSet subsets, FINCH matches or improves audio-only baselines across retrieval AUROC, detection cmAP, and Top-1 accuracy.
• Fixed global fusion weights offer limited gains, highlighting the value of per-sample adaptivity.
• The contextual prior alone performs poorly in isolation, confirming that gains come from selective integration rather than context alone.
• FINCH provides state-of-the-art performance on CBI and competitive results on BirdSet using lightweight, interpretable fusion.