[Paper Review] Absolute Geometry Calibration of Distributed Microphone Arrays in an Audio-Visual Sensor Network

Florian Jacob, Reinhold Haeb‐Umbach|arXiv (Cornell University)|Apr 13, 2015

Speech and Audio Processing18 references3 citations

TL;DR

This paper proposes two methods for absolute geometry calibration of distributed microphone arrays in audio-visual sensor networks using audio-visual correlates—specifically, joint direction-of-arrival (DoA) measurements from both modalities. The first method maps acoustic sensor positions to a visual coordinate system via trajectory alignment, while the second jointly solves for acoustic sensor positions using a system of nonlinear equations. The joint calibration method achieves a mean positioning error of 0.20 m even in highly reverberant environments.

ABSTRACT

Joint audio-visual speaker tracking requires that the locations of microphones and cameras are known and that they are given in a common coordinate system. Sensor self-localization algorithms, however, are usually separately developed for either the acoustic or the visual modality and return their positions in a modality specific coordinate system, often with an unknown rotation, scaling and translation between the two. In this paper we propose two techniques to determine the positions of acoustic sensors in a common coordinate system, based on audio-visual correlates, i.e., events that are localized by both, microphones and cameras separately. The first approach maps the output of an acoustic self-calibration algorithm by estimating rotation, scale and translation to the visual coordinate system, while the second solves a joint system of equations with acoustic and visual directions of arrival as input. The evaluation of the two strategies reveals that joint calibration outperforms the mapping approach and achieves an overall calibration error of 0.20m even in reverberant environments.

Motivation & Objective

• To resolve the scale ambiguity in acoustic sensor network calibration when visual sensor positions are known but coordinate systems are misaligned.
• To enable absolute geometry calibration of distributed microphone arrays without requiring clock synchronization or artificial calibration signals.
• To improve calibration accuracy in reverberant environments by leveraging audio-visual correlates such as speaker DoA estimates from both modalities.
• To compare two calibration strategies: coordinate mapping of acoustic trajectories and joint estimation using acoustic and visual DoA measurements.
• To evaluate performance under varying reverberation times and demonstrate robustness using real-world DoA estimation models.

Proposed method

• Uses direction-of-arrival (DoA) estimates from both acoustic and visual sensors as input, avoiding the need for time-of-flight or time-difference-of-arrival synchronization.
• Employs a coordinate mapping approach that estimates rotation, translation, and scale between acoustic and visual coordinate systems by aligning speaker trajectories derived from DoA data.
• Develops a joint calibration method that formulates a system of nonlinear equations combining acoustic and visual DoA measurements to jointly estimate acoustic sensor positions.
• Applies the RANSAC algorithm to reject outliers in both acoustic and visual DoA estimates, improving robustness to poor measurements.
• Simulates visual DoA errors using Hidden Markov Models (HMMs) with states for detection, missed detection, and false detection, based on HOG and SVM analysis of the AV16.3 corpus.
• Uses a filter-and-sum beamformer with adaptive filtering to estimate acoustic DoA over time, enabling real-time tracking of moving speakers.

Experimental results

Research questions

• RQ1 Can audio-visual correlates be effectively used to resolve the scale ambiguity in acoustic sensor network calibration?
• RQ2 How does joint calibration using both acoustic and visual DoA measurements compare to coordinate mapping in terms of accuracy and robustness?
• RQ3 What is the impact of reverberation on the performance of DoA-based calibration methods in audio-visual sensor networks?
• RQ4 To what extent does RANSAC-based outlier rejection improve calibration accuracy in the presence of noisy DoA estimates?
• RQ5 Can a small number of well-distributed spatial events achieve calibration performance comparable to full trajectories?

Key findings

• The joint calibration method outperforms the coordinate mapping approach across all reverberation times, achieving a mean positioning error (MPE) of 0.20 m in highly reverberant environments (500 ms RT60).
• The coordinate mapping approach suffers from significant scale estimation errors, which dominate its performance, especially at low reverberation times.
• An oracle experiment where the scale factor is known shows that the coordinate mapping approach performs nearly as well as the joint method in low reverberation, confirming that scale error is the primary limitation.
• Sensor orientation errors are below 2° for both methods across all reverberation times, indicating high angular accuracy.
• Using only 15 strategically selected events with good spatial distribution, the joint calibration achieves similar performance to full-trajectory calibration, demonstrating the importance of event geometry over quantity.
• The joint calibration method maintains a consistent MPE of 0.14 m at 300 ms RT60 and 0.23 m at 500 ms RT60, showing robustness to reverberation.