[Paper Review] DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices

Run Wang, Felix Juefei-Xu+5 more|arXiv (Cornell University)|May 28, 2020

Music and Audio ProcessingComputer Science49 references17 citations

TL;DR

DeepSonar proposes a novel, neuron behavior-based approach to detect AI-synthesized fake voices by analyzing layer-wise activation patterns in speaker recognition models, achieving 98.1% average accuracy and low false alarm rates (≈2%) across multiple datasets, while demonstrating strong robustness against voice conversion and real-world noise attacks.

ABSTRACT

With the recent advances in voice synthesis, AI-synthesized fake voices are indistinguishable to human ears and widely are applied to produce realistic and natural DeepFakes, exhibiting real threats to our society. However, effective and robust detectors for synthesized fake voices are still in their infancy and are not ready to fully tackle this emerging threat. In this paper, we devise a novel approach, named \emph{DeepSonar}, based on monitoring neuron behaviors of speaker recognition (SR) system, \ie, a deep neural network (DNN), to discern AI-synthesized fake voices. Layer-wise neuron behaviors provide an important insight to meticulously catch the differences among inputs, which are widely employed for building safety, robust, and interpretable DNNs. In this work, we leverage the power of layer-wise neuron activation patterns with a conjecture that they can capture the subtle differences between real and AI-synthesized fake voices, in providing a cleaner signal to classifiers than raw inputs. Experiments are conducted on three datasets (including commercial products from Google, Baidu, \etc) containing both English and Chinese languages to corroborate the high detection rates (98.1\% average accuracy) and low false alarm rates (about 2\% error rate) of DeepSonar in discerning fake voices. Furthermore, extensive experimental results also demonstrate its robustness against manipulation attacks (\eg, voice conversion and additive real-world noises). Our work further poses a new insight into adopting neuron behaviors for effective and robust AI aided multimedia fakes forensics as an inside-out approach instead of being motivated and swayed by various artifacts introduced in synthesizing fakes.

Motivation & Objective

• Address the growing threat of AI-synthesized fake voices that are indistinguishable from real human voices and pose risks in fraud and misinformation.
• Overcome the limitations of existing detection methods that rely on detectable artifacts in synthesized voices, which are often absent or subtle in modern systems.
• Develop a detection framework that is both effective and robust in real-world conditions, including noisy environments and post-processing manipulations.
• Explore an inside-out detection approach based on internal neuron behaviors rather than external artifacts, offering improved generalization and interpretability.

Proposed method

• Monitor layer-wise neuron activation patterns in a pre-trained speaker recognition (SR) model to extract discriminative representations for real vs. fake voice detection.
• Use the internal representations of the DNN as a signal source for classification, avoiding reliance on synthetic artifacts introduced during voice synthesis.
• Train a classifier on the aggregated neuron activation patterns across multiple layers to distinguish between real human voices and AI-generated fakes.
• Apply data augmentation and normalization techniques to improve generalization across diverse voice types and languages (English and Chinese).
• Evaluate robustness by applying voice conversion and additive real-world noises (e.g., wind, rain, laughter) to test inputs, simulating real-world deployment conditions.
• Compare performance against a baseline using bispectral artifact analysis, demonstrating superior robustness under manipulation attacks.

Experimental results

Research questions

• RQ1 Can layer-wise neuron activation patterns in a speaker recognition model effectively distinguish between real human voices and AI-synthesized fake voices?
• RQ2 How does the proposed neuron-behavior-based detection method perform under real-world noise conditions and voice conversion attacks?
• RQ3 Does monitoring internal DNN behaviors yield better detection performance and robustness than relying on external artifacts such as bispectral features?
• RQ4 How does the method generalize across different languages and commercial TTS/VC systems (e.g., Google, Baidu)?

Key findings

• DeepSonar achieves an average detection accuracy of 98.1% across three diverse datasets, including commercial TTS and voice cloning systems.
• The false alarm rate remains low at approximately 2%, indicating high reliability in distinguishing real from fake voices.
• Under voice conversion attacks, DeepSonar maintains strong performance, with detection accuracy decreasing by less than 7% on average across five noise magnitudes.
• For additive real-world noises, performance degrades by less than 18% for indoor noises (e.g., fireworks, trains) and up to 25% for wind and rain, but remains effective at SNR > 35 dB.
• DeepSonar significantly outperforms the bispectral artifact-based baseline across all manipulation types, with consistently higher AUC scores.
• The method shows resilience to complex noise mixtures, particularly when denoising is applied, suggesting potential for integration with voice enhancement techniques in future work.