Audio deepfake detection · Real communication channels

RealCommBenchmarking and Adapting Audio Deepfake Detection
over Real Communication Channels

Li Wang^1,*, Jindong Wang^1,*, Wan Lin^1,*, Kunyu Feng^1,*, Yufan Lin¹, Lei Wang², Rizhao Cai²,
Ce Fang², Jinzhe Xue², Jie Shi², Haizhou Li¹, Zhizheng Wu¹

¹The Chinese University of Hong Kong, Shenzhen ²Huawei Technologies Co., Ltd.

* Equal contribution

Manuscript draft GitHub Audio examples Leaderboard · SpoofRadar 中文研究文档

Abstract

Acoustic capture, device processing, and transmission coding in real calls jointly alter speech signals and may affect the reliability of audio deepfake detection. To systematically evaluate these effects, we introduce RealComm, a dataset pairing digital speech with recordings of the same utterances transmitted through real mobile calls. The acquisition design covers acoustic and wired injection, handset and speakerphone operation, and caller-side denoising, with bona fide and synthetic speech recorded under matched conditions. The dataset includes RealCommBench for evaluation and RealCommTrain for adaptation. We further develop RealComm-Aug, a staged waveform augmentation pipeline for simulating acquisition and transmission effects, and investigate its combination with real-call training. Experiments with existing detectors reveal substantial degradation after call transmission, even for models with strong digital-domain performance; acoustic injection is particularly challenging. Paired quality and spectral analyses show that quality degradation is associated with poorer detection across recording conditions for the evaluated detectors, while high-frequency attenuation alone does not explain detection difficulty. Simulated augmentation and real-call training provide complementary benefits, with their combination achieving the lowest call-domain error for each of three evaluated detectors. For Teffic-Audio, the equal error rate decreases from 37.94% to 22.37%.

Dataset and acquisition

Every source utterance has a digital reference and multiple real-call versions. Bona fide and synthetic speech traverse the same recording conditions; all versions of a source remain in one split.

Figure 1. RealComm acquisition and device partitions. OtA = over-the-air; LtM = line-to-microphone. Handset and speakerphone refer to the caller’s operating mode. Vivo is reserved for testing.

Source partitions and real-call recordings · Table II in the manuscript
Split	Digital sources	Recorded sources	Conditions / source	Call recordings
Train	1,792	224	44	9,856
Development	448	56	44	2,464
Test	560	560	42	23,520

The source pool is balanced by authenticity and language (English / Mandarin), with synthetic speech from seven generators. Train and test share the generator set. Repeated call versions expand condition coverage rather than the number of independent source utterances.

Paired audio examples

Explore 16 source examples in English and Mandarin: bona fide speech and all seven TTS systems, spanning eight directed device routes. Within each example, the digital reference and three call configurations share the same source utterance; the call route is fixed.

Target text

Choose a language and voice source above. For bona fide and CosyVoice2 examples, switch denoising off/on to compare matched recordings. Other examples show their recorded denoising state below each player; “no user-controlled switch” is distinct from off. OtA uses acoustic injection; LtM uses a wired microphone input. WAV files are unchanged and are not loudness-normalized; selection does not use detector scores.

Zero-shot detection

Real-call transmission substantially degrades the evaluated detectors. EER rises by more than 30 percentage points for every detector below, despite digital-domain EERs of 0–15.71%. Acoustic injection is consistently harder than wired injection, and degradation occurs in both languages and across all seven generation systems.

Complete RealComm evaluation results will be published and maintained on SpoofRadar, our unified leaderboard for speech deepfake detection. This page presents the manuscript’s fixed results; the platform will provide full evaluation tables and subsequent updates, identifying model/checkpoint versions and evaluation settings.

Zero-shot equal error rate (%) ↓ · Figure 2(a) in the manuscript
Detector	Digital	Call	ΔEER (pp)
WavLM-L	4.64	40.00	+35.36
Whisper	15.71	47.88	+32.17
AASIST	7.50	40.19	+32.69
Teffic-Audio	0.00	37.95	+37.95
DF Arena	7.86	38.77	+30.91

Six panels compare digital and call EER, input configurations, denoising, languages, and seven generation systems for WavLM-L, Whisper, AASIST, Teffic-Audio and DF Arena. — **Figure 2.** Zero-shot EER by domain (a), input configuration (b), denoising (c), language (d), and generation system in digital speech (e) and real calls (f). Off/on subsets are source-, route-, and configuration-matched. All panels share a 0–60% scale.

The main comparisons follow the current manuscript. The initial XLSR-SLS evaluation is retained in its protocol discussion (38.57% digital / 50.31% call EER); its weak digital baseline limits interpretation of the additional channel effect.

Speech quality and spectral changes

Poorer speech quality is associated with greater detection degradation across recording conditions. Real calls increase recognition errors and reduce speaker similarity and predicted overall quality. These quality changes track EER increases for the evaluated detectors, while high-frequency attenuation alone does not explain which conditions are harder.

Mean WER, CER, speaker similarity for bona fide and synthetic speech, and DNSMOS across domains and call conditions. — **Figure 3.** Paired quality analysis using Seed-TTS Eval evaluators and DNSMOS P.835 OVRL. Lower WER/CER and higher SIM/DNSMOS are better.

Normalized mean spectra and changes in successive 1-kHz bands. — **Figure 4.** Normalized spectral changes after real calls. Acoustic injection reduces the relative 1–2 kHz energy share; LtM remains close to the digital reference in this band. LtM retains the least 7–8 kHz energy yet has the best detection performance. These are relative energy shares, not absolute attenuation measurements.

Spearman correlations between quality or spectral deterioration and EER increases across 42 recording conditions. — **Figure 5.** Condition-level Spearman association. Greater quality deterioration is associated with larger EER increases for the evaluated detectors; high-frequency energy loss alone shows weak, mixed associations.

Correlations are descriptive, not causal. Quality metrics use full utterances; detection uses the model-specific input prefix protocol described in the manuscript.

RealComm-Aug

RealComm-Aug samples one signal-path scenario and its operations for each waveform, with the same distribution for bona fide and synthetic speech. Acoustic environment, device processing, file coding, and call transmission form distinct stages and exits. The receiver adds no further replay.

Staged augmentation flow with acoustic and direct input, device processing, a file-coding branch, call transmission and intermediate outputs. — **Figure 6.** Augmentation stages and output paths. Noise and reverberation can be used independently of the call branch. Packet-loss effects are approximated with decoded waveform-frame masks.

FExisting model; no RealComm updates.

AOriginal digital training mixture with RealComm-Aug.

TOriginal digital mixture plus RealCommTrain, retaining the original digital augmentation.

A+TThe same data as T, with RealComm-Aug on digital examples.

Every adapted model starts from its corresponding F checkpoint. Real-call examples bypass the added data-side augmentation; existing model-internal neural-codec augmentation retains its training configuration. No augmentation is applied at evaluation.

Adaptation results

Real-call training gives larger gains than simulation alone. Their combination produces the lowest pooled call EER for all three models, reaching 22.37% for Teffic-Audio. Digital-domain results reveal a tradeoff for AASIST.

Adaptation EER (%) ↓ · values plotted in Figure 7 of the manuscript
Detector	Domain	F	A	T	A+T
WavLM-L	Digital	4.64	4.29	3.21	3.21
WavLM-L	Call	40.00	38.71	35.48	34.16
AASIST	Digital	7.50	8.93	6.07	8.21
AASIST	Call	40.19	38.49	37.96	36.90
Teffic-Audio	Digital	0.00	0.36	0.36	0.00
Teffic-Audio	Call	37.94	34.92	23.84	22.37

Bold indicates the best setting within each model and domain. Teffic-Audio F uses a matched FP32 re-evaluation here (37.94%); the frozen zero-shot submission is 37.95%. Results are from single training runs. T and A+T use RealCommTrain, which is not included in the benchmark release; these adaptation results are distinct from zero-shot benchmark entries.

T minus A+T EER differences grouped by input configuration, denoising, device route, language and DNSMOS quality loss, with one panel per detector. — **Figure 8.** Incremental benefit of A+T over T. Positive teal bars indicate improvement; negative orange bars indicate regression. WavLM-L and Teffic-Audio improve across the displayed groups, while AASIST has condition-dependent tradeoffs.

Performance on public test sets

Call-domain gains do not guarantee improvement on every external test set. Augmentation alone improves ASVspoof 2021 LA results for all three adapted detectors. Combined adaptation improves real-call detection, but increases In-the-Wild EER by 0.38, 0.78, and 0.85 percentage points for WavLM-L, AASIST, and Teffic-Audio, respectively.

EER changes for A, T and A+T relative to F on ASVspoof 2021 LA, ASVspoof 5 and In-the-Wild, using a shared minus four to plus four percentage point scale. — **Figure 9.** Public-test EER changes relative to each detector’s frozen baseline. Negative values indicate improvement; all panels share the same scale. Baseline EERs are reported by the model contributors; adapted EERs are recomputed from prediction files.

Resources

GitHubProject repository · benchmark access and evaluation resources ManuscriptIEEE Transactions format · research draft Leaderboard · SpoofRadarComplete RealComm evaluation results · forthcoming 中文研究文档与论文对应的数据设计、实验结果与分析

The current study covers a fixed set of devices, two languages and seven generators. The results do not establish robustness to arbitrary hardware, communication services or unseen synthesis systems.

RealCommBenchmarking and Adapting Audio Deepfake Detection over Real Communication Channels