Contents
Maybe, and a detector will give you a probability with a known error rate rather than a verdict. The number matters, but so does the fact that in real-world conditions that error rate is far higher than the laboratory figure a vendor quotes, and a single score can point at the wrong thing entirely.
What does the confidence number actually mean?
A voice detector outputs “how AI-like this sounds” at a chosen threshold, usually reported in research as equal-error rate, the point where false accepts and false rejects are equal. The trap is that the same tool’s confidence means very different things depending on where the audio came from. AASIST scores 0.83% EER on the ASVspoof 2019 benchmark (Jung, Heo, Tak et al., ICASSP 2022), and RawNet2 reaches 4.66% (Tak, Patino, Todisco et al., ICASSP 2021), but both climb into the 35-percent range on in-the-wild recordings (Yi, Wang, Tao et al., 2023). A “94 percent AI” reading from a lab-tuned model and the same reading on a random phone clip are not the same claim, even though the interface shows one number.
Why is lab accuracy so much better than real life?
Because the benchmark is a fixed, hard set, not the open world. The people who built ASVspoof were explicit that their synthetic speech was convincing: it included spoofed audio that “cannot be differentiated from bona fide utterances even by human subjects” (Wang, Yamagishi, Todisco et al., Computer Speech & Language 2020). That difficulty is exactly why lab numbers do not transfer. Move off the training distribution and detection collapses: a neural-vocoder detector that scores 0.13% EER on the vocoder it learned degrades to between 4.6 and 45.35 percent on unseen vocoders (Sun et al., CVPRW 2023). The confidence bar does not know which side of that gap your clip sits on.
Can the flag come from something other than the deepfake?
Yes, and this is the failure that quietly inflates lab scores. Detectors often learn a dataset shortcut instead of the voice. Negroni, Cuccovillo, Bestagini and colleagues (ICASSP 2026) note that many models “base their decisions on unvoiced timesteps, frequency bands where no speech occurs, or even background noise, rather than the actual speech.” A high confidence score can be reading the recording conditions, the microphone, or the silence at the start of a clip, not the speaker at all.
Do commercial detectors behave like the research ones?
No, and this is why you cannot port research numbers onto a consumer tool. In our internal testing, one widely used commercial voice detector’s score barely moved even on audio we had altered in ways that clearly change how it sounds, which points to a narrow underlying feature set. The method behind that test stays in-house, but the finding is ours to report: a published equal-error rate from a research benchmark is a poor predictor of how a commercial tool will score any given clip. Model-specific tools say as much themselves. The ElevenLabs classifier documentation limits it to the first one minute of audio and states it “does not reliably classify ElevenV3,” and it targets ElevenLabs-made speech rather than clones from other systems.
So how should you read a single “AI” verdict?
Read it as a probability at an operating point you are usually not shown, and interrogate it. Which model produced the score, what threshold was applied, and what is the false-positive rate at that threshold? One tool on one clip is weak evidence in either direction. A stronger read combines independent methods, provenance where it exists, and the surrounding context, and it treats disagreement between tools as information rather than a glitch, the same way two image detectors split on one picture (see why do AI detectors give different results).
The bottom line is that confidence is not certainty. The lab produces the reassuring numbers, the wild produces the real ones, and the gap between them is not noise but the actual reliability of the tool. A score without its operating point cannot be interpreted, and a score that might be reading background silence should not, on its own, decide anything. For the first-pass method this points to, see how to tell if audio is AI-generated.
Sources
- Jung, Heo, Tak et al. (2022). AASIST: Audio Anti-Spoofing Using Integrated Spectro-Temporal Graph Attention Networks. ICASSP 2022.
- Tak, Patino, Todisco et al. (2021). End-to-End Anti-Spoofing with RawNet2. ICASSP 2021.
- Wang, Yamagishi, Todisco et al. (2020). ASVspoof 2019: A Large-Scale Public Database of Synthesized, Converted and Replayed Speech. Computer Speech & Language 2020.
- Sun et al. (2023). AI-Synthesized Voice Detection Using Neural Vocoder Artifacts. CVPRW 2023.
- Negroni, Cuccovillo, Bestagini et al. (2026). Multi-Task Transformer for Explainable Speech Deepfake Detection via Formant Modeling. ICASSP 2026.
- Yi, Wang, Tao et al. (2023). Audio Deepfake Detection: A Survey.