Contents
Real-time voice detection works as a fast first screen, not as a verdict. Running a classifier on a live stream forces it to decide from short rolling windows before a sentence is even finished, and the features that carry the most signal are exactly the ones that need the whole utterance. That trade lowers the ceiling below the offline benchmark figures, and real call conditions lower it again. For a live call it also has to answer two questions at once: is the voice synthetic, and is the speaker live rather than a recording played into the microphone?
Can a detector run on a live stream at all?
Yes. A real-time detector runs the same kind of classifier as an offline one, but under a latency budget, on a rolling window of audio rather than a finished file. Raw-waveform models such as AASIST and RawNet2 are fast enough to score short segments as they arrive, which is what makes a live screen possible. AASIST reaches about 0.83% equal error rate on ASVspoof 2019 (Jung, Heo, Tak et al., ICASSP 2022) and RawNet2 reaches 4.66% (Tak, Patino, Todisco et al., ICASSP 2021), but those are offline figures on a full benchmark utterance, not the numbers you get from a two-second window.
Why is the ceiling lower than offline?
Because the strongest features need the whole utterance, and streaming does not have it. Phoneme-level detection reaches 4.27% equal error rate, but it does so by comparing discrepancies across phonemes and pooling the entire utterance (Zhang, Hua, Lan, AAAI 2025). The segmental forensic method that reaches 4.4% equal error rate needs a full recording and reference audio of the real speaker (Yang, Sun, Lyu, Rose, 2025). Neither is available two seconds into a live call. A streaming system falls back on short-window raw-waveform classifiers, which is a weaker position than the offline benchmark implies, because the model has to commit early, before enough audio has accumulated to average out a noisy moment.
Why does a live call need two checks?
Because “is this synthetic” and “is this live” are different questions, and a live call is exposed to both. A convincing recording of a real person is not synthetic at all, yet playing it into a call is still an attack. The ASVspoof challenge formalized this split, running a physical-access scenario for replayed audio alongside the logical-access scenario for synthesis, and its authors describe automatic speaker verification as “vulnerable to spoofing, also referred to as ‘presentation attacks’” (Wang, Yamagishi, Todisco et al., Computer Speech & Language 2020). A live pipeline that only checked for synthesis would pass a genuine recording straight through. The mechanics of that second check are covered in how does live deepfake detection work.
Do the numbers survive a real call?
Less well than the lab. Constant-Q cepstral coefficients reach 1.85% equal error rate on the RedDots-Replayed set (Todisco, Delgado, Evans, 2017), but that is a curated offline benchmark. A replay detector that reads loudspeaker distortion reaches a 97.78 percent true-positive rate on known equipment and falls to a floor of roughly 79 to 90 percent on unseen playback hardware (Ren, Fang, Liu, 2019). The synthetic side shows the same shape: moved off their training distribution, detector equal error rates climb into the 35 to 55 percent range, a pattern the survey by Yi, Wang, Tao et al. (2023) names as the “poor generalization of existing detection methods to unknown fake attacks.” A streaming system is already deciding from less, so that gap bites harder.
What about commercial live-call detectors?
Treat them as indicative screens unless they publish operating points. A commercial interface may show a score, a color, or a warning label, but you still need to know the threshold, the false-positive rate, the false-negative rate, and which attacks it was trained to catch. A synthesis-only detector is not a replay detector, and a model-specific detector is not a general one. First-party tools are narrow by design and say so: the ElevenLabs classifier documentation limits it to the first one minute of audio and states it “does not reliably classify ElevenV3.” A live-call detector with no such disclosure is harder to interpret, not easier.
So does real-time voice detection work?
It works as what it is: a fast, useful screen that raises a flag mid-call so a human can slow down and verify. It does not work as a final ruling, and the latency budget means its confidence should be read as a running estimate under pressure rather than a settled verdict. For a suspected scam, the safe action is still procedural: hang up and call back a number you already trust, agree a code word in advance, and slow down any payment or credential request, as covered in voice-clone scam detector.
Sources
- Jung, Heo, Tak et al. (2022). AASIST: Audio Anti-Spoofing Using Integrated Spectro-Temporal Graph Attention Networks. ICASSP 2022.
- Tak, Patino, Todisco et al. (2021). End-to-End Anti-Spoofing with RawNet2. ICASSP 2021.
- Zhang, Hua, Lan (2025). Phoneme-Level Feature Discrepancies: A Key to Detecting Sophisticated Speech Deepfakes. AAAI 2025.
- Yang, Sun, Lyu, Rose (2025). Forensic Deepfake Audio Detection Using Segmental Speech Features.
- Wang, Yamagishi, Todisco et al. (2020). ASVspoof 2019: A Large-Scale Public Database of Synthesized, Converted and Replayed Speech. Computer Speech & Language 2020.
- Todisco, Delgado, Evans (2017). Constant Q Cepstral Coefficients: A Spoofing Countermeasure for Automatic Speaker Verification. Computer Speech & Language 2017.
- Ren, Fang, Liu (2019). Replay Attack Detection Based on Distortion by Loudspeaker for Voice Authentication. Multimedia Tools and Applications 2019.
- Yi, Wang, Tao et al. (2023). Audio Deepfake Detection: A Survey.
- ElevenLabs (2026). AI Speech Classifier.