Contents
Voice liveness detection answers one question: is this a live human speaking into the microphone, or a recording being played back at it? That is a different question from whether the voice is synthetic. A replayed clip has traveled through a loudspeaker, across a room, and into a second microphone, and that journey leaves physical marks a live voice does not carry. Liveness detectors are classifiers trained to read those marks.
Liveness is not AI detection
An AI-voice detector asks whether speech was synthesized. A liveness detector asks whether the audio arriving right now is a first-hand acoustic event. The two fail independently: a genuine recording of a real person contains no synthesis to find, yet playing it into a bank’s voice-verification line is still an attack. The ASVspoof benchmark series formalized the split, running a physical-access track for replayed audio alongside the logical-access track for synthetic speech. As the benchmark’s organizers put it, automatic speaker verification is “vulnerable to spoofing, also referred to as ‘presentation attacks’” (Wang, Yamagishi, Todisco et al., Computer Speech & Language 2020), and replay is the cheapest of them. Our companion piece on how live deepfake detection works covers the streaming side; this article covers the liveness mechanism itself.
Why a replay is physically different from a live voice
Three tells dominate the literature. First, loudspeaker nonlinear distortion: a small speaker cannot reproduce a voice perfectly, and its harmonic distortion pattern is measurable in the re-captured audio. Second, low-frequency attenuation: consumer playback hardware rolls off the low end, so a replayed voice arrives with a thinner spectral floor than a directly captured one. Third, a doubled channel: the replay passes through a second room and a second microphone, stacking a fresh acoustic signature on top of the original one. Ren, Fang and Liu (Multimedia Tools and Applications 2019) built a support-vector-machine classifier on loudspeaker-distortion features and reported a true-positive rate of 97.78% against a true-negative rate of 99.62%, where the far-field baseline they compared against managed 89.91%.
The features detectors actually read
The workhorse front-end of the physical-access literature is the constant-Q cepstral coefficient, or CQCC, whose static coefficients carry the replay signal; Todisco and colleagues introduced it as a spoofing countermeasure and reported an equal-error rate of 1.85% on the RedDots-Replayed corpus (Todisco et al., 2017). The high end of the spectrum matters too. Witkowski, Kacprzak, Żelasko, Kowalczyk and Gałka (INTERSPEECH 2017) showed that restricting CQCC features to the 6 to 8 kHz band cut the equal-error rate from 11.86% to 5.13% on the ASVspoof 2017 development set, because the anti-aliasing filters in a replay chain’s digital-to-analog conversions leave their strongest traces in exactly that band.
The forensic tell: a second mains hum
Outside the biometric literature there is a court-adjacent version of the same idea. The electrical-network-frequency signature, the faint 50 or 60 Hz mains hum a recording device picks up, is acquired at capture time. A file that contains more than one ENF series has been through an analog copying pass, because the second series was picked up while the first recording was being replayed and re-captured. The European Network of Forensic Science Institutes names this dual-ENF check in its authenticity best-practice manual (ENFSI FSA-BPM-002, §5.4.1), which makes it the closest thing replay detection has to a standards-grade indicator.
Where liveness detection runs today
The ASVspoof 2017 and 2019 physical-access evaluations institutionalized the task and gave it public corpora and baselines (Wang, Yamagishi, Todisco et al., Computer Speech & Language 2020). Commercially, the same problem is packaged as voice anti-spoofing in call-center and mobile-biometric products; in the SASV 2022 challenge, which scored spoofing-aware speaker verification end to end, the winning system from ID R&D reported a combined equal-error rate of 0.136%. Those numbers describe controlled evaluations, and the gap between them and deployment is the subject of the rest of this cluster.
What the layer closes, and what it cannot
The reason the layer exists is stark: Villalba and Lleida (BioID 2011) measured that a speaker-verification system with no liveness check accepts 68% of far-field replay attempts at its own operating threshold. Liveness detection closes that hole, but not absolutely. Ren, Fang and Liu’s own cross-loudspeaker test dropped their true-positive rate to a floor near 89.7% when the playback device changed, and every acoustic tell in this article assumes there is a loudspeaker at all: audio injected electrically, with no playback and no second room, presents none of these features. The method families behind each tell, and their measured limits, are laid out in live vs recorded speaker detection: the methods, and what happens when someone deliberately attacks the layer is covered in can voice liveness detection be bypassed?. For the first-pass check on whether a clip is synthetic at all, see how to tell if audio is AI-generated.
Sources
- Ren, Fang & Liu (2019). Replay Attack Detection Based on Distortion by Loudspeaker for Voice Authentication. Multimedia Tools and Applications. DOI: 10.1007/s11042-018-6834-3.
- Villalba, J. & Lleida, E. (2011). Detecting Replay Attacks From Far-Field Recordings on Speaker Verification Systems. BioID 2011.
- Todisco, M. et al. (2017). Constant Q Cepstral Coefficients: A Spoofing Countermeasure for Automatic Speaker Verification. Computer Speech & Language.
- Witkowski, M., Kacprzak, S., Żelasko, P., Kowalczyk, K. & Gałka, J. (2017). Audio Replay Attack Detection Using High-Frequency Features. INTERSPEECH 2017. DOI: 10.21437/Interspeech.2017-776.
- Wang, Yamagishi, Todisco et al. (2020). ASVspoof 2019: A Large-Scale Public Database of Synthesized, Converted and Replayed Speech. Computer Speech & Language 2020.
- European Network of Forensic Science Institutes (2022). Best Practice Manual for Digital Audio Authenticity Analysis (ENFSI-FSA-BPM-002).