Contents
- How big is the lab-to-wild gap?
- Why do unseen generators break detection?
- Why does compression make it worse?
- Can the detector be reading the production instead of the voice?
- Why doesn’t a benchmark number predict a commercial tool?
- Does any method survive the wild better?
- What does this mean for trusting a result?
- Sources
Music detectors fail in the real world because they are trained on a benchmark’s generators and clean audio, and deployed on everything else. Move off that ground, an unseen model, a lossy codec, a resample, a dense mix, and equal error rates that read near one percent in the lab climb into the tens of percent. The gap between the benchmark figure and the one you get on your track is not measurement noise. It is the actual reliability of the tool.
How big is the lab-to-wild gap?
Large enough to invert the answer. Afchar, Meseguer-Brocal, Akesbi and Hennequin (2025) built a detector that scores 100 percent on the generator versions it trained on, then watched it fall to 39.83 percent, worse than guessing, on an unseen version. The singing-voice literature shows the same cliff: the SingFake system reaches a 4.62 percent equal error rate on singers it was trained on, then rises to 42.77 percent on an unseen language or musical style (Zang, Zhang, Heydari, Duan, ICASSP 2024). The benchmark figure describes the easiest case, not the next model that ships.
Why do unseen generators break detection?
Because a detector learns the fingerprint of the generators in its training set, and each generator’s peaks are its own. Afchar and colleagues proved the fingerprint is architecture-specific, which is exactly why it does not transfer: a detector tuned on one model’s peaks has no reference for another’s. Off-the-shelf speech anti-spoofing models make the point at the extreme, collapsing to roughly 50 percent, no better than a coin flip, when pointed at singing at all (Zang et al., 2024). Because new versions of Suno and Udio ship faster than detectors retrain, the track you care about is often from a generator no detector in reach was built to read.
Why does compression make it worse?
Because the fingerprint sits at fixed frequencies, and lossy compression cuts exactly there. Afchar, Meseguer-Brocal and Hennequin (2024) measured a detector that scores about 99.8 percent on clean audio degrading under ordinary transforms:
| Transform | Detector accuracy |
|---|---|
| Clean | 99.8% |
| MP3, 64 kbps | 73.8% |
| Opus, 64 kbps | 64.6% |
| AAC, 64 kbps | 58.4% |
AAC at 64 kbps, a common streaming setting, pushes accuracy below 60 percent. A two-semitone pitch shift lands at 66.6 percent. Time-domain edits are the exception, surviving at 88.6 percent for time-stretching and 96.9 percent through added reverb, but almost every song reaches you already re-encoded, so the detector is usually reading a degraded signal before the generalization gap is even counted. The full picture is in does MP3 compression defeat AI music detectors.
Can the detector be reading the production instead of the voice?
Often, yes, and it does not transfer to a cloned vocal. Desblancs, Meseguer-Brocal, Hennequin and Moussallam (ISMIR 2024) found that singer-identification models built on full mixtures encode the instrumental background more than the voice, and degrade sharply on cloned singers. A model that learned to lean on arrangement and mastering will read a new track’s production, not its performer, and a real instrumental under a cloned vocal is exactly the case it handles worst. That blind spot is why the partly-AI song defeats every deployed family, a point developed in is this song AI-generated.
Why doesn’t a benchmark number predict a commercial tool?
Because a vendor’s accuracy claim is not a measured reliability figure. Cros Vila, Sturm, Casini and Dalmazzo (2025) tested a deployed commercial AI-music detector and found it “easily fooled by simply resampling audio to 22.05 kHz,” a change that does not alter how the track sounds. Our own work points the same way: in our internal testing, a processing cascade drove a deployed commercial AI-music detector to 0 percent AI on tracks it would otherwise flag. The method stays in-house, but the finding is ours to report, and it matches the published result, a consumer detector is far easier to defeat than its marketing suggests.
Does any method survive the wild better?
One family fails differently, which is the useful kind of robustness. Frohmann, Epure, Meseguer-Brocal et al. (2025) skip the signal fingerprint entirely, transcribe a song’s lyrics with speech recognition, and classify the words. Because lyrics survive processing that destroys spectral artifacts, their method holds around 85 to 90 percent where a spectral detector falls to 56.9 percent on an unseen generator, and stays near 89 percent under a pitch shift while the artifact network drops to 59.0 percent. It has its own limit, it can be beaten by changing the transcribable words, but it fails on different inputs than the signal readers, which is exactly why running both beats trusting either. A separate line keeps more of the signal instead of less: high-resolution singing models retain the frequencies above 8 kHz that speech-derived detectors discard, on the finding that singing carries distinguishing cues in that band (Chen, Hu, Huang, Wu, Lee, Jang, 2026). The same authors behind the lyrics work note the scale of the problem, more than 10,000 AI tracks a day delivered to one streaming platform alone.
What does this mean for trusting a result?
It means a benchmark number is a ceiling, not a promise. The lab produces the reassuring figures and the wild produces the real ones. Treat a confident verdict on an unfamiliar generator, a compressed track, or a dense mix as weak evidence, corroborate with a second method that reads a different trace and with provenance where it exists, and reserve “undetermined” for the degraded and unverifiable cases where an honest tool cannot know. For how far any of these tools can be trusted across media, see are AI detectors accurate.
Sources
- Afchar, Meseguer-Brocal, Akesbi, Hennequin (2025). A Fourier Explanation of AI-Music Artifacts. ISMIR 2025.
- Afchar, Meseguer-Brocal, Hennequin (2024). Detecting Music Deepfakes Is Easy but Actually Hard.
- Zang, Zhang, Heydari, Duan (2024). SingFake: Singing Voice Deepfake Detection. ICASSP 2024.
- Desblancs, Meseguer-Brocal, Hennequin, Moussallam (2024). From Real to Cloned Singer Identification. ISMIR 2024.
- Cros Vila, Sturm, Casini, Dalmazzo (2025). The AI Music Arms Race: On the Detection of AI-Generated Music. TISMIR 8(1).
- Frohmann, Epure, Meseguer-Brocal, Schedl, Hennequin (2025). AI-Generated Song Detection via Lyrics Transcripts. ISMIR 2025.
- Chen, Hu, Huang, Wu, Lee, Jang (2026). Joint Fullband-Subband Modeling for High-Resolution SingFake Detection.