Which AI made this song?

Sometimes, with a condition attached that most tool marketing leaves out: attribution works inside a closed set of generators a classifier has already seen, and it stops working outside it. If the song came from a Suno or Udio version the model was trained on, published systems can name it with high accuracy. If it came from a model, or even a model version, the classifier has never seen, accuracy collapses toward a coin flip. And no system, academic or commercial, names the person who wrote the prompt.

Three questions that get collapsed into one

“Which AI made this” is the middle question of three. Detection asks whether the track is AI at all, the ground covered in is this song AI-generated?. Attribution asks which generator produced it. Identity asks whose voice, or whose authorship, is behind it, and that last one has no validated answer from audio at all. The middle task is defined in the research as “identifying which model or method generated a deepfake” (Phukan et al., 2025): a source-family classifier, not authorship, singer identity, or copyright ownership. Keeping the three apart matters because a confident answer to one is routinely presented as an answer to another, and the error bars are completely different.

What the published numbers actually show

The benchmark built for exactly this task is FakeMusicCaps, which scores detection and generator attribution across five text-to-music systems. Inside the known set the result looks finished: a ResNet-plus-spectrogram baseline reaches a 1.00 F1 score at naming the generator (Comanducci, Bestagini, Tubaro, 2024). Then the floor drops. The same benchmark’s open-set case, where the track may come from a generator the classifier never trained on, falls to 0.80 F1 with a thresholding rule and 0.56 F1 with an SVM. Closed-set attribution is nearly solved; open-set attribution, the situation you are actually in when a new model ships, is close to guessing.

The mechanism that makes closed-set attribution work also explains its limit. Generators stamp architecture-fixed spectral peaks into their output, so distinctive that Deezer’s interpretable detector reaches 100.00 percent on seen Suno v3.5 and Udio-130, then drops to 39.83 percent on an unseen Udio-32, worse than a coin flip (Afchar, Meseguer-Brocal, Akesbi, Hennequin, 2025). Each generator’s fingerprint belongs to that generator, and even a version bump can break it.

The unseen-generator cliff is the field’s consistent finding. Probing token spaces across generator shift, one system held a strong 99.92 percent in-distribution score but fell to a 2.6 percent held-out Udio detection rate; a codec-token variant did better, at 45.1 percent, but that is still generator-specific (Park, Kim, Koh, Saito, 2026). The same work found the direction matters: an attributor trained on Udio and tested on Suno transferred at 0.940 to 0.972 F1, but trained on Suno and tested on Udio it fell to 0.629 to 0.778. Attribution is not one skill that generalizes; it is a set of per-generator skills that do not.

The singing-voice case is narrower still

If the track is specifically a singing-voice deepfake rather than a fully generated song, there is a dedicated attribution benchmark, SVDSA, whose best model, LanguageBind, reaches 82.37 percent accuracy and a 5.35 percent equal error rate (Phukan et al., 2025). The condition is the same story in miniature: that figure was measured on a closed set of eight seen generators, labeled A01 to A08, on clean vocals with no accompaniment. Unseen generators and accompanied tracks are outside the validated envelope, so the vocal-stem version of “which AI” inherits the same closed-set boundary as the full-song version.

Commercial tools now sell attribution. Check what stands behind it

Several commercial systems advertise generator attribution: ACRCloud’s detector lists named generators including Suno, Udio, Sonauto and ElevenLabs Music, Deezer reports flagging over 13.4 million AI tracks amounting to 44 percent of new uploads, and Pex’s AI Song Detector couples matching with platform attribution. Almost none publishes an error rate. Where a vendor number can be checked against research, it does not survive: IRCAM Amplify advertises “99% accuracy” and “Less than 1% false positives,” yet an independent study measured 4.7 percent false positives on human music from a comparable commercial detector (Cros Vila, Sturm, Casini, Dalmazzo, 2025), and the same study fooled one detector with a plain 22.05 kHz resample. The academic substrate these products would be measured against describes the commercial field as “closed-source and without any associated research publication” (Afchar, Meseguer-Brocal, Hennequin, 2025). A commercial “this is Suno” label is a closed-set guess whose error rate you have not been shown.

Four questions to ask before you trust the label

Because the reliability depends entirely on conditions, four questions separate an informative answer from a guess. Was the generator in the tool’s training set, or is it a new model or version? Was the file clean, or re-encoded by a social platform? Is the tool scoring the full mix or an isolated stem? And does the tool offer an open-set “unknown” option, or does it force every track into one of its known labels? “Known generator, clean file, disclosed candidate set” is the case where a name is informative. “Unknown model, compressed social clip, proprietary label” is a lead, not a conclusion.

How to read a “which AI” answer

Treat the answer as conditional on the tool’s training set. A named generator is plausible when the file is unprocessed and the candidate models are mainstream and current, because that is the seen, closed-set case. It weakens with every step away from that: a new model version, a private fine-tune, heavy processing of the kind that defeats artifact detectors, or a hybrid track where only the vocal is synthetic, the case examined in can a detector catch an AI voice-cover?. Why two tools disagree on the same file is covered in why music detectors disagree, and the forensic-examiner version of this question, what an analyst can defensibly report about generator attribution, is at which AI made this music?. The short version holds for both audiences: attribution is real, closed-set, and easy to overstate.

Sources

• Comanducci, Bestagini, Tubaro (2024). FakeMusicCaps: A Dataset for Detection and Attribution of Synthetic Music. arXiv:2409.10684.
• Park, Kim, Koh, Saito (2026). Probing Token Spaces for AI-Music Attribution under Generator Shift. arXiv:2606.08663.
• Afchar, Meseguer-Brocal, Akesbi, Hennequin (2025). A Fourier Explanation of AI-Music Artifacts. ISMIR 2025. arXiv:2506.19108.
• Afchar, D., Meseguer-Brocal, G., Hennequin, R. (2025). AI-Generated Music Detection and its Challenges. arXiv:2501.10111.
• Phukan, O. C., Girish, Akhtar, M. M., Behera, S. R., Mallick, P. (2025). Towards Source Attribution of Singing Voice Deepfake with Multimodal Foundation Models. arXiv:2506.03364.
• Cros Vila, Sturm, Casini, Dalmazzo (2025). The AI Music Arms Race: On the Detection of AI-Generated Music. TISMIR 8(1).