a440 News Corpus
a440 Benchmark v0.1 - Standings
Leaderboardordered by checks passed - per-test results only, no composite scoreupdated Sep 24, 2026
ModelVendorSoundcheckChecks passedMetadata layer (C2PA)Watermark layerTested
Music v1 / v2ElevenLabs00114 / 81 UI downloads signed; API deliveries carry nothing; dies on any re-save Found 5/5 originals incl. API; survives 9 attacks; gone after +5% tempo or +6% pitch Sep 24, 2026
v6-miniSuno0012 / 41 Present on download; gone after a metadata-preserving remux No public interface exists to test Sep 24, 2026
ElevenLabs Music v1 / v226 passed · 47 did not · 8 in progress
Suno v6-mini2 passed · 38 did not · 1 not testable
ElevenLabs watermark: 25 of 81 reads still in progress - the public detector caps daily checks. Every row is backed by the public corpus: originals, attacked copies, hashes, commands, verbatim detector responses. A disputed result gets rerun against the corpus.
Soundcheck 001 · first wave · tested September 24, 2026
Suno v6-minia440 Benchmark v0.1bought as a customer, $4.99
CheckQuestionVerdictEvidence
PROV-01Original credential presentPASSThe untouched download carries a signed credential declaring the audio AI-generated.
INT-01Vendor public verifier recognizes re-saved filesFAILVerifies the untouched download; finds nothing in all 9 re-wrapped copies, lossless WAV and FLAC included.
INT-02Credential trusted by the standard official toolsFAILOfficial C2PA Verify tool: issuer not recognized. Self-issued certificate, not on the trust list.
INT-03Independent verification path existsFAILThe fallback fingerprint sits on the vendor servers; no independent check exists.
Provenance and interoperability checks above; all durability results in the battery below - label gone on every attack, watermark not testable (no public interface; the strongest open mark dies to a 5% speed change and streaming compression). Evidence A, B and C archived in the corpus with raw detector responses. 1 of 4 checks passed.
2 / 41 public tests passed
One check NOT TESTED - no public interface. The marking exists, but it does not survive the real world.
Layers in scope: provenance, durability, interoperability. Per-test results only, no composite score.
ElevenLabs Music v1 / v2a440 Benchmark v0.1UI and API pathsbought as a customer
CheckQuestionVerdictEvidence
PROV-01aCredential on UI downloadsPASSSigned C2PA manifest, issuer Eleven Labs Inc. (DigiCert), action c2pa.created, digitalSourceType trainedAlgorithmicMedia, on WAV and MP3 from the app.
PROV-01bCredential on API deliveriesFAILAll 4 API MP3s carry zero provenance metadata. Same account, same models.
Provenance checks above; watermark-on-every-path and all durability results in the battery below - intact through container, codec, dynamics and edits, gone on tempo, pitch, or the acoustic path. The public detector errored for one window (10:27-10:28, recovered by 11:11); its classifier channel is speech-oriented, the watermark channel is the music verdict. 1 of 2 checks passed.
14 / 81 public tests passed
Two provenance layers with opposite failure modes: the metadata layer is UI-only and dies on any re-save; the watermark layer covers every path and survives format churn, then breaks on tempo, pitch, or the acoustic path.
Per-test results only, no composite score. 25 watermark reads still in progress - the public detector caps daily checks.
Attack battery v2 - 38 named tests, cited per test
Durability suitesources: [SoK25] arXiv:2503.19176 · [RAW25] Interspeech 2025, arXiv:2505.19663 · [AS24] AudioSeal, ICML 2024 · framing: [AMR25] TISMIR 2025, 10.5334/tismir.254 · [AFC25] arXiv:2501.10111 · a440 = our addition, labeledPASS = the layer still verifies after the attack
TestAttackSourceSuno v6-mini C2PASuno v6-mini watermarkElevenLabs C2PA (UI)ElevenLabs watermark
Container and metadata
DUR-04Remux, losslessa440FAILNO INTERFACEFAILPASS
DUR-05Metadata stripa440FAILNO INTERFACEFAILPASS
Codec chains
DUR-06MP3 128kSoK25, AS24FAILNO INTERFACEFAILPASS
DUR-07MP3 64kSoK25FAILNO INTERFACEFAILNT
DUR-08AAC 128kRAW25FAILNO INTERFACEFAILPASS
DUR-09Opus 96kRAW25FAILNO INTERFACEFAILNT
DUR-10EnCodec 6 kbps round-tripRAW25FAILNO INTERFACEFAILNT
Temporal
DUR-11Tempo +5%, pitch preservedSoK25, RAW25, AS24FAILNO INTERFACEFAILFAIL
DUR-12Pitch +6%, tempo preservedSoK25FAILNO INTERFACEFAILFAIL
DUR-13Tempo + pitch combinedSoK25FAILNO INTERFACEFAILNT
DUR-14Tempo -5%AS24FAILNO INTERFACEFAILNT
Spectral
DUR-15Lowpass 8 kHzSoK25, AS24FAILNO INTERFACEFAILNT
DUR-16Highpass 200 HzSoK25, AS24FAILNO INTERFACEFAILNT
DUR-17EQ scoop -12 dB, 1-4 kHzAS24FAILNO INTERFACEFAILNT
Dynamics and gain
DUR-18Compressor 2:1a440FAILNO INTERFACEFAILNT
DUR-19Hard limitera440FAILNO INTERFACEFAILNT
DUR-20Loudnorm -14 LUFSa440FAILNO INTERFACEFAILPASS
DUR-21Gain -12 dBRAW25, AS24FAILNO INTERFACEFAILNT
Spatial and effects
DUR-22Mono downmixa440FAILNO INTERFACEFAILNT
DUR-23Reverb / IR convolutionRAW25, SoK25FAILNO INTERFACEFAILNT
DUR-24Chorusa440FAILNO INTERFACEFAILNT
Edits
DUR-25Trim first 30 sSoK25FAILNO INTERFACEFAILPASS
DUR-26Trim middle 30 sSoK25FAILNO INTERFACEFAILPASS
DUR-27Trim last 30 sSoK25FAILNO INTERFACEFAILPASS
DUR-28Clip 10 sSoK25FAILNO INTERFACEFAILPASS
DUR-29Fade in / outAS24FAILNO INTERFACEFAILNT
DUR-30Splice 15 s foreign audioAS24FAILNO INTERFACEFAILNT
Hostile-realistic
DUR-31Streaming upload sima440 / RAW25FAILNO INTERFACEFAILPASS
DUR-32Acoustic path sim (simulated)SoK25FAILNO INTERFACEFAILFAIL
Literature additions
DUR-33Gaussian + background noiseSoK25, AS24, RAW25FAILNO INTERFACEFAILNT
DUR-34Bitcrush 8-bitSoK25FAILNO INTERFACEFAILNT
DUR-35Sample suppressionSoK25FAILNO INTERFACEFAILNT
DUR-36Resample 48k-16k-48kSoK25, AS24FAILNO INTERFACEFAILNT
DUR-37Polarity inversionRAW25FAILNO INTERFACEFAILNT
DUR-38Echo 100 msRAW25, AS24FAILNO INTERFACEFAILNT
DUR-39Boost +6 dBAS24FAILNO INTERFACEFAILNT
DUR-40Silence maskAS24FAILNO INTERFACEFAILNT
DUR-41Telephone bandpassAS24FAILNO INTERFACEFAILNT
Columns are the provenance layers a third party can actually check: C2PA manifest via the c2pa reference SDK, watermark via the vendor public detector. Suno exposes no public watermark interface, so that column is not testable. FAIL on a C2PA column means the manifest is gone; every attacked copy still plays. Suno C2PA column measured on corpus track s01 (Sep 24, 2026); ElevenLabs watermark column measured on the 002 corpus; NT cells are queued, corpus expansion in progress. Raw results: corpus.
Survival by attack familysame data, grouped - green passed, red did not, gray read in progress
FamilySuno labelElevenLabs labelElevenLabs watermark
Container and metadata
0/2
0/2
2/2
Codec chains
0/5
0/5
2/5
Temporal
0/4
0/4
0/4
Spectral
0/3
0/3
0/3
Dynamics and gain
0/4
0/4
1/4
Spatial and effects
0/3
0/3
0/3
Edits
0/6
0/6
4/6
Hostile-realistic
0/2
0/2
1/2
Literature additions
0/9
0/9
0/9
Suno exposes no public watermark interface, so only its label column is measured. ElevenLabs watermark reads still landing - gray turns green or red as the daily detector cap allows.
Detection check - one open detector against this corpus
Baseline: un-attacked trackssource: [SNX25] Rahman et al., SONICS / SpecTTTra, ICLR 2025, arXiv:2408.14080authors' published model, run locally, Sep 24 2026
TrackOriginFake probabilityDetector callVerdict
s01Suno v6-mini, Sep 20260.17realMISS
s03Suno v6-mini, Sep 20260.05realMISS
s05Suno v6-mini, Sep 20260.12realMISS
s07Suno v6-mini, Sep 20260.18realMISS
el01ElevenLabs Music v2, Sep 20260.30realMISS
el02ElevenLabs Music v2, Sep 20260.08realMISS
hc01Human: Joplin, Maple Leaf Rag, 1899 (public domain)0.04realCORRECT
Decision line 0.5. Scores from the authors' released SpecTTTra-alpha-120s model, middle-chunk inference per their code. The detector was trained on 2024-25 Suno and Udio output; every AI track above is current-generation and un-attacked. Control: the human recording is also clean in the ElevenLabs watermark check (no mark found, 15:56 PDT). One open detector, not every detector. Per-file scores archived in the corpus.
Under the 38-attack batterysame detector, the same attacked copies the battery measuresattacks where fake probability crosses 0.5
CorpusCrosses the lineWhich attacks
Suno2 of 38DUR-05 metadata strip 0.75 · DUR-28 clip 10 s 0.56
ElevenLabs5 of 38DUR-23 reverb 0.501 · DUR-25 trim first 30 s 0.67 · DUR-26 trim middle 30 s 0.75 · DUR-27 trim last 30 s 0.78 · DUR-33 noise 0.90
Attacking does not hide a track from this detector. Most attacks leave it dead; a few wake it up - trimming an ElevenLabs track is what finally flags it. The numbers move both ways, which is itself the finding: detection confidence is not a stable signal.
Reference experiment - independent measurement, not part of the Suno / ElevenLabs standings
SilentCipher: a reference watermarkSony's open research watermark, embedded by a440 into one ElevenLabs Music track (el01 from the 002 corpus), then run through 20 of the same battery attacksreference round, not a vendor claim
13 full · 5 partial · 2 dead, of 20 attacks
Why this exists: a control for the question "is ElevenLabs' mark unusually weak" - an independent mark, our own encode and decode, the same battery. It dies on the same attack that kills ElevenLabs, and on additive noise.
TestAttackSilentCipherElevenLabs mark
DUR-04Remux, losslessFULLPASS
DUR-05Metadata stripPARTPASS
DUR-24ChorusPARTqueued
DUR-25Trim first 30 sFULLPASS
DUR-26Trim middle 30 sFULLPASS
DUR-27Trim last 30 sPARTPASS
DUR-28Clip 10 sFULLPASS
DUR-29Fade in / outFULLqueued
DUR-30Splice 15 s foreign audioPARTqueued
DUR-31Streaming upload simFULLPASS
DUR-32Acoustic path sim (simulated)DEADFAIL
DUR-33Gaussian + background noiseDEADqueued
DUR-34Bitcrush 8-bitFULLqueued
DUR-35Sample suppressionFULLqueued
DUR-36Resample 48k-16k-48kFULLqueued
DUR-37Polarity inversionFULLqueued
DUR-38Echo 100 msPARTqueued
DUR-39Boost +6 dBFULLqueued
DUR-40Silence maskFULLqueued
DUR-41Telephone bandpassFULLqueued
FULL = every 10-second chunk of the payload recovered, PARTIAL = some chunks, DEAD = none. The ElevenLabs column shows only attacks already read through its public detector (about one read per hour); the rest are queued. One track, 40-bit payload, github.com/sony/silentcipher. Raw matrix: corpus.
a440 Benchmark v0.1 - Methodology
The principle, the evidence chain, the six layers, the 38 attacks - paintedHow a recording earns an a440 verdict →