What instruments are in a song

Accuracy is measured on the labelled OpenMIC-2018 dataset and differs per instrument. Guitar, piano, synthesizer and drums are identified confidently — on held-out data those answers are right 87–97 % of the time. Violin, cello, saxophone and organ are weaker. Vocals, cymbals, accordion and mallet percussion are poorly separated, so they never enter the main answer and are shown as “possibly”.

The model cannot reliably tell bass from other low parts: on the held-out split it “heard” bass in almost every track, which means it was guessing rather than detecting. Showing that is worse than showing nothing, so bass is excluded. For the same reason trumpet, trombone, flute and clarinet are not separated, and banjo, mandolin and ukulele usually land under guitar.

They are two thresholds. An instrument reaches “confirmed” only if, on the held-out split, a score like that really did mean the instrument was present at least 8 times out of 10. “Possibly” is a lower bar — roughly 6–7 times out of 10 — so do not rely on it without listening.

This tool identifies, it does not separate. For pulling out individual tracks use the stem splitter; for a note-level transcription use Audio to Notes.

The model listens in 10-second windows, so the map resolves to about 5–10 seconds. That is verse-and-chorus level, not note level. For note accuracy use Audio to Notes.

The same model analyses a rolling ten-second window and refreshes the list roughly every one and a half seconds. It suits rehearsals and lessons. For best results turn off noise suppression in your browser.

There is no hard limit. Long recordings are sampled with an even grid of windows, capped in number so the analysis stays quick.