Classify Audio Event (Zero Shot CLAP)

@laion1 VersionClassify

Embeds each AudioFrame and a runtime-supplied label set into one CLAP space and ranks the labels by audio-text similarity, emitting all labels as [Classification], the single best as Classification, or the raw scores as [Double].

How it fits

Typical backends

Score recorded audio against a custom sound watchlist and route on the match through a conditional branch.

Watch live microphone audio for a named sound and log when it is heard.

Caveats

State lifetimeThe label set, the CLAP checkpoint, and the device are fixed at startup, while the prompt wording and the competitive-versus-independent scoring mode are re-read every frame, with the label embeddings recomputed only when the wording actually changes.
CompatibilityAudio is resampled to 48 kHz internally regardless of the upstream sample rate, matching the rate CLAP was trained at.
AccuracyIn competitive mode the scores form a softmax over the labels and sum to one; in independent mode each label is a separate sigmoid and the scores do not sum to one.
AccuracyScores are relative similarities within the supplied label set only, and phrasing each label as a natural sentence rather than a bare word materially improves ranking quality.
I/O contractWith no labels configured the output is empty; an empty AudioFrame yields zero scores for every label without running the audio encoder, so downstream consumers must handle both cases.

Versions

98551fa1latestdefault
Automated release
6/26/2026