Classify Audio Zero Shot (HuggingFace)

@huggingface1 VersionClassify

Zero-shot audio classifier: self-downloads a HuggingFace CLAP checkpoint and ranks a runtime label set against each AudioFrame by audio-text similarity. Emits [Classification], a top Classification, or raw [Double] scores. Model hot-swaps.

How it fits

Typical backends

Score recorded audio against a custom sound watchlist of free-text labels and route on the match through a conditional branch.

Watch live microphone audio for a named sound and log when it is heard.

Caveats

State lifetimeThe CLAP checkpoint is watched at runtime and hot-swapped in place when the model id changes, clearing the GPU cache and recomputing the label embeddings for the new encoder; the label set, device and precision are fixed at startup, while the prompt wording and competitive-versus-independent mode are re-read every frame.
CompatibilityAudio is resampled to 48 kHz internally regardless of the upstream sample rate, matching the rate CLAP was trained at.
AccuracyIn competitive mode the scores form a softmax over the labels and sum to one; in independent mode each label is a separate sigmoid and the scores do not sum to one.
AccuracyScores are relative similarities within the supplied label set only, and phrasing each label as a natural sentence rather than a bare word materially improves ranking quality.
I/O contractWith no labels configured the output is empty; an empty AudioFrame yields zero scores for every label without running the audio encoder, so downstream consumers must handle both cases.

Versions

8d6cc911latestdefault
Automated release
6/26/2026