Classify Speech Emotion (Zero Shot)

@laion1 VersionClassify

Embeds each AudioFrame and a runtime-supplied set of emotion labels into one CLAP space and ranks the labels by audio-text similarity, emitting [Classification], the raw [Double] scores, or just the best label as a String.

How it fits

Typical backends

Score recorded speech against a custom emotion label set and log the result.

Score live microphone audio against custom emotion labels and route on the match through a conditional branch.

Caveats

State lifetimeThe label set, the CLAP checkpoint, and the device are fixed at startup, while the prompt wording and the competitive-versus-independent scoring mode are re-read at runtime.
CompatibilityAudio is resampled to 48 kHz internally regardless of the upstream sample rate, matching the rate CLAP was trained at.
AccuracyIn competitive mode the scores form a softmax over the labels and sum to one; in independent mode each label is a separate sigmoid and the scores do not sum to one.
AccuracyScores are relative similarities within the supplied label set only, and phrasing each emotion as a natural spoken-tone sentence rather than a bare word improves ranking quality.
I/O contractWith no labels configured the output is empty, so downstream consumers must handle the empty case.

Versions

9d644990latestdefault
Automated release
6/29/2026