Translate Audio (SeamlessM4T v2)

@meta1 VersionQuery / AI

Downmixes one AudioFrame to mono, resamples to 16 kHz, and runs SeamlessM4T v2 speech-to-text translation into a configured target language, returning a single String; silent or failing audio yields the empty string.

How it fits

Typical backends

Translate a recorded clip set into another language and publish the running text. The frame source paces audio, this component decodes and translates each chunk, and the JSON sink streams the strings out.

Compare same-language transcription against cross-language translation on the same source by fanning one audio stream into a verbatim ASR path and this translation path.

Caveats

FallbackEmpty audio short-circuits to the empty String before the model runs, and any exception raised during decoding is caught and also returned as the empty string, so a failing call is indistinguishable from genuine silence.
I/O contractEach call returns exactly one String; multi-channel input is downmixed to mono and any sample rate other than 16 kHz is resampled internally before inference.
Hard constraintThe target language must be supplied as a valid three-letter SeamlessM4T language code; an unrecognised code surfaces as a failed call and therefore an empty result.
State lifetimeThe model name and device are read once at container start and the model is loaded a single time; changing the target language for a run requires the container to be restarted.
CostThe default checkpoint is roughly a billion-parameter model carrying a non-commercial weight licence and a multi-gigabyte first-run download; GPU is strongly preferred because CPU inference is slow.

Versions

e9c998bblatestdefault
Automated release
6/29/2026