Translate Audio (Whisper)

@openai1 VersionQuery / AI

Resamples each AudioFrame to 16 kHz and runs Whisper's speech-to-English translate task, emitting an English String (empty on silence). The output is always English; the checkpoint is chosen by model_size.

How it fits

Typical backends

Translate a foreign-language recording into English and post it as JSON.

Speech-gated live translation feeding an assistant.

Caveats

Hard constraintmodel_size must be a multilingual Whisper checkpoint; English-only (.en) and turbo checkpoints do not support the translate task and fail to load.
I/O contractThe output is always English; this component only supports X-to-English. Use a multilingual-target audio translator for other languages.
Parameter interactioncompute_type should match device (a 16-bit or int8 mode for GPU, int8 or 32-bit for CPU); a mismatched pairing fails at startup. vad_filter true gates silence before decoding and reduces hallucinations.
I/O contractlanguage is a source-language hint only; empty auto-detects the spoken tongue (slower) and never changes the English output. Empty audio returns the empty string without running the model.
FallbackThe GPU build requires CUDA 12 with cuDNN 9; a mismatched runtime silently runs on CPU.

Versions

9760ed71defaultlatest
Automated release
6/25/2026