Translate Audio (Canary)

@nvidia1 VersionQuery / AI

NVIDIA Canary v2 speech-to-text translator. Resamples each AudioFrame to 16 kHz mono and runs the model between a configured source and target language, emitting the translated String (empty on silence). Covers a European language set under CC-BY-4.0 weights.

How it fits

Typical backends

Spoken audio from a file is translated into another language and posted as JSON.

Only speech segments from a live mic are translated, then handed to an assistant LLM.

Caveats

Hard constraintThe source and target languages must be codes the model supports (the covered European set plus English); an unsupported code degrades or empties the output.
Parameter interactionWhen the source and target language are set equal the model transcribes in place; setting them different switches it to translation.
I/O contractPunctuation and capitalization can be turned off when downstream consumers want raw lowercase tokens; the single output is always a String, and empty or silent input returns the empty string without running the model.
FallbackA per-call failure is caught and reported, and the component emits an empty string rather than crashing, so a bad frame does not stop the stream.
CostFirst run downloads several gigabytes of weights and the toolkit pulls a heavy dependency tree; pre-seed the model cache for offline deployments and expect to run on a GPU, as the billion-parameter model is slow on CPU.
CompatibilityThe model is released under CC-BY-4.0; confirm compatibility with your deployment terms.
State lifetimeConfiguration is read once at startup; changing the model, languages, punctuation, or device requires a restart.

Versions

dff16acclatestdefault
Automated release
6/29/2026