Translate Text (NLLB)

How it fits

Typical backends

Caveats

• I/O contractThe output arm follows the resolved downstream port: a single String joins all segments, a [String] keeps them split, and the language-tagged record arms attach the configured source and target codes (NLLB carries both distinctly, unlike a model that auto-detects its source).
• Hard constraintBoth source and target must be NLLB-200 FLORES-200 codes in the script-tagged form such as eng_Latn or zho_Hans; the source is set on the tokenizer and the target is resolved to the forced decode-start token, so an unknown target code yields a wrong start token. The target code has no default and must be supplied.
• Parameter interactionThe device is watched and changing it rebuilds the tokenizer and model in-process, so the first call after such a change pays the load cost; both language codes are re-read every call and are free to switch because the source token is set and the target token is resolved per request.
• Hard constraintThe NLLB-200 checkpoint is staged offline through the model file slot — a transformers snapshot (config, weights, tokenizer) bound to the worker and loaded locally with no runtime download. Pick the variant by binding the matching staged checkpoint; the distilled-600M default is the public, non-gated repo.
• FallbackA requested CUDA device falls back to CPU when no GPU is visible. Empty input, or a list whose entries are all empty, returns an empty result without invoking the model, and a fault inside translation degrades to that same empty result rather than dropping the message.
• CostThe distilled six-hundred-million default is light enough for modest GPUs; the 1.3B and 3.3B variants trade memory and latency for quality. A higher num_beams widens beam search for quality at the cost of latency. A dedicated NMT model is cheaper and faster than routing translation through a chat LLM.