Generate Text (Ollama)

@ollama1 SürümGenerate

Drives a chat turn against a locally served Ollama model from a user and system String, streaming the reply as String chunks and closing with a StreamEnd; for a bundled GPU sidecar prefer generate_text_sglang.

How it fits

Typical backends

Voice assistant — transcribe speech, reason locally, speak the answer.

Document Q&A — OCR a page, answer over it, emit the result as JSON.

Caveats

Hard constraintA reachable Ollama runtime is required; it is declared as a dependency and deployed alongside, with its address read from the environment at startup. model_name is pulled on startup if it is not already cached, so a missing model or unreachable runtime fails before any prompt is served.
I/O contractThe two inputs are the user message and the system message for one chat turn; the generated text leaves on the first output and an end-of-turn marker on the second.
I/O contractWith streaming on, the reply arrives as incremental String chunks; with it off the whole reply is emitted as a single chunk at completion. emit_stream_end is what produces the StreamEnd on the second output, so consumers that need an end-of-turn signal must keep it on.
Parameter interactionhistory_messages set to zero runs stateless, a negative value keeps the full conversation, and a positive value keeps the last N turns; max_parallel only takes effect in the stateless case, since carrying history forces sequential turns.
Parameter interactioncontext_tokens sizes the context window the server holds a turn in, and left at zero the server's own default applies. Whatever the system message, the retained history and the user message do not fit into that window is dropped without an error, and the reply is written from what survived, so a long system message or a long history needs the window raised to match.
Parameter interactionpreempt_on_new_input cancels an in-flight generation when a fresh prompt arrives, so downstream sees a truncated stream for the abandoned turn.
Parameter interactiondisplay_reasoning appends a structured reasoning instruction to the system message; the structure is requested, not enforced, and it costs extra tokens and latency. JSON mode likewise only requests a JSON object from the model rather than guaranteeing one.
State lifetimeConversation history lives in the process and resets on restart, and every knob binds once at startup, so changes require a redeploy.

Sürümler

74be85e8latestdefault
Automated release
02.08.2026