Analyze Image (vLLM VLM)

@vllm1 VersionClassify

Sends one or more images plus a String prompt to a vision-language model served by a local vLLM sidecar and returns the model's reply as a String. The image input is a oneof spanning a single image, an image-with-label, a batch, or labeled batch.

How it fits

Typical backends

Visually reason over frames from a video file and emit a strict-JSON answer that a downstream service parses into typed fields.

Analyze a file-loaded image and route the textual answer to a log sink for monitoring.

Caveats

CompatibilityA local vLLM sidecar serving the bound vision model must be reachable; the model bound to weights is staged into the sidecar's volume at deploy time and loaded from disk, so the first deployment pays the staging and warm-up cost before serving.
State lifetimetemperature, max_tokens, json_mode and timeout_seconds are hot-reloadable per request, while the model, image encoding and concurrency wiring are fixed once at startup.
Hard constraintmax_parallel sizes the request pool at startup, so scaling concurrency requires a redeploy; an out-of-range hot update to a tunable is rejected and the previous value is kept.
I/O contractjson_mode asks the endpoint to constrain output to a JSON object, but the result still arrives as a String — pair it with a JSON parser downstream to recover typed fields.
CostEach image is re-encoded to image_format.jpeg or image_format.png and base64-embedded into the request, so large batches inflate payload size and per-call latency.

Versions

1f498531defaultlatest
Rebuild against the current pipeml
7/29/2026