Generate Image Caption (BLIP)

1 VersionGenerate

Captions each Image with the HuggingFace checkpoint bound at model (Florence-2 or a BLIP / BLIP-2 bundle) and emits either a plain String caption or a structured Caption record, depending on the downstream arm.

How it fits

Typical backends

Caption frames read from files and deliver the captions over HTTP.

Seed a caption for each frame, then expand it into richer narration with a text model.

Caveats

I/O contractThe output arm is fixed when the backend deploys: a text consumer pins the String caption, a structured consumer pins the named Caption record. The decoder exposes no calibrated score, so the record's confidence field is a constant `1.0` placeholder.
Hard constraintmodel is a bound HuggingFace bundle directory (a `config.json`, safetensors weights, and the processor/tokenizer files) loaded offline. There is no repo-id parameter and no runtime download; the checkpoint is part of the deployed backend.
Parameter interactionThe architecture family is chosen at startup from the bundle's `config.json` (`architectures` / `model_type`): a Florence-2 checkpoint loads the causal-LM path, anything else loads the BLIP vision encoder-decoder path. prompt is the Florence-2 task tag (`<CAPTION>`, `<DETAILED_CAPTION>`, `<MORE_DETAILED_CAPTION>`) that selects caption detail; BLIP-family checkpoints ignore it.
Fallbackdevice requesting CUDA resolves to CPU when no compatible accelerator is visible at startup, though these models are slow on CPU. A frame that decodes to zero pixels is captioned as the empty string for the resolved arm rather than crashing, so every input still yields exactly one output.
AccuracyThe output is a single generated caption, not grounded detection; for prompt-driven visual question answering use a vision-language model instead.
State lifetimeThe bound model, the resolved output arm, the prompt tag, and the device are read once at startup; changing them requires a redeploy.

Versions

b01f3d78latestdefault
Automated release
6/29/2026