Describe Image Regions (Florence 2)

How it fits

Typical backends

Caveats

• I/O contractEach region pairs an axis-aligned Rectangle<Double> with a short free-text caption; there is no class id or confidence score to threshold on.
• AccuracyRegion proposals and captions are entirely model-decided, so the count varies with scene complexity and a near-empty frame may yield no regions at all.
• FallbackA frame whose inference fails is logged and returns an empty list rather than crashing, and the model falls back to CPU when no GPU is present — impractical for live video on this autoregressive model.
• LatencyCaptions are decoded autoregressively with beam search, so per-frame latency grows with the number of regions and is well above that of a plain detector.
• State lifetimeThe checkpoint and the inference backend are captured once at startup; changing either requires a redeploy.