Segment Image (Ultralytics YOLOE)

@ultralytics1 VersionSegment

Open-vocabulary instance segmentation over a single Image. Masks instances with a YOLOE checkpoint, optionally steered by prompt_classes, then projects the result onto whatever the downstream connection requests, e.g. [(Segmentation, BoundingBox)].

How it fits

Typical backends

Segment a still image, overlay the masks and boxes, and serve the annotated frame over HTTP.

Segment every frame of a video and stream the live annotated view to a browser.

Detect instances and forward the boxes to a tracker that maintains identities across frames.

Caveats

Parameter interactionSet prompt_classes only for promptable checkpoints; prompt-free weights require an empty prompt list, and steering a prompt-free checkpoint produces undefined results.
Hard constraintThe model slot must resolve to exactly one checkpoint file; zero or multiple matches stop the component at startup.
Parameter interactionEmpty object_classes keeps every detected category; a non-empty list restricts inference to those class indices.
Parameter interactionthreshold drops low-confidence instances and iou controls overlap suppression; both shape how many instances survive to the output.
I/O contractThe output shape is fixed once the downstream connection is wired; every supported projection is a different view of the same per-instance result.
CompatibilityA non-empty prompt_classes fetches the MobileCLIP text encoder once at startup to embed the prompt names, so prompt mode needs outbound network at boot; prompt-free weights with an empty prompt list start with no fetch.

Versions

54e8353clatestdefault
Automated release
7/2/2026