Segment Image (Mask2Former)

How it fits

Typical backends

Caveats

• I/O contractThe output arm is a single downstream-resolved choice fixed at deploy time; the per-instance arms reproject one shared segment list so masks, boxes and class labels stay aligned, while the semantic head produces a per-pixel label map directly.
• Hard constraintA weights file is required and the checkpoint determines which heads are meaningful; task only chooses which head decodes the same forward pass, so a head the loaded weights were not trained for gives empty or degenerate output.
• Parameter interactionthreshold (detection score), mask_threshold (mask binarisation) and overlap_mask_area_threshold (visible area) apply only to the panoptic and instance heads; the semantic head labels every pixel and ignores them.
• Parameter interactionobject_classes keeps only the listed class ids after inference, max_segments caps the per-instance result count, and sort_by_area orders segments largest-first so the cap keeps the largest.
• State lifetimeThe model binding and all configuration are captured once at startup; changing any setting requires a redeploy.
• CostA GPU is strongly preferred; transformer segmentation on CPU is slow for live streams.