Detect Objects (Open Vocabulary Grounding DINO)

How it fits

Typical backends

Caveats

• I/O contractThe output is a oneof resolved at deploy time by the connected sink, and every shape is a projection of one per-detection record so the shapes agree detection-for-detection: [BoundingBox] is box geometry plus class, [Rectangle<Double>] is box geometry only, [DetectedClass] is class id plus confidence only, UInt64 is the detection count, and Image is the source frame with boxes drawn. There is no mask shape; Grounding DINO is detection-only.
• Hard constraintThe detection vocabulary is fixed at startup as an ordered list of phrases; an empty list detects nothing, every entry must be a non-empty phrase, and an empty entry aborts startup. The emitted class id is the zero-based index into that list, so its order defines the class id space.
• Parameter interactionA detection must pass both the per-box confidence floor and the phrase-match score floor before it is kept; a decoded phrase that maps to no prompt entry is discarded.
• AccuracyDecoded phrases can be substrings of a prompt entry and are matched back by longest-entry-first whole-phrase containment, so heavily overlapping or mutually-contained class phrases reduce mapping reliability.
• State lifetimeAll configuration keys, the class prompt, and the bound checkpoint are captured once at startup; runtime changes have no effect and require a redeploy. The checkpoint is loaded offline from the bound model artifact; nothing is fetched from the HuggingFace hub at runtime.
• FallbackAn empty/zero-size frame, or a frame whose inference raises (CUDA OOM, a transient fault), is logged and yields the empty-but-valid value for the resolved arm — an empty list, a zero count, or a blank overlay frame — so one bad frame never breaks the one-output-per-input contract. device `cuda` falls back to CPU when CUDA is unavailable.