Classify Image (Zero Shot SigLIP)

@google1 VersionClassify

Scores each Image against free-form text candidate_labels with a SigLIP/CLIP model, builds one ranked result bounded by threshold and top_k, and projects it onto the connected output shape (top class, ranked list, label, pairs, or class id).

How it fits

Typical backends

Camera frames routed by which of a free-form label set they best match.

HTTP-served images scored against a candidate label list and published as JSON.

Video frames classified and overlaid for visual inspection.

Caveats

Parameter interactionhf_model_id and device are hot-swappable at runtime; each change reloads the model in place and reclaims previous GPU memory.
AccuracyScores are independent per label and do NOT sum to one across candidate_labels; an image can score high or low on every label at once.
AccuracyPhrasing each candidate_labels entry as a short descriptive caption separates scores better than bare nouns.
I/O contractEvery output shape is a projection of one ranked result, so the label-, score-, and id-bearing shapes agree class-for-class; id shapes carry each label's position in candidate_labels.
I/O contractthreshold drops labels below it and top_k caps the count in score-descending order before projection; scalar shapes carry only the single top label.
FallbackAn empty candidate_labels list, or an undecodable frame, emits the empty form of the resolved output arm without running the model; downstream must handle the zero-element case.

Versions

45633205latestdefault
Automated release
6/28/2026