Recognize Hand Gesture (MediaPipe)

How it fits

Typical backends

Caveats

• I/O contractEvery frame emits exactly one output. The gesture vocabulary is fixed by the bound bundle; on the list arm an empty list, and on the single arm a zero-confidence empty label, both mean no hand was found. A frame that cannot be processed does not degrade to that empty result: see the unprocessable-frame caveat.
• Hard constraintA frame MediaPipe cannot process (its image wrapper or recognition call returns an error) stops the worker with the full MediaPipe status and message, and a gesture category without a name stops it too. Nothing is substituted for the failed frame.
• Hard constraintThe worker fails at startup unless the bound model slot holds exactly one file ending in .task, and also fails at startup when MediaPipe cannot create a recognizer from that bundle. Both errors name the slot or bundle and the cause.
• LimitationOne gesture is named per detected hand, not a ranked set of alternatives for one hand - the recognizer's classification head emits a single winning category per hand. The list therefore holds at most num_hands entries, ordered by confidence.
• Parameter interactionnum_hands caps the hand detector, which picks which hands to keep by how confidently it locates them, not by how confidently their gesture is named. Lowering it therefore drops a hand the detector ranked lower even when that hand carried the higher gesture confidence, so the leading entry of the list can change with it. Set it to the most hands a frame should ever carry rather than tuning it down to shorten the list.
• AccuracyThe gesture labels come from the label map inside the bound .task bundle, so the vocabulary is whatever that bundle defines. Gestures the bundle does not name as a category come back under the bundle's own none label, which is a real classification rather than an error - read it as "a hand is present but holds no recognised gesture".
• LatencyInference runs on CPU. The task layer is not re-entrant, so concurrent frames are serialized through one recognizer and a slow frame delays the ones queued behind it. Per-frame time has not been measured.
• State lifetimeThe bundle bound at model is opened once at startup and the recognizer lives for the process. num_hands and the two hand confidence floors are fixed into the graph at that point and cannot change without a redeploy; only score_threshold is re-read per frame. Recognition runs in single-image mode, so no hand is tracked from one frame to the next.