Components

Estimate Object Movement

@bleenco

Reduces a dense OpticalFlow tensor against [Mask] into one [Vector2d] per object, carrying the mask centroid as position and the thresholded mean per-mask flow as orientation, ready for smoothing or overlay.

Extract Text (Florence 2)

@microsoft

Transcribes text in each Image with Microsoft Florence-2, emitting either a plain String transcript or per-region records with a Polygon quad or Rectangle box and the recognised text.

Input Video URL (GStreamer)

@gstreamer

Opens a remote url or explicit gst_backend pipeline via GStreamer and emits VideoFrame chunks. drop_frames controls back-pressure; retry_delay_ms governs reconnect cadence. Use for custom GStreamer caps or hardware-accelerated decode.

Label COCO MPII Body Landmarks

@bleenco

Reindexes a flat [Landmark] body keypoint list into the 22-joint Landmarks2d.Human22 schema, interpolating neck, thorax, spine, and (COCO mode) head so COCO-17 or Halpe-26 detectors can both feed Human22 consumers.

Detect Object Containment

@bleenco

Polygon containment gate for Rectangle; emits Bool true when the anchor point (foot or centre, per anchor_mode) lies strictly inside the polygon defined in roi. Fast single-bbox in-zone check with no ML model.

Output VTT File

Renders each upstream [Subtitle] list to a WebVTT .vtt file under output_path, writing the WEBVTT header and formatting cue timings as HH:MM:SS.mmm, with the file named from name_template plus a 6-digit index.

Segment Image (RF DETR)

@roboflow

Transformer instance segmentation on an Image with an RF-DETR-Seg weights file. The sink resolves the projection: mask-plus-box pairs, masks, boxes, parallel arrays, a merged Mask, a UInt64 count, or a colourised label Image.

Segment Referring Expression (Florence 2)

@microsoft

Referring-expression segmentation with Florence-2: takes an Image and a String phrase naming one object, and emits the referent's contours as [Polygon], rasterised [Mask], or [Segmentation], resolved by the sink.

Output Browser Stream

Publishes a variadic mix of Image, VideoFrame, and AudioFrame inputs as separate live WebRTC tracks to browser subscribers, with optional wall-clock pacing, so an analytics backend can show a real-time preview in a browser tab.

Upscale Image (DiffBIR)

@xpixelgroup

Restores a degraded Image with the two-stage DiffBIR model — a SwinIR restorer feeds an IRControlNet-guided diffusion prior — and emits a super-resolved Image at outscale. Tiling bounds VRAM; min-size gates skip small frames.

Output Audio File

Encodes an AudioFrame stream into a chosen container and writes it as a set of rolling files on disk, starting a new segment on a fixed time interval and keeping only the most recent ones, so a generator or recorder leaves a bounded audio archive.

Gate

Streams a value-exact subset of any t stream out as bare t, only on the ticks a firing policy selects, so a costly downstream stage runs at a reduced rate. Each fire emits its read window as that many ordered t messages: an omega fan-out, never a list.

Upscale Audio (FLowHigh)

@korea_prml

Reconstructs the high band of a band-limited AudioFrame in a single flow-matching step with the FLowHigh model and a BigVGAN vocoder, emitting a 48 kHz mono AudioFrame. Output rate and layout are fixed.

Translate Audio (Whisper)

@openai

Resamples each AudioFrame to 16 kHz and runs Whisper's speech-to-English translate task, emitting an English String (empty on silence). The output is always English; the checkpoint is chosen by model_size.

Deblur Image (NAFNet)

@megviiresearch

NAFNet restorer that cleans an Image with a task-specific checkpoint (motion deblur, general frame restoration, or real-camera denoise) and emits a restored RGB Image at the same resolution. Use when blur or noise hurts downstream accuracy.

Analyze Face (FaceXFormer)

@kartiknarayan

Unified face analysis: takes an Image plus per-face [BoundingBox], runs one FaceXFormer pass per crop, and emits a downstream-resolved oneof of landmarks, face-parse segmentation, part masks, a colorized label map, demographic [Classification], or metrics.

Shuffle

Per-tick group-by over a list of key/value pairs [(k, v)], collapsing each input list into [(k, [v])] — one entry per distinct key carrying the list of all its values. The output entry count equals the number of distinct keys in that list.

Recognize Named Entities NER (HuggingFace)

@huggingface

Runs zero-shot GLiNER named-entity recognition on each String and emits the spans that clear a confidence floor as either anonymous record spans or [NamedEntity] carriers, depending on the downstream connection.