Components

Estimate Camera Gravity (GeoCalib)

@cvg

Single-view calibration with GeoCalib over each Image, emitting the full {intrinsics, gravity, roll, pitch} record or, on demand, just the CameraIntrinsics, the unit gravity Point3d, or the {roll, pitch} tilt in radians.

Track Points (AllTracker)

@aharley

Long-range point tracker (AllTracker) over an Image stream, with a resolved oneof output. The sparse path emits one TrackedPoint per chosen point (id, position, trajectory, visible) at a fixed query frame; the dense paths emit OpticalFlow or Mask.

Extract Text (vLLM)

@vllm

Sends each Image to an OCR vision-language model served by a local vLLM sidecar over the OpenAI-compatible chat endpoint, then emits a joined String transcript or per-block regions with Polygon/Rectangle bounds and confidence.

Animate Face (face vid2vid)

@kaiwenzheng

Reenacts a fixed source face with the head pose and expression of each incoming driving Image frame using the face-vid2vid keypoint-and-warp stack, emitting one animated Image per frame so a downstream Image stream becomes the animated video.

Window

Gathers consecutive messages of any stream t into fixed-size lists [t] of size elements, emitting one list every step arrivals. The count-based stream batcher and categorical dual of flatten, which maps [t] back to t.

Generate Text (vLLM)

@vllm

Drives a chat turn against a colocated vLLM service from a user and system String, streaming the reply as String chunks and closing with a StreamEnd; supports history and parallel stateless turns. Prefer generate_text_sglang for SGLang.

Segment Foreground (U2 Net)

@xuebinqin

Salient-object foreground segmentation on an Image with U2-Net. The single most prominent subject is isolated and the sink resolves the arm: (Segmentation, BoundingBox), (Mask, DetectedClass), a [Mask], or a colourised foreground Image.

Detect Objects (Triton)

@nvidia

Letterboxes and normalizes each Image, runs a Triton-hosted detector, NMS-filters the raw tensors into one per-detection record, then projects that record onto the connected output shape: boxes, geometry, labels, a count, or an overlay.

Extract Text (SGLang)

@lmsys

Sends each Image to an OCR vision-language model served by a local SGLang sidecar over the OpenAI-compatible chat endpoint, then emits a joined String transcript or per-block regions with Polygon/Rectangle bounds and confidence.

Classify Audio Event (Zero Shot CLAP)

@laion

Embeds each AudioFrame and a runtime-supplied label set into one CLAP space and ranks the labels by audio-text similarity, emitting all labels as [Classification], the single best as Classification, or the raw scores as [Double].

Transform

Builds a runnable component from code you paste inline, in Python or C++, at deploy time with no separate publish step. The pasted code uses the standard component worker SDK; the input $as and output $bs are independent typevars the surrounding graph fixes on each side, so one step adapts whatever flows in into whatever the code returns.

Transcribe Audio (Faster Whisper)

@systran

faster-whisper transcriber. Resamples each AudioFrame to 16 kHz mono and decodes Whisper with optional voice-activity gating, emitting oneof t[String, [Subtitle], [{segment: Subtitle, confidence: Double}]] chosen by the connected downstream.

Estimate Optical Flow (PTLFlow)

@hmorimitsu

In-process dense optical flow via any ptlflow-zoo model: takes Image frames and emits per-pixel OpticalFlow against an earlier frame at source resolution. The model is chosen at deploy, real-time to top-accuracy.

Deblur Image (Restormer)

@swz30

Transformer restorer for an Image that runs a per-task checkpoint — motion or defocus deblur, real / Gaussian denoise, or deraining — and emits a restored RGB Image at the original resolution. Use when the degradation matches one of the task variants.

Transcribe Audio (Canary Qwen)

@nvidia

NVIDIA Canary-Qwen-2.5B (NeMo SALM) transcriber. Resamples each AudioFrame to 16 kHz mono, transcribes it in fixed-length windows, and emits a single joined String. English only, with punctuation and capitalization.

Output TOML File

Serialises each upstream record (generic { $ns: $ts... }) to a TOML document under output_path, mapping records to tables, lists to arrays, and scalars to their TOML counterparts, one file per call named from name_template.

Summarize Tracked PPE

Per-person PPE compliance summariser. Fuses two-class [[Double]] probabilities, [(Image, Double)] body-part crops with confidence, and [UInt64] track ids into a confidence-weighted per-person history, emitting a status tuple or event per track.

Input SRT File (SubRip Text)

Parses SubRip subtitle files from the bound directory and emits the whole file as one [Subtitle] list per document, optionally looping. Use it for offline subtitle replay or comparing recognised speech against ground-truth captions.