Komponenten

Analyze Image (vLLM VLM)

@vllm

Sends one or more images plus a String prompt to a vision-language model served by a local vLLM sidecar and returns the model's reply as a String. The image input is a oneof spanning a single image, an image-with-label, a batch, or labeled batch.

Upscale Audio (MossFormer2)

@modelscope

MossFormer2 speech super-resolution. Mixes each AudioFrame to mono, resamples up into a 48 kHz container, and runs MossFormer2_SR_48K to restore the high band lost to low-rate capture, emitting one mono 48 kHz AudioFrame per input.

Rerank Text (Ollama)

@ollama

LLM-as-reranker over a local Ollama runtime. Sends the query and documents from a RerankInput to a chat model, parses its JSON relevance scores, sorts descending, optionally truncates, and emits [RerankResult] with each document, score and original index.

Generate Image Caption (BLIP)

Captions each Image with the HuggingFace checkpoint bound at model (Florence-2 or a BLIP / BLIP-2 bundle) and emits either a plain String caption or a structured Caption record, depending on the downstream arm.

Segment Image (Ultralytics YOLOE)

@ultralytics

Open-vocabulary instance segmentation over a single Image. Masks instances with a YOLOE checkpoint, optionally steered by prompt_classes, then projects the result onto whatever the downstream connection requests, e.g. [(Segmentation, BoundingBox)].

Resample Audio Frame

@bleenco

Resamples each AudioFrame to a target rate while preserving the channel count, emitting an AudioFrame at the new rate. Use it to match the fixed input rate a downstream audio model or sink expects.

Denoise Audio (MossFormer2)

@modelscope

Full-band MossFormer2 speech denoiser. Mixes each AudioFrame to mono, resamples to 48 kHz, denoises with the MossFormer2_SE_48K checkpoint, and emits one mono 48 kHz AudioFrame per input. Pick for offline batch cleanup where enhancement quality outweighs latency.

Detect Landmarks (rtmlib)

@tauj

Detects every person in a single Image and emits their keypoints in one RTMPose pass — no separate detector or top-down crop stage — packaged per person as [[Landmark]].

Segment Image (HuggingFace)

@huggingface

Image segmentation on an Image from a Hub model id, auto-routing to a Transformers, ONNX or Ultralytics YOLO backend by repo contents. The sink resolves one of nine projections, from [(Segmentation, BoundingBox)] to a count.

Animate Face (LivePortrait)

@kuaishou

Real-time face reenactment with LivePortrait: transfers the head pose and expression of each driving Image frame onto a fixed source face via implicit-keypoint warping with stitching and retargeting, emitting one animated Image per frame.

Detect Objects (SAM 3)

@meta

Exhaustively detects every instance matching a SAM 3 concept phrase in each Image, running comma-separated phrases in order, and projects detections onto the connected output: boxes, geometry, classes, a count, or an overlay. Offline from the bound model.

Generate Video (HuggingFace)

@huggingface

Synthesises a short video clip from each String prompt using an in-process HuggingFace diffusers text-to-video pipeline; emits one Image per frame so the output stream forms the generated video.

Limit Message Rate

Throttles any stream t to a target rate in Hz with a token bucket, dropping surplus messages rather than buffering them, so a fast producer cannot overwhelm a slower consumer and no back-pressure is introduced.

Register Point Clouds (ICP)

@bleenco

Rigidly aligns a source PointCloud onto a target PointCloud with iterative closest point. The sink resolves the arm: a metrics record with the transform plus fitness and RMSE, a bare rigid transform, or the source cloud rebased into the target frame.

Compare Images (PSNR SSIM)

@opencv

Compares a candidate Image against a reference Image and emits full-reference quality metrics, either as a {psnr: Double, ssim: Double, mse: Double} record or as [Analytics.Metric].

List Length

Takes a list-typed stream [t] and emits its element count as UInt64, exactly once per input tick. Element identity and content are discarded; only cardinality reaches the output.

Flatten

Turns a list-typed stream [element_type] into a stream of its individual elements element_type, emitting one output tick per element. Purely structural: it removes one layer of list nesting and carries the element type through unchanged.

Pack Tuple

Packs N positional input streams, each carrying a type $pack_types, into a single tuple ($pack_types...), preserving wiring order with each element's type kept verbatim and no field names. Inverse of unpack_tuple.