Docs
Visualize Expression avatar

Visualize Expression

1 version
Open in App

Visualize Expression

General-purpose image transformation/effects worker. It evaluates a Python expression that can use NumPy (np), OpenCV (cv2), and scikit-image (sk) on the incoming messages and always returns an Image.

Use This When

  • You want to quickly prototype image operations without writing a custom worker
  • You want to combine NumPy/OpenCV/scikit-image primitives flexibly in a pipeline

Caveats

  • Expression result must be convertible to a NumPy array.
  • Output array has to have correct shape (2D or H×W×C)
  • The evaluator does not support lambda expressions (NotImplementedError); use statements or comprehensions instead.
  • Builtins like getattr are not available inside expressions; use provided helpers (e.g., bbox_class(bb)).
  • String literal .format(...) calls are blocked; prefer f‑strings inside expressions.
  • No disk or network I/O; keep expressions compute‑bound.
  • Avoid per‑pixel Python loops; prefer vectorized operations for consistent performance.
  • When mixing RGB/BGR sources, set output_color_model or convert explicitly in the expression; defaults assume BGR semantics.

What It Does

  • Binds inputs as x0, x1, x2, ...
  • Evaluates expression with np, cv2, and sk available
  • Uses the expression result as the output image after strict validation (no auto-fixing)
  • Non-image inputs are exposed directly by their named types:
    • Tensor<Float> → NumPy array with shape = tensor.shape
    • Point<Double> → raw object (p.x, p.y)
    • Rectangle<Double> → raw object (r.top_left.x/y, r.bottom_right.x/y)
    • BoundingBox → raw object (use helper bbox_class(bb) → id, confidence; rectangle via bb.rectangle.top_left.x/y, bb.rectangle.bottom_right.x/y)
    • Typed lists (Points/Rectangles/BBoxes) → Python lists of raw elements; lists of Images/Tensors are not supported

Expression String

  • Expression is a single String; the worker evaluates that string.
  • Newlines are allowed inside the string (it's still one string), or use semicolons to separate statements on one line.
  • The final value of the string must be a uint8 NumPy array with a valid image shape and must match output_color_model (or AUTO).

Examples (as single strings)

  • GRAY, one line: "cv2.Canny(x0, 100, 200)"
  • GRAY, multiple statements with newline: "g = cv2.cvtColor(x0, cv2.COLOR_BGR2GRAY)\ncv2.Canny(g, 100, 200)"
  • GRAY, multiple statements on one line (semicolon): "g = cv2.cvtColor(x0, cv2.COLOR_BGR2GRAY); cv2.Canny(g, 100, 200)"
  • BGR overlay, one line: "cv2.addWeighted(x0, 0.7, cv2.cvtColor(cv2.Canny(x0,50,150), cv2.COLOR_GRAY2BGR), 0.3, 0)"

Expression Environment

Available symbols inside expression:

  • np → NumPy
  • cv2 → OpenCV
  • sk → scikit-image
  • plt → Matplotlib pyplot (Agg backend; headless)
  • fig2bgr() → capture current Matplotlib figure to BGR uint8 ndarray
  • x0, x1, x2, ... → ordered inputs:
    • Images as BGR ndarray (shape (H, W, 3), dtype uint8)
    • Tensors as ndarray
    • Points as raw objects (p.x, p.y)
    • Rectangles as raw objects (r.top_left.x/y, r.bottom_right.x/y)
    • BBoxes as raw objects (bb.class.id/confidence, bb.rectangle.top_left.x/y, bb.rectangle.bottom_right.x/y)

Result Handling

  • Expression must return a NumPy array with dtype=uint8.
  • Accepted shapes: (H, W) or (H, W, C) with C ∈ {1, 3, 4}.
  • Output must already match the selected output_color_model:
    • GRAY → (H, W) or (H, W, 1)
    • BGR/RGB → (H, W, 3)
    • BGRA/RGBA → (H, W, 4)
  • Any mismatch raises an error; the worker does not convert or normalize.

Examples

  1. Edge detection (OpenCV Canny) on first image
expression: cv2.Canny(x0, 100, 200)
output_color_model: GRAY
  1. Sobel magnitude (scikit‑image) as grayscale
expression: sk.filters.sobel(cv2.cvtColor(x0, cv2.COLOR_BGR2GRAY))
output_color_model: GRAY
  1. Heatmap overlay (blend edges into color)
expression: cv2.addWeighted(x0, 0.7, cv2.cvtColor(cv2.Canny(x0,50,150), cv2.COLOR_GRAY2BGR), 0.3, 0)
output_color_model: auto
  1. Plot with Matplotlib and return the figure
expression: plt.figure(figsize=(4,3)); plt.plot([0,1,2],[0,1,0],'r-'); plt.title('Demo'); fig2bgr()
output_color_model: BGR
  1. Draw rectangle (second input is Rectangle<Double>)
expression: tl=(int(x1.top_left.x), int(x1.top_left.y)); br=(int(x1.bottom_right.x), int(x1.bottom_right.y)); cv2.rectangle(x0, tl, br, (0,255,0), 2); x0
output_color_model: BGR
  1. Draw bbox + label (second input is BoundingBox)
expression: tl=(int(x1.rectangle.top_left.x), int(x1.rectangle.top_left.y)); br=(int(x1.rectangle.bottom_right.x), int(x1.rectangle.bottom_right.y)); cv2.rectangle(x0, tl, br, (255,0,0), 2); c=bbox_class(x1); cv2.putText(x0, f"id {c.id} ({c.confidence:.2f})", tl, cv2.FONT_HERSHEY_SIMPLEX, 0.6, (255,0,0), 2); x0
output_color_model: BGR
  1. Draw all bounding boxes from a list ([BoundingBox])
expression: [(cv2.rectangle(x0, (int(bb.rectangle.top_left.x), int(bb.rectangle.top_left.y)), (int(bb.rectangle.bottom_right.x), int(bb.rectangle.bottom_right.y)), (255,0,0), 2), cv2.putText(x0, f"id {bbox_class(bb).id} ({bbox_class(bb).confidence:.2f})", (int(bb.rectangle.top_left.x), int(bb.rectangle.top_left.y)), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (255,0,0), 2)) for bb in x1]; x0
output_color_model: BGR
  1. Mark points (second input is [Point<Double>])
expression: [cv2.circle(x0, (int(p.x), int(p.y)), 4, (0,0,255), -1) for p in x1]; x0
output_color_model: BGR

Versions

  • cb7bb560latestdefaultlinux/amd64

    Automated release