Triangulate 3D Points (Depth)

@bleenco1 VersionCalculate

Monocular depth unprojection (NOT multi-view triangulation). Bilinearly samples a MetricDepthMap at each [Point<Double>] and inverse-pinhole back-projects through a CameraIntrinsics into camera-frame 3D.

How it fits

Typical backends

Unproject 2D points into camera-frame 3D and print the results for inspection. The three typed inputs (a 2D point list, a metric depth map, and camera intrinsics) feed this component, whose 3D point list is logged. Connecting a plain [Point3d<Double>] sink selects the strict arm, which asserts every point has valid depth.

Publish the recovered 3D coordinates over HTTP for a downstream consumer instead of logging them. Connecting a [Maybe<Point3d<Double>>] sink selects the optional arm, which keeps one entry per input point and so stays index-aligned with the input even when some points lack valid depth.

Caveats

CompatibilityDespite the historical name this is single-camera depth back-projection, NOT multi-view triangulation: it recovers each point's depth from one MetricDepthMap rather than intersecting rays from several cameras.
CompatibilityOnly a pinhole projection is applied; wide-angle and fish-eye lenses must be undistorted upstream before back-projection, otherwise the recovered coordinates are warped.
I/O contractThe output is a oneof resolved once from the connected sink: a strict [Point3d<Double>] asserts every input point has valid depth, while a [Maybe<Point3d<Double>>] emits one entry per input point (Some when liftable, Nothing when vetoed) so positional alignment with the input is always preserved.
Hard constraintThe focal lengths and principal point come entirely from the CameraIntrinsics message, not from configuration; intrinsics whose principal point does not match the camera bias the result without any error.
AccuracyBilinear sampling is strict: if any of the four contributing neighbour depths is non-finite or not strictly positive, the whole point is vetoed rather than interpolated from the valid neighbours.
FallbackA malformed depth frame (wrong rank, or a shape that disagrees with its data size) or intrinsics with a non-finite or non-positive focal length emits the resolved arm's safe value for that frame (an empty list, or all-Nothing) and logs, so the deployment keeps serving instead of aborting.

Versions

8bd3367blatestdefault
Automated release
6/25/2026