Decompose Intrinsic Image (Marigold)

How it fits

Typical backends

Caveats

• I/O contractExactly one Image is emitted per frame — the chosen appearance layer; obtaining both albedo and material requires two instances configured for different layers.
• I/O contractWhen the material layer is selected the output packs BRDF properties per channel (red roughness, green metallicity, blue unused) and is not a natural-colour image — consumers must read the channels, not the appearance.
• CompatibilityThe configured checkpoint must be an appearance-decomposition Marigold IID model exposing both albedo and material targets; a lighting-decomposition checkpoint exposes different targets and fails the target check at startup.
• Parameter interactionThe denoising step count must suit the checkpoint — the default appearance model is tuned for a handful of steps, while older non-distilled heads need around ten — and averaging several diffusion passes per frame raises precision at proportional latency and memory cost.
• AccuracyInference runs at a capped longest side and the result is resampled back to the source size; configuring it to keep native resolution costs more time and memory.
• LatencyThe step count, ensemble count, and inference resolution can be changed at runtime and take effect on the next frame; the checkpoint and the selected layer are fixed at startup, so changing them requires a redeploy.
• FallbackA frame that fails inference (including a transient GPU out-of-memory) is logged and replaced by a black Image of the input size for that frame only, keeping the stream alive.
• LimitationThe inference backend is fixed at startup, so switching between CPU and GPU requires a redeploy; CPU diffusion inference is impractically slow.