Separate Speakers (MossFormer2)

How it fits

Typical backends

Caveats

• I/O contractOutput is always 16 kHz mono regardless of input rate or channels; the input is mixed down to mono before inference, so add a resample stage downstream if the original rate must be restored.
• LimitationThe checkpoint separates a fixed two voices; a mixture with three or more concurrent speakers degrades noticeably and needs an upstream diarisation stage.
• OrderingThe per-speaker order in the output list is not stable across frames, so a downstream consumer cannot assume a given index is the same person from one tick to the next.
• AccuracyThe separated streams are not loudness-matched to each other or to the input; normalise each stream downstream if relative level matters.
• FallbackAn empty input frame is echoed back as a single-element list without running the network.
• State lifetimeThe model and the compute device are captured once at startup; a GPU is strongly preferred since CPU separation is impractical for live audio. The MossFormer2_SS_16K weights are fetched by the ClearVoice runtime on first deploy and cached, so the first call also pays the download and cold-kernel cost.
• CompatibilitySeparation is the only ClearVoice task here that fans one frame out to a list; denoise_audio_mossformer2 and denoise_audio_clearervoice_studio share the same runtime but emit a single cleaned AudioFrame, and `upscale_audio_mossformer2` is the super-resolution sibling. Pair each separated stream with transcribe_audio_faster_whisper downstream for per-speaker transcription.