Animate Portrait Audio (SadTalker)

@tencent1 VersionGenerate

Audio-driven talking head with SadTalker: consumes a driving AudioFrame speech stream, extracts 3D head-pose and expression motion, and renders a bound source portrait into a sequence of Image frames that form the talking-head video.

How it fits

Typical backends

Animate a portrait from a recorded speech file and publish the generated frames over HTTP for a downstream consumer.

Synthesize speech from text and drive the bound portrait with it, streaming the talking head to a browser viewer.

Caveats

I/O contractThis is an offline clip generator: a window of driving audio is buffered, then a full animated clip is produced and emitted frame by frame onto a single Image stream; an empty audio window yields no frames.
LatencyAudio is consumed and a clip is generated only after the window closes, so the first frames appear after a generation delay rather than in lockstep with incoming audio.
Hard constraintThe source portrait is bound once at startup from its configured image input; pointing it at a directory uses the first image found, and changing the animated subject requires a redeploy.
CompatibilityIncoming audio is resampled to 16 kHz mono before driving the model; the renderer runs on CPU but is far faster on a GPU, which the automatic device option prefers.
CostA larger rendered crop size and frame batch raise sharpness and throughput respectively at the cost of GPU memory, while suppressing head motion keeps the head near-static for portrait shots.

Versions

2df8e15bdefaultlatest
Automated release
6/27/2026