Extract Text (TrOCR)

@microsoft1 VersionExtract

Recognises the text in a single cropped Image line with a TrOCR vision encoder-decoder and emits either a bare String transcript or a {text: String, confidence: Double} record pairing the transcript with the mean per-token recogniser score.

How it fits

Typical backends

Crop one text line from a page, recognise it, and post the transcript over HTTP.

Recognise lines from a cropped image region and summarise the running text with a text model.

Caveats

I/O contractThe output arm is fixed when the backend deploys: a text consumer pins the bare String transcript, a consumer that also needs a score pins the {text: String, confidence: Double} record whose `confidence` is the mean per-token recogniser score in `0..1`.
LimitationTrOCR recognises ONE line per call and does not detect or localise text, so feed it a cropped single line; a full multi-line page is collapsed to a single recognised line.
Parameter interactionnum_beams widens the search for accuracy at higher latency, and max_tokens bounds the recognised line length.
Hard constraintmodel must resolve to a TrOCR vision encoder-decoder bundle (the printed or handwritten checkpoint matching the input style); an encoder-only or unrelated checkpoint fails to load.
Fallbackdevice requesting CUDA resolves to CPU when no compatible accelerator is visible at startup; inference continues at CPU latency.
State lifetimeThe recogniser, the resolved output arm, and the device are built once at startup; changing them requires a redeploy.

Versions

253ed8aalatestdefault
Automated release
6/28/2026