Reads each PDF in the bound directory and emits a oneof t[String, Image, Document, Document.Page] chosen by what it is wired to: a text consumer receives one String per document (the extracted text), an image consumer receives one Image per page (the rasterized page), a Document consumer receives one fully-structured document per file, a Document.Page consumer receives one structured page streamed as it renders. Use it to feed native PDFs into language backends, to rasterize pages for OCR when the source has no text layer, or to take a page apart into the blocks and typography it was set with.
Typical backends
Document Q&A — extract a PDF's text, answer over it with a local model, report as JSON.
Scanned-PDF OCR — rasterize each page to an Image and read it downstream when the source has no embedded text.
Caveats
- Parameter interactionThe output form follows the connected downstream: wiring a String consumer extracts document text, wiring an Image consumer rasterizes pages, wiring a Document consumer emits the whole structured document, wiring a Document.Page consumer streams one structured page per page. There is no mode switch.
- FallbackOn the text side, a scanned image-only PDF with no embedded text layer extracts to an empty string silently; wire the Image side and OCR the pages instead when the source is scanned.
- FallbackOnce the directory is exhausted, leaving
repeat off keeps the source open and idle rather than stopping; turning it on loops back to the first document. - Parameter interactionOn the image side,
dpi sets render resolution and first_page / last_page clamp which 1-based pages are rendered; last_page 0 renders through the final page. - Parameter interactionOn the text side,
document_context prepends a `Contents of document <filename> :` header to each emission. - I/O contractOn the Document and Document.Page arms a content block is a paragraph, not a text line. Consecutive lines the page set as one paragraph are joined into a single block whose text reads as running prose and whose box is the rectangle those lines cover together, so a consumer that rewords a block gets whole sentences and a column-wide box to set the new wording in. A line joins the one above it only when face, leading, left edge and the predecessor's right edge all agree and it opens no list marker, so a heading, a list item, and a column or table line each stay a block of their own. A word hyphenated across a line break is closed up.
- I/O contractA block states how the page set it, not merely what it said. Each line carries its own rectangle and its runs, and each run states the type size it was set in, the face name behind it, whether that face is bold or italic, the ink it was painted in, its rotation and whether a space followed it. A paragraph states the measured step between its baselines and the edge its lines were set against — left, right, centre or justified. A single line states neither, having no step and no second edge to have measured. A renderer downstream therefore sets the replacement wording the way the page set the original instead of guessing type from the size of a box.
- I/O contractA page states its width and height in points — the sheet the source page actually was — beside the pixel size of its raster and the dpi that raster was rendered at. The two are independent measurements of the same page, and the points are the exact one.
- LimitationThe directory is enumerated once at startup and not walked recursively, so nested folders and files dropped in later are not picked up.
- I/O contractA run set in a symbolic font is read through that font's own encoding first. Such a font declares no character set of its own, so its codes come back raw or lifted into the private-use area, and neither can be copied, searched or translated. Symbol's codes have a published meaning, so a run set in it is respelled into the characters the page was showing. A run in any other symbolic font is left out of the document — one such codepoint drops the whole run, since keeping the rest would state wording the page never carried — which leaves those pixels to the page raster, the only place they can be reproduced from.
- I/O contractA run set smaller than its line and sitting off that line's baseline is a superscript or a subscript, and is spelled as the character it is where Unicode has one: "I2C" arrives as "I²C". That way the term survives being reworded, translated or copied rather than flattening to a digit on the line. A run whose characters have no such form is left alone, since subscript letters barely exist in Unicode.
- State lifetimeAll knobs bind once at startup, so changing any of them requires a redeploy.
