Answer Question (HuggingFace)

How it fits

Typical backends

Caveats

• I/O contractThe first input is the question as a String and the second is the context String (the passage the answer is extracted from). The answer is always a span copied verbatim from the context, never generated; the downstream sink selects whether the output is the answer String or a QAAnswer record (`{answer, start, end, score}`) where `start` / `end` are character offsets into the context.
• Hard constraintThe reader has a fixed position-embedding budget (512 tokens for the DeBERTa-v3 / BERT default). A context longer than max_input_tokens is tokenised into overlapping sliding windows of doc_stride overlap so a long context still answers; max_input_tokens is clamped down to that real budget so it can never overflow.
• Parameter interactionmax_answer_len caps the answer span length in tokens and n_best widens the start/end span search per window. A larger doc_stride keeps more overlap (an answer near a window boundary is less likely to be split) at the cost of more windows per call.
• FallbackAn empty question, an empty context, a context from which the reader finds no answer (the SQuAD2.0 unanswerable signal), or an inference failure on a single message all emit one safe value -- an empty answer String or an empty-text QAAnswer record -- rather than crashing or dropping; every input emits exactly one output.
• Latencymodel is hot-swappable at runtime and triggers a reload (and a GPU memory reclaim on CUDA) on the next call, producing a one-call latency spike. device is captured once at startup.