Rerank Text (HuggingFace)

How it fits

Typical backends

Caveats

• I/O contractEach RerankResult keeps the document's position in the original RerankInput document list as its index, preserved across the sort and any truncation, so downstream consumers can map results back to inputs.
• FallbackAn empty query or an empty document list returns an empty [RerankResult] with no error rather than failing the message, and a fault inside scoring is logged and degraded to that same empty list so every input emits exactly one output.
• Parameter interactiontop_n zero keeps every reranked document; a positive value truncates to that many after the descending sort, so it only ever drops the lowest-scoring tail.
• Parameter interactionnormalize maps the raw cross-encoder logits through a sigmoid into [0, 1] before sorting; this changes only the reported score scale, never the ranking order. max_length truncates each (query, document) pair and batch_size sets how many pairs are scored per forward pass.
• Hard constraintThe checkpoint named by model must be a cross-encoder reranker that loads under `AutoModelForSequenceClassification` (e.g. the BGE reranker family); a bi-encoder or other architecture loads but produces meaningless scores. A gated or private repo additionally needs an access token via hf_token.
• Latencymodel is hot-swappable at runtime and triggers a model reload (and a GPU memory reclaim on CUDA) on the next call, producing a one-call latency spike; the previous model keeps serving if the new id fails to load.
• AccuracyScores rank relevance within a single invocation and are not calibrated for comparison across separate queries.
• State lifetimedevice and hf_token are captured once at startup; the model is loaded once and reused across messages, while model, top_n, max_length, batch_size and normalize are re-read on each call.