Extract Text Embedding (HuggingFace)

How it fits

Typical backends

Caveats

• Hard constraintmodel is loaded directly with AutoModel + AutoTokenizer, so it must be a transformers-loadable encoder repo. The embedding dimension is fixed by the checkpoint, so switching model produces vectors of a different width that an existing index will reject.
• Parameter interactionpooling must match the checkpoint's model card: `cls` for BGE / XLM-RoBERTa dense heads (the default, correct for `BAAI/bge-m3`), `mean` for sentence-transformers / E5 / GTE / MiniLM models. The wrong pooling silently degrades vector quality rather than erroring.
• Parameter interactionquery_prefix should carry the checkpoint's retrieval instruction on the query side only (BGE's `Represent this sentence...`, or E5's `query: `) and stay empty on the passage side; mismatching the two sides degrades retrieval quality.
• I/O contractAn empty String short-circuits to a zero-length Embedding without invoking the model. Feeding that into a similarity stage breaks the math, so filter empty inputs upstream. Every input emits exactly one output.
• Parameter interactionnormalize produces unit-length vectors so cosine similarity equals the inner product; keep it consistent across the index and the queries.
• Latencymodel is hot-swappable at runtime and triggers a model reload (and a GPU memory reclaim on CUDA) on the next call, producing a one-call latency spike. device, pooling, normalize and query_prefix are read once at startup and require a redeploy.
• Fallbackdevice requesting CUDA resolves to CPU when no compatible accelerator is visible at startup; smaller checkpoints run acceptably on CPU.