Files
Enterprise-Knowledge-Base/internal/models/embedding
wizardchen 200c344065 feat(models): cap background embedding concurrency per model
Extend the per-model background concurrency governor to embedding, the
highest-volume background model call (document ingestion vectorises every
chunk, so a batch upload can burst the whole worker pool against one provider).

The wrapper sits innermost (below debug/langfuse) because BatchEmbedWithPool
fans a batch out into per-sub-batch BatchEmbed calls via the pooler, and the
pooler invokes BatchEmbed on whichever Embedder is threaded down as `model`;
sitting innermost routes those per-sub-batch provider round-trips through Gate
so the semaphore bounds real concurrent calls rather than one coarse per-doc
unit. Only background (asynq) calls are throttled; interactive query embedding
is never gated.
2026-07-09 20:40:51 +08:00
..