Files
Enterprise-Knowledge-Base/internal/models/vlm/concurrency_wrapper.go
T
wizardchen 98ad41cda4 fix(models): harden background concurrency governor
Follow-up hardening on the per-model background concurrency governor:

- limiter: heartbeat now refreshes the ZSET key's PEXPIRE alongside the
  lease score, so a semaphore that stays saturated with no slot turnover
  can't let the whole key expire (which would drop every lease and admit
  over the limit); reuse one poll timer instead of allocating time.After
  each iteration under contention.
- governor: extract the process-wide governor + Gate helper into the
  limiter package so chat and vlm share one limiter/limit without
  importing each other; add an in-process localLimiter for Lite mode.
- vlm: gate background multimodal (VLM) calls through the same governor;
  previously only chat was throttled.
- chat: ChatStream releases its slot when the consumer abandons the
  stream and cancels ctx (select on ctx.Done, drain inner), fixing a
  slot/goroutine leak.
- container/sync_task: install the local governor in Lite mode and tag
  SyncTaskExecutor runs as background; honour model.max_concurrency=0 as
  an explicit disable instead of falling back to the default.
- tests: cover the governor gating, local limiter, and stream release.
2026-07-09 20:40:51 +08:00

36 lines
1.2 KiB
Go

package vlm
import (
"context"
"github.com/Tencent/WeKnora/internal/models/limiter"
)
// Multimodal enrichment (image OCR / caption) is a high-volume, slow background
// stage that hits the same provider budget as chat. Like chat, it must be
// governed at the client layer so an image-heavy ingestion storm can't burst
// the whole worker pool against one VLM provider. Only background (asynq
// worker) calls are throttled — see limiter.Gate / types.IsBackgroundTask.
type concurrencyVLM struct {
inner VLM
}
func (w *concurrencyVLM) GetModelName() string { return w.inner.GetModelName() }
func (w *concurrencyVLM) GetModelID() string { return w.inner.GetModelID() }
func (w *concurrencyVLM) Predict(ctx context.Context, imgBytes [][]byte, prompt string) (string, error) {
release := limiter.Gate(ctx, w.inner.GetModelID())
defer release()
return w.inner.Predict(ctx, imgBytes, prompt)
}
// wrapVLMConcurrency installs the background concurrency governor as the
// outermost VLM decorator. Always applied; a cheap passthrough when no limiter
// is installed or the call is interactive.
func wrapVLMConcurrency(v VLM, err error) (VLM, error) {
if err != nil || v == nil {
return v, err
}
return &concurrencyVLM{inner: v}, nil
}