fix(models): harden background concurrency governor

Follow-up hardening on the per-model background concurrency governor:

- limiter: heartbeat now refreshes the ZSET key's PEXPIRE alongside the
  lease score, so a semaphore that stays saturated with no slot turnover
  can't let the whole key expire (which would drop every lease and admit
  over the limit); reuse one poll timer instead of allocating time.After
  each iteration under contention.
- governor: extract the process-wide governor + Gate helper into the
  limiter package so chat and vlm share one limiter/limit without
  importing each other; add an in-process localLimiter for Lite mode.
- vlm: gate background multimodal (VLM) calls through the same governor;
  previously only chat was throttled.
- chat: ChatStream releases its slot when the consumer abandons the
  stream and cancels ctx (select on ctx.Done, drain inner), fixing a
  slot/goroutine leak.
- container/sync_task: install the local governor in Lite mode and tag
  SyncTaskExecutor runs as background; honour model.max_concurrency=0 as
  an explicit disable instead of falling back to the default.
- tests: cover the governor gating, local limiter, and stream release.
This commit is contained in:
wizardchen
2026-07-09 18:30:44 +08:00
committed by lyingbug
parent 63198c5d9f
commit 98ad41cda4
9 changed files with 440 additions and 61 deletions
+4 -1
View File
@@ -80,7 +80,10 @@ func (e *SyncTaskExecutor) Enqueue(task *asynq.Task, opts ...asynq.Option) (*asy
time.Sleep(delay)
}
ctx := context.Background()
// Tag as a background worker execution so the per-model concurrency
// governor throttles Lite-mode ingestion/enrichment LLM calls, mirroring
// the asynq backgroundTaskMiddleware in the Redis path.
ctx := types.WithBackgroundTask(context.Background())
start := time.Now()
logger.Infof(ctx, "[SyncTask] Executing task type=%s id=%s", task.Type(), taskID)