fix(models): harden background concurrency governor
Follow-up hardening on the per-model background concurrency governor: - limiter: heartbeat now refreshes the ZSET key's PEXPIRE alongside the lease score, so a semaphore that stays saturated with no slot turnover can't let the whole key expire (which would drop every lease and admit over the limit); reuse one poll timer instead of allocating time.After each iteration under contention. - governor: extract the process-wide governor + Gate helper into the limiter package so chat and vlm share one limiter/limit without importing each other; add an in-process localLimiter for Lite mode. - vlm: gate background multimodal (VLM) calls through the same governor; previously only chat was throttled. - chat: ChatStream releases its slot when the consumer abandons the stream and cancels ctx (select on ctx.Done, drain inner), fixing a slot/goroutine leak. - container/sync_task: install the local governor in Lite mode and tag SyncTaskExecutor runs as background; honour model.max_concurrency=0 as an explicit disable instead of falling back to the default. - tests: cover the governor gating, local limiter, and stream release.
This commit is contained in:
@@ -80,7 +80,10 @@ func (e *SyncTaskExecutor) Enqueue(task *asynq.Task, opts ...asynq.Option) (*asy
|
||||
time.Sleep(delay)
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
// Tag as a background worker execution so the per-model concurrency
|
||||
// governor throttles Lite-mode ingestion/enrichment LLM calls, mirroring
|
||||
// the asynq backgroundTaskMiddleware in the Redis path.
|
||||
ctx := types.WithBackgroundTask(context.Background())
|
||||
start := time.Now()
|
||||
logger.Infof(ctx, "[SyncTask] Executing task type=%s id=%s", task.Type(), taskID)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user