feat(worker): replace aggregate asynq concurrency with per-pool governance

Split the monolithic upstream worker budget into explicit core, post-process,
enrichment, maintenance, and shared elastic pools, expose live cluster capacity
on the runtime dashboard, and retire the legacy asynq.concurrency setting.
This commit is contained in:
wizardchen
2026-07-13 14:55:59 +08:00
committed by lyingbug
parent 5eefa70e6f
commit 07ba01ec93
27 changed files with 645 additions and 209 deletions
+22 -9
View File
@@ -716,22 +716,35 @@ DOCREADER_TRANSPORT=grpc
# WEKNORA_DOCUMENT_PROCESS_TIMEOUT=2h
# --- Async pipeline tuning (optional) -----------------------------------------
# Worker pool size for the PARSE asynq server (document parse / post-process /
# embedding / multimodal / graph / question — everything except wiki). Default
# 32 — raise it on machines that handle many concurrent uploads (default Go
# runtime.NumCPU() under-provisions for the I/O-bound document pipeline). Can
# also be set in the management UI under Settings → System settings
# (asynq.concurrency); UI changes require a process restart.
# WEKNORA_ASYNQ_CONCURRENCY=32
# Per-instance guaranteed worker capacity for document parse / manual reparse.
# Core and enrichment can additionally borrow from the shared elastic pool.
# WEKNORA_ASYNQ_CORE_CONCURRENCY=8
# Dedicated lightweight post-process/fan-out workers. Keeping this separate
# prevents completed parses from waiting behind long DocReader jobs.
# WEKNORA_ASYNQ_POSTPROCESS_CONCURRENCY=2
# Per-instance guaranteed capacity for summary, image, graph, and question
# enrichment tasks.
# WEKNORA_ASYNQ_ENRICHMENT_CONCURRENCY=12
# Per-instance capacity for datasource sync, batch operations, and cleanup.
# Maintenance is not eligible for shared burst capacity because its tasks can
# be long-running.
# WEKNORA_ASYNQ_MAINTENANCE_CONCURRENCY=4
# Elastic capacity shared by core parse and enrichment queues. Queue dequeue is
# atomic, so these workers safely supplement both dedicated pools.
# WEKNORA_ASYNQ_SHARED_CONCURRENCY=6
# Worker pool size for the DEDICATED wiki asynq server. The wiki generation
# pipeline runs on its own pool with its own concurrency budget, hard-isolated
# from the parse pool above: heavy document parsing can never starve wiki
# generation during upload storms, and a wiki burst can never starve
# user-facing parsing. Default 16. Wiki work is synthesis-LLM-bound, so size
# user-facing parsing. Default 8. Wiki work is synthesis-LLM-bound, so size
# this to your LLM provider's concurrency budget. Also settable in the UI
# (asynq.wiki_concurrency); requires a process restart.
# WEKNORA_WIKI_ASYNQ_CONCURRENCY=16
# WEKNORA_WIKI_ASYNQ_CONCURRENCY=8
# Default per-model concurrency cap for background tasks (document ingest /
# enrichment). Shared across replicas, keyed by model ID. Only affects