The 000036 evaluator probes the rules table and runs two regexes for every
catalog file row; the first-level page of the 8k-file education source took
2.5-5s per query and the browser hung on 'reading first-level directory'.
- source_catalog_nodes gains a STORED generated ext column (000038)
- catalog_filter_context() resolves the connection's source-level allow/
exclude sets once; hot queries match node.ext against arrays (000039)
- filename_is_filtered stays the exact fallback, used only when the
connection has directory-scoped rules
- ListCatalogSources, ListCatalogNodes (current/subtree/total/items) and
RouteCatalogFiles fan-out adopt the fast path
Education root page: 4770ms -> 12ms; all-source counts: 54ms.
- filter rules gain mode ('exclude'|'allow'); nearest allow rule governs a
directory subtree, extensions outside it are filtered before fetch
- filename_extension() now requires a well-formed suffix, so dated names like
'CRA_V3.1手册_2024-07-31_en' no longer fake an extension; extensionless
files are never filtered
- API accepts {mode, extensions}; UI editor becomes grouped format chips with
keep/exclude modes (keep-list default: docs/text/scripts/images)
- migration 000036 + integration tests for allow, nesting and restore
Address two gaps in the first DingTalk-session UI:
- add an explicit admin-only '+ 新增钉钉知识库' action on the data-source
panel; it accepts an alidocs space URL, resolves the live name/root through
the authenticated sidecar, writes the inventory root atomically, upserts a
deterministic source connection, and enqueues its first catalog refresh
- read DingTalk's real QR canvas pixels rather than taking a DOM screenshot;
select by dark-pixel content, composite onto a white quiet zone, and render
at 320x320 so the dialog contains a large complete code, not a page thumbnail
The live refresh path resolves registered roots from the connection locator,
while existing name-based DingTalk sources remain compatible.
Routing targets move from env-only to a UI-managed binding (migration
000034 routing_kb_bindings, single active row):
- GET/PUT /api/v1/routing-bindings + /options: env defaults act as the
bootstrap fallback; a UI binding wins and takes effect immediately
(per-request resolution, no restart), and clearing falls back to env
- PUT validates against the connected WeKnora instance (real knowledge
base list via the scoped API key) and rejects same-KB bindings;
saves require the admin role
- route/catalog capabilities now resolve through the binding so both
annotation paths share one source of truth
- system settings > 服务配置 gains a 知识库绑定 panel: dropdown picker
(or raw ID when WeKnora is unreachable) showing bound name, origin
(UI vs env default) and last author
- contract tests: OpenAPI, handler matrix (fallback, validation,
same-KB rejection, capability resolution), frontend fetch/save layer
jobs.idempotency_key is globally unique, but test-poll:<pub>:<seq> and
test-validate:<pub>:<seq> were reused verbatim by every manual retry. The
first round's completed rows made ON CONFLICT DO NOTHING silently swallow
the next round's schedule SQL: a retried publication re-uploaded, polled
once, and then froze in pending/uploaded with no successor job and no
error. Keys and job ids are now derived from (publication, retry_count,
sequence) inside the schedule SQL so every retry round owns fresh
orchestration rows.
Annotation in 来源与目录 is now the only import decision surface:
- migration 000033: catalog_import_intents (durable, rescan-proof per-file
intent with locator snapshot + fetch lifecycle), source_items.policy_state,
planned publication state, ui_setting_revisions.ingestion_concurrency
- marking fans a folder decision out to per-file intents, materializes
versioned items, reuses already-verified local bytes (compare, don't pull),
and plans one publication per routed KB; blocked retires planned rows
while live publications stay untouched (需下架 shows in the KB diff)
- artifact.fetch worker lane (scripts/artifact_fetch.py): DingTalk export
tree/live box API, AList /api/fs/get direct download with SSO-redirect
fail-closed, PP ledger via locator hash; content-level dedupe on sha256
- 开始处理 promotes planned->pending inside the claim-time concurrency budget
from system settings; claims enforce it in SQL
- monitor overview rebuilt as knowledge-base diff: planned/needs_removal per
KB + fetch pipeline summary; frontend lanes 待入库/需下架 replace the
deprecated snapshot change-set confirmation
- spool: images (png/jpg/gif/bmp/webp) accepted, ppt/pptx rejected in favour
of PDF twins; DingTalk PPT resolves its converted PDF pair
姚章浩
zheromiku