Commit Graph

1517 Commits

Author SHA1 Message Date
dependabot[bot] 484d08da36 chore(deps): bump github.com/google/jsonschema-go from 0.4.2 to 0.4.3
Bumps [github.com/google/jsonschema-go](https://github.com/google/jsonschema-go) from 0.4.2 to 0.4.3.
- [Release notes](https://github.com/google/jsonschema-go/releases)
- [Commits](https://github.com/google/jsonschema-go/compare/v0.4.2...0.4.3)

---
updated-dependencies:
- dependency-name: github.com/google/jsonschema-go
  dependency-version: 0.4.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-09 12:48:53 +08:00
wizardchen 1c643dc11f chore(dependabot): expand coverage beyond /cli
Add weekly Dependabot updates for the root and /client Go modules,
the frontend and miniprogram npm packages, and the docreader Python
package. Existing /cli gomod and github-actions entries are kept.
Semver-major bumps remain ignored to reduce noise.
2026-05-09 12:45:40 +08:00
dependabot[bot] 24f89e7777 chore(deps): bump github.com/mattn/go-runewidth in /cli
Bumps [github.com/mattn/go-runewidth](https://github.com/mattn/go-runewidth) from 0.0.16 to 0.0.23.
- [Commits](https://github.com/mattn/go-runewidth/compare/v0.0.16...v0.0.23)

---
updated-dependencies:
- dependency-name: github.com/mattn/go-runewidth
  dependency-version: 0.0.23
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-09 12:45:04 +08:00
dependabot[bot] 80703b6551 chore(deps): bump actions/setup-go from 5 to 6
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-09 12:44:23 +08:00
dependabot[bot] 5a15cdb197 chore(deps): bump docker/build-push-action from 3 to 7
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3 to 7.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v3...v7)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-09 12:43:52 +08:00
dependabot[bot] 7f07ea80d6 chore(deps): bump actions/download-artifact from 4 to 8
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 8.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v8)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-09 12:43:39 +08:00
dependabot[bot] 3dd42c1cec chore(deps): bump docker/metadata-action from 5 to 6
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5 to 6.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-09 12:43:27 +08:00
dependabot[bot] a364b93e28 chore(deps): bump actions/setup-node from 4 to 6
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4 to 6.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-09 12:43:11 +08:00
dependabot[bot] 07ce223054 chore(deps): bump github.com/spf13/cobra from 1.8.1 to 1.10.2 in /cli
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.8.1 to 1.10.2.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.8.1...v1.10.2)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-version: 1.10.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-09 12:43:00 +08:00
dependabot[bot] 3b909964f0 chore(deps): bump github.com/mattn/go-isatty in /cli
Bumps [github.com/mattn/go-isatty](https://github.com/mattn/go-isatty) from 0.0.20 to 0.0.22.
- [Commits](https://github.com/mattn/go-isatty/compare/v0.0.20...v0.0.22)

---
updated-dependencies:
- dependency-name: github.com/mattn/go-isatty
  dependency-version: 0.0.22
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-09 12:42:49 +08:00
nullkey bb592a59a6 feat(cli): contract test suite + dependabot (PR-8)
cli/acceptance/contract/:
  envelope_test.go    — 16 envelope golden cases (9 commands × {success/error
                        variants}; 3 cases dropped with rationale: doctor.success
                        non-offline has unstable timing detail; auth_login.* needs
                        stdin/keyring scaffold deferred to v0.2; context_use.error
                        needs leaf-local --json deferred to follow-up)
  errorcodes_test.go  — single-direction AST scan of cli/cmd/ extracting first
                        arg of cmdutil.NewError / cmdutil.Wrapf calls;
                        ClassifyHTTPError dynamic-classify bridged via
                        cmdutil.ClassifyHTTPErrorOutputs() per spec §4.3.
  testdata/envelopes/ — 16 JSON golden files

helpers_test.go (PR-6 scaffold) extended:
  runCmd now wires cobra Out/Err sinks (version uses c.OutOrStdout) AND
  replicates cmd.Execute()'s error-envelope path so error-case goldens are
  populated. Without this, every error scenario's golden was 0 bytes.

cli/cmd/root.go: mapCobraError → MapCobraError, wantsJSONOutput → WantsJSONOutput
                 (exported so the contract test helper can replicate Execute()'s
                 envelope-printing path without calling Execute() itself).
                 root_test.go updated to use new exported names.

.github/dependabot.yml (新增):gomod /cli + github-actions weekly,gh-style
                              ignore semver-major to avoid noise. Open-source
                              dependency safety,independent of release cadence.

v0.1 不发布到任何分发平台 (release infra 推迟到发布窗口 milestone)。
2026-05-09 12:18:01 +08:00
nullkey cf84bf2a38 feat(cli): add whoami / doctor / kb / context commands (PR-7)
5 new leaf commands wired into the root tree:
  whoami      — simplified `auth status` (user_id + tenant_id only)
  doctor      — 4-item self-check (base_url / auth / server_version / cred_storage)
                with --offline / --no-cache flags + skip cascade + summary.all_passed
                防 agent 看到 envelope.ok=true 误判命令整体 success
  kb list     — list KBs (default updated_at desc; 0 KB → "(no knowledge bases)")
                tabwriter 4-col (ID/NAME/DOCS/UPDATED), display-width truncation
  kb get      — show single KB details (KEY: VALUE, suppress empty fields)
  context use — switch default context (writes config.current_context),
                带 levenshtein distance ≤ 2 的 did-you-mean hint

Each command uses the v0.0 narrow Service interface pattern (testable via
fakes), agent.SetAgentHelp for AI-friendly hints, and ClassifyHTTPError
for stable error code mapping.

cmdutil/errors.go: 新增 CodeLocalContextNotFound for `context use`,加入 AllCodes() 注册集.

cli/cmd/root.go: NewRootCmd 改为 exported (acceptance/contract 测试需要),
                 注册 4 个新命令 + 1 parent group; root_test.go 跟随更新.
2026-05-09 12:18:01 +08:00
nullkey 92d78dea72 feat(cli): add internal/{text,compat} packages and acceptance scaffold (PR-6)
text/      gh-style display-width string helpers (Truncate / FuzzyAgo / Pluralize)
compat/    server-CLI version skew probe (kubectl) + 24h TTL cache (gh internal/update)
acceptance/contract/  helpers_test.go for in-process cobra exec + golden assert
                      (uses iostreams.SetForTest singleton replacement; no t.Parallel)
cmdutil/errors.go: AllCodes() + ClassifyHTTPErrorOutputs() bridge for the
                   contract test suite (no behavior change in error handling)

go-runewidth promoted from indirect to direct (used by text.Truncate for
display-width-aware CJK truncation).

acceptance/contract/helpers_test.go forward-references cmd.NewRootCmd which
is exported in PR-7 — that file does not compile until then by design.
2026-05-09 12:18:01 +08:00
wizardchen b7388f10e1 fix(wiki-ingest): reconcile summary links and log feed when reduce LLM fails
When a doc's entity/concept page generation hit a transient LLM error in
the reduce phase, the page never got written, but the doc's summary page
(generated in parallel with the slug list baked into its content) was
already persisted with [[entity/foo|name]] links pointing at the missing
page. The wiki log feed also surfaced the failed slugs as clickable
entries that 404'd. cleanDeadLinks only ran for retract batches, so the
debris persisted indefinitely.

reduceSlugUpdates now reports addition-path failures back to the batch
driver, which collects them and:

  - rewrites this batch's summary pages, replacing dead [[slug|display]]
    refs with plain display text (or a humanized slug tail);
  - filters dead slugs out of docResults[].Pages before the wiki log
    entries are flushed;
  - broadens the cleanDeadLinks trigger to fire whenever any page was
    touched or any slug failed, as a long-tail safety net.

Pure text replacement; no extra LLM calls. Failed slugs are picked up
naturally on the next ingest of the same document via the existing slug-
continuity rules, so the system self-heals without manual reingest.
2026-05-09 02:10:15 +08:00
wizardchen 21a4259fc5 feat(wiki/graph): 优化操作帮助入口与图例视觉
- 把"操作帮助"按钮从图例底部挪到搜索框右侧,使用 placeholder
  色的图标按钮,hover 高亮主题色,更易被发现且不与图例操作项
  混在一起
- 状态卡片(全库概览/当前焦点)简化排版:去掉大写字母+字间距,
  主标题降字号去粗体,分隔改为 1px 虚线,整体视觉更轻
- 抽屉打开时图例自动向左平移 480px(抽屉宽度),避免被覆盖,
  使用与 TDesign drawer 一致的 cubic-bezier 缓动
2026-05-09 00:49:28 +08:00
wizardchen fc6f160eff fix(retriever/doris): code review cleanup
针对 4cce6f2e(接入 Apache Doris)的 code review 修复,主要修正若干阻断性
问题与可读性问题,并剔除不应进入主仓的本地工作流文件。

阻断性修复:
- docker-compose: Doris 镜像由 2.1.0 升至 4.1.0。原 2.1.0 不支持 HNSW
  ANN、cosine_distance_approximate 与 Stream Load partial_columns,
  按当前 DDL 一启动就会失败。
- DSN 字面量拼接改用 mysql.Config.FormatDSN()。原 fmt.Sprintf 在用户名/
  密码包含 `@`/`:`/`/` 等字符时会跑偏。覆盖 health check 与 engine
  factory 两处。

健壮性修复:
- 新增 validateEmbedding,写入与查询前拒绝 NaN/±Inf;strconv.FormatFloat
  对非有限值会输出 "NaN"/"+Inf" 拼成无效 SQL。
- waitANNReady 改为后台 goroutine + 独立 context,避免新维度首次写入路径
  阻塞最长 30s;ANN 未就绪时 Doris 会自动退化为 brute-force。

清理:
- annIndexReady 移除最终两个分支都 return true 的冗余写法。
- Save 移除冗余的双重 toDorisVectorEmbedding。
- testDorisConnection 把 "5.7.99 Doris-4.1.0" 解析为裸 "4.1.0",与
  Postgres/ES 的版本格式对齐。

剔除(不应合入主仓):
- docs/wiki/集成扩展/Doris改动与上游同步.md:纯 fork 维护工作流文档。
- scripts/e2e-doris.sh:作者本地 E2E 验证清单。

测试:
- repository_test 用 require.Eventually 适配 ANN 异步轮询。
- 现有 doris 单测全部通过。
2026-05-09 00:31:03 +08:00
issunion 4cce6f2e99 feat(retriever): 接入 Apache Doris 4.1 作为向量数据库
为 RetrieveEngine 体系新增 Doris 后端,与现有 Qdrant/Milvus/Weaviate
等保持完整能力对齐:向量检索、关键词检索、健康检查、环境变量与多实例
DB 配置、前端类型注册、单元测试、Docker Compose 模板。

实现要点:
- 协议分工:主链路用 MySQL 协议(database/sql + go-sql-driver/mysql)
  做 DDL / 查询 / 删除;批量更新走 Stream Load HTTP API,并启用
  partial_update=true、merge_type=APPEND,自动按 1MiB 切分批次并处理
  307 重定向。
- 表结构:UNIQUE KEY(id) + enable_unique_key_merge_on_write=true 以
  支持 upsert/部分列更新;按维度分表(<base>_<dim>),每张表上建
  HNSW ANN 索引(metric_type=cosine_distance)和 INVERTED 索引
  (parser=chinese)。
- 分数语义:使用 cosine_distance_approximate,再以 1 - dist 转换为
  "越大越相似",与现有 KVHybridRetrieveEngine 约定一致。
- 异步索引:ANN 索引为后台构建,ensureTable 通过轮询 SHOW INDEX 等
  待索引就绪后再放行写入,避免首次检索召回为空。
- ARRAY<FLOAT> 序列化:go-sql-driver/mysql 不支持数组占位符,
  embeddingLiteral 将 []float32 转成 SQL 字面量字符串再拼接。

新增文件:
- internal/application/repository/retriever/doris/{structs,schema,
  query,repository,streamload,repository_test}.go
- scripts/e2e-doris.sh:E2E 验证清单
- docs/wiki/集成扩展/Doris改动与上游同步.md:fork-and-rebase 工作流
  与改动清单

修改文件(接线 + 文档):
- internal/types/{retriever,tenant,vectorstore}.go:新增
  DorisRetrieverEngineType、env 解析、表单 schema 与索引参数校验
- internal/container/{container,engine_factory}.go:环境变量驱动
  与 VectorStore 配置驱动两条路径都支持 Doris
- internal/application/service/vectorstore{,_healthcheck}.go:连接
  校验 + Ping/Version 健康检查
- docker-compose.yml:新增 doris-fe / doris-be 服务(profile=doris)
- .env.example:DORIS_* 环境变量与示例
- docs/{使用其他向量数据库,wiki/集成扩展/集成向量数据库}.md:
  使用说明与索引/分数行为说明

依赖:go.mod/go.sum 新增 github.com/go-sql-driver/mysql(运行时)和
github.com/DATA-DOG/go-sqlmock(测试)。

测试:repository 层 SQL 形状、Stream Load HTTP 行为、whereBuilder
逻辑、embeddingLiteral 往返、健康检查错误路径均有单测覆盖。
2026-05-08 21:59:35 +08:00
wizardchen e66c42343b fix(wiki-ingest): harden conflict retry and right-size retry budget
Avoid treating Redis pending-length read failures as empty queues, reset stale fail-count keys on fresh ingest/drop paths, and reduce wiki ingest MaxRetry to a moderate shared constant to limit queue churn.
2026-05-08 21:58:51 +08:00
wizardchen d0144f3586 perf(wiki): move ingest log to event table and index to on-demand API
Fixes O(n²) write amplification during wiki ingest on large KBs. Previously
every ingest/retract op re-wrote the single `slug='log'` row end-to-end and
every batch re-wrote the entire `slug='index'` directory markdown. On a 40k-
doc KB the log row grew to tens of MB and the index row to several MB, so
each batch triggered giant TOAST updates that dominated ingest wall time.

Log: new `wiki_log_entries` event table (`id DESC` indexed per KB) replaces
the single TEXT row. Batch ingest now collects entries and flushes them
once per batch via `AppendBatch`. Each entry stores `pages_affected` as
JSONB `[{slug,title}]` so the UI can render real titles; custom Scan falls
back to legacy `[]string` so older rows still deserialize.

Index: `wiki_pages[slug=index].content` keeps only the LLM-generated intro
(a few KB). The directory is now served by a structured paginated API
(`GetIndexView`) that reads `slug/title/summary` per type with cursor
pagination, so the agent and the frontend only pull the slice they need.
`RebuildIndexPage` degrades to a no-op; agent `wiki_read_page('index')`
synthesizes a small top-K overview and points callers at `wiki_search`.

Ingest resilience: LLM calls wrap with 3-attempt exponential backoff on
transient errors (5xx/408/429, transport resets/timeouts). Summary/extract
failures now bubble up so the batch's failed-op requeue path runs instead
of silently dropping the doc.

Frontend: sidebar Index/Log entries switch to dedicated views. Index view
streams intro → Summary → Entity → Concept → Synthesis → Comparison via
IntersectionObserver (with a nextTick re-check so small KBs still load
every section). Log view uses cursor pagination; pages_affected renders
titles with slug tooltip.

- new migration: migrations/versioned/000040_wiki_log_entries.{up,down}.sql
- tests: log repo pagination + legacy Scan, ListByTypeLight windowing,
  renderIndexOverviewForAgent output, isTransientLLMError classifier

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 21:48:52 +08:00
toy0116 bf12868923 fix(wiki): cap requeueFailedOps retry count to prevent queue pile-up
Root cause: requeueFailedOps unconditionally RPush'd every failed op back to
wiki:pending, causing unbounded growth when a document consistently triggers
LLM timeouts. Observed: 553 entries in wiki:pending:08134644 (one KB), mostly
duplicates for ~5 unique documents that each timed out on every batch cycle.

Fix:
- Add wikiFailCountKeyPrefix ("wiki:failcount:") and wikiMaxFailRetries (5)
  constants in wiki_ingest.go
- requeueFailedOps now atomically Incr(wiki:failcount:{kbID}:{knowledgeID})
  before each RPush; skips re-queue (drops op permanently) if count > 5
- Expire the fail-count key at wikiPendingTTL (24h) on every update
- wiki_ingest_batch.go: Del the fail-count key at ingestSucceeded++ so
  transient errors don't permanently burn through a document's retry budget

Evidence of problem severity: 5 persistently-failing docs × ~110 requeue
cycles each = 553 pending entries; had to use Lua atomic dedup to recover.

Behaviour after this fix:
- First 5 failures → normal retry (re-queued, follow-up batch scheduled)
- 6th failure → logged as WARN "dropping op … after 6 failures (limit 5)",
  not re-queued; prevents the feedback loop
- If document is fixed upstream and re-ingested via EnqueueWikiIngest, the
  op is a fresh entry with no existing fail-count key → full 5-retry budget

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:46:36 +08:00
toy0116 edbcfa80ab fix(wiki-ingest): prevent retry exhaustion on concurrent lock conflict
Root cause: when multiple wiki:ingest tasks queue up for the same KB,
they all hit ErrWikiIngestConcurrent and retry every 15 s. With
MaxRetry=10 (150 s window), any batch taking longer than ~2.5 min
caused pending tasks to exhaust retries and land in archived.

Confirmed via Redis inspection: archived wiki:ingest task had
retry=10, retried=10, error="concurrent wiki task active".

Two-part fix:

1. Early-exit at lock conflict when pending list is empty.
   If another batch is active and the pending list is already empty,
   there is nothing left to process — the concurrent task will handle
   everything. Return nil (success) immediately instead of burning
   through retry slots on a guaranteed no-op. Only retry when the
   pending list still has items, i.e., we have real work to do.

2. Increase MaxRetry 10 → 25 across all four enqueue sites.
   25 × 15 s = ~6 min window; accommodates large KB batches and the
   60 s orphan-lock expiry with generous headroom. Combined with the
   early-exit above, redundant tasks now consume 0 retries instead of
   10, so the remaining budget is entirely available for genuine
   concurrency conflicts.

No API or schema changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 21:46:36 +08:00
DuYunfan 16d31cb87b fix(frontend): use npm to match committed lockfile
The Dockerfile invoked corepack + pnpm install, but the repo only
ships package-lock.json (no pnpm-lock.yaml). On node:24-alpine
corepack resolves to pnpm v10, whose default --strict-dep-builds
rejects unbuilt postinstall scripts (esbuild, vue-demi,
@vue-office/pptx) and aborts with ERR_PNPM_IGNORED_BUILDS, breaking
docker compose build.

Switching to npm ci / npm run build aligns the image build with
CLAUDE.md and the local dev workflow that already uses npm.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 20:10:04 +08:00
wizardchen 576671e64a fix: propagate user attachments to agent query in AgentQA
Attachments uploaded from the UI were correctly parsed by the handler and
set on QARequest.Attachments, but the Agent path dropped them before
calling engine.Execute. Only the KnowledgeQA pipeline injected them via
Attachments.BuildPrompt(), so in "smart reasoning" (Agent) mode the model
never saw the uploaded file and answered as if no attachment existed.

Append Attachments.BuildPrompt() to agentQuery alongside QuotedContext,
mirroring chat_pipeline/into_chat_message.go. This keeps the engine
signature unchanged and matches the existing KnowledgeQA behavior.
2026-05-08 19:09:58 +08:00
aqilaziz d48638427f Fix authentication API documentation links 2026-05-08 16:48:07 +08:00
wizardchen 86f11d98e8 fix(frontend): configure pnpm v11 build approvals in workspace file
pnpm v11 no longer supports onlyBuiltDependencies in package.json and does not
accept install-time --allow-build flags. Move build-script approvals to
pnpm-workspace.yaml and copy it before install in Dockerfile so CI Docker builds
can apply allowBuilds during pnpm install.
2026-05-08 16:34:33 +08:00
wizardchen de7a241f8b fix(frontend): pass allow-build flags to pnpm install in Docker build
Some CI runners still fail with ERR_PNPM_IGNORED_BUILDS even with package-level
configuration. Pass explicit allow-build flags to `pnpm install` in Dockerfile so
required postinstall scripts are always permitted during image builds.
2026-05-08 16:28:12 +08:00
wizardchen 483576a9b6 fix(frontend): allow build scripts for pnpm to unblock Docker build
pnpm 10+ refuses to run dependency build scripts by default and exits
with ERR_PNPM_IGNORED_BUILDS, causing `pnpm install` to fail in the
frontend Docker image. Whitelist the packages that legitimately need
their install scripts (esbuild native binary, @vue-office/pptx,
vue-demi) via `pnpm.onlyBuiltDependencies`.
2026-05-08 16:17:52 +08:00
wizardchen a4c3c39556 perf(wiki): lazy-load sidebar page list with virtual-scroll tabs
?tab=wiki used to call listWikiPages({page_size: 500}) in a loop up to
50 times, loading ~25k pages into memory before rendering the sidebar
on knowledge bases with tens of thousands of pages. All rows were then
materialized as DOM nodes inside collapsible groups stacked vertically.

This commit reshapes the sidebar around per-type buckets that are
fetched on demand and rendered through a single virtual scroller:

* pagesByType holds one bucket per page_type (summary / entity /
  concept / synthesis / comparison), each with its own items list,
  pagination cursor, KB-wide total, and loading flag. Initial load
  fires one 50-item request per type in parallel.
* Index and Log are fetched directly via getWikiIndex / getWikiLog
  instead of scanning the full page list to find them.
* Groups are surfaced as a horizontal tab bar; switching tabs swaps
  the RecycleScroller's items. Only one bucket is visible at a time,
  so we never stack multiple virtual viewports.
* The scroller runs in page-mode with the outer .wiki-page-list as the
  scrollable ancestor; a 1px sentinel after the list is watched by
  IntersectionObserver to trigger the next page fetch. After each
  batch lands we call scroller.updateVisibleItems(true) on the next
  tick so appended rows render in order without requiring the user
  to jiggle the scroll.
* Switching tabs resets scrollTop so the retained position of the
  previous tab can't re-trigger the sentinel and cascade loads on
  the new (shorter) list.
* Search enters a dedicated flat-result mode (searchResults ref);
  clearing the query flips back to the bucketed view without
  refetching the buckets the user already had open.

Adds vue-virtual-scroller dependency and imports its stylesheet.
2026-05-08 15:07:59 +08:00
wizardchen df1e9a1bfd perf(wiki): subgraph API and interactive exploration for large graphs
Wikis with tens of thousands of pages used to crash the browser trying
to render the entire link graph at once (30MB+ JSON, 100k+ SVG elements).
This change moves the graph viewer from "fetch everything, render
everything" to "fetch a slice, expand on demand" with several matching
UX improvements.

Backend

* GET /wiki/graph now accepts `mode` (overview | ego), `center`, `depth`,
  `types`, `limit` query params. Default is overview top-500 by
  link_count, capped at 2000. Response includes a Meta object so the
  frontend can render a truncation hint and drive the UI.
* Pure helper `computeGraphSubset` extracted for testability; six unit
  tests cover overview truncation / type filter / ego BFS / missing
  center error.
* WikiLintService passes Limit=0 (uncapped) so link integrity checks
  still walk every page.
* Repo `Search` adds CASE-based relevance ranking (title 4, slug 3,
  summary 2, content 1) so full-text results put the most obvious
  matches first instead of whatever was updated most recently.

Frontend

* Graph viewer fetches overview on entry; ego pivot on double-click,
  search, wiki-link click, URL ?slug=, global issues jump.
* Shift+click or hover ⊕ button blooms neighbors onto the current
  canvas additively. Bloom tracks generations and LRU-evicts oldest
  when total exceeds 1500 nodes; ego center / selected node / latest
  anchor are always protected.
* "Grow frontier (N)" legend action expands every dashed-ring node in
  one click (6-way concurrency) while skipping Index/Log super-nodes
  that would otherwise dump the whole wiki onto the canvas.
* Node dashed expansion ring + drawer "X/Y neighbors shown" hint tell
  users which nodes still have neighbors to load.
* Search dropdown uses remote full-text search (debounced, sequence-
  numbered to drop stale responses) and falls back to the overview
  top-500 snapshot when the input is empty.
* Type filters now round-trip to the server so top-N is always
  computed from the user's active type set rather than hiding nodes
  client-side and shrinking the view.
* Status card replaces the cramped "centered on X · N hops · M nodes"
  line with a structured focus/overview summary, resolving slugs to
  page titles.
* Help popover in the legend lists every canvas shortcut.

i18n entries added for zh-CN / en-US / ko-KR / ru-RU.
2026-05-08 13:32:54 +08:00
nullkey fc5d16e331 feat(cli): top-level search command (PR-5)
The fourth and final v0.0 command — chunk hybrid retrieval, the demo
headline operation per ADR-3 (only one search command in the tree).

Maps to client.HybridSearch / GET /knowledge-bases/{id}/hybrid-search;
no SDK or server changes.

Flags:
- --kb (required): target knowledge base
- --top-k (default 8): max results
- --vector-threshold / --keyword-threshold: similarity floors
- --no-vector / --no-keyword: disable individual channels (mutually
  exclusive at the validation gate; checked before the SDK client is
  built so flag misuse fails fast)
- --json: emit envelope JSON (otherwise pretty list)

Service interface narrowed to just HybridSearch; tests inject fakes via
Factory.Client closure override.

Pretty rendering is a minimal text indent today; lipgloss tables arrive
in PR-3 of v0.2 (format/ split into presenters/style/tableprinter).
2026-05-07 23:19:46 +08:00
nullkey 811802485e feat(cli): auth login + auth status commands (PR-4)
First two leaf commands and the auth/ command group, registered under
the root command tree.

auth login:
- Email + password (interactive huh prompter) OR --with-token (read API
  key from stdin) for headless / CI use
- Validates --host (required, must be http/https URL)
- Persists access + refresh tokens (password mode) or api_key (token mode)
  into the secrets store wired in PR-3 — uses Store.Ref to record the
  URI scheme without leaking concrete backend type
- Writes the named context to ~/.config/weknora/config.yaml and sets it
  as current_context
- Typed loginResult payload for --json output (no map[string]any)

auth status:
- Calls client.GetCurrentUser -> /auth/me, prints context / host / user /
  tenant. --json emits a typed statusResult envelope with _meta.context
  and tenant_id populated for agents.
- Service interface narrowed to just GetCurrentUser; tests inject fakes
  by overriding Factory.Client (no runF gymnastics).

cmd/root.go: register \`auth\` parent under the root tree.
2026-05-07 23:19:46 +08:00
nullkey 16f11380ee feat(cli): keyring backend + TTY prompter (PR-3)
Adds the OS keyring secret backend and the huh-based TTY Prompter so
PR-4 (auth login) can persist credentials and read interactive input.
The Factory's Client closure SDK wiring + Secrets closure landed in
PR-1's foundation tree to keep PR-1 self-contained.

internal/secrets/keyring.go:
- KeyringStore: zalando/go-keyring backed; namespace "weknora:<context>".
  KeyringStore.Ref -> keychain://weknora/<context>/<key>
- NewBestEffortStore probes the OS keyring with a sentinel get and falls
  back to FileStore on unsupported platforms (WSL without DBus, etc.)

internal/prompt/tty.go:
- TTYPrompter: huh.NewForm / huh.NewInput / huh.NewConfirm wrapper
  satisfying Prompter. Used when iostreams.IO reports both stdout and
  stderr are TTYs.

Dependencies (zalando/go-keyring, charmbracelet/huh) are declared in
PR-1's go.mod since the foundation tree referenced them via the Store
interface and TTY-detection helpers.
2026-05-07 23:19:46 +08:00
nullkey 37d4fbde5d feat(client): split WithToken, add Raw + auth helpers (PR-2)
Five additive SDK changes that PR-3 (CLI Factory adapter) and PR-4
(auth login / status) consume.

WithToken split (Breaking + alias):
- WithAPIKey(key) sets X-API-Key header (long-lived API keys)
- WithBearerToken(token) sets Authorization: Bearer header (JWTs)
- WithToken kept as deprecated alias for WithAPIKey for two minor versions
- doRequest now sets X-API-Key and Authorization headers independently;
  callers can carry both kinds of credentials simultaneously

applyAuthHeaders extraction:
- Header injection (X-API-Key, Authorization, X-Request-ID, X-Tenant-ID)
  is now a Client method shared between doRequest and the multipart path
  in CreateKnowledgeFromFile. Eliminates ~30 lines of drift risk and
  fixes the latent missing X-Tenant-ID on file uploads.
- Drops the dead second tenant-header switch in doRequest.

Raw escape hatch:
- Client.Raw(ctx, method, path, body) is the public passthrough used by
  the future \`weknora api\` command. godoc marks it experimental.

Auth methods (auth.go, new):
- Login(ctx, LoginRequest) -> POST /auth/login (existing server endpoint)
- GetCurrentUser(ctx) -> GET /auth/me, returns *CurrentUserResponse
- RefreshToken(ctx, refreshToken) -> POST /auth/refresh
- Types: LoginRequest, LoginResponse, AuthUser, AuthTenant,
  CurrentUserResponse, RefreshTokenResponse

No server changes. The auth methods consume endpoints that already exist;
PR-3/PR-4 wire them through the CLI.
2026-05-07 23:19:46 +08:00
nullkey a39805aa89 feat(cli): scaffold weknora CLI module foundation (PR-1)
Empty cli/ Go module with Factory + Options skeleton that all v0.x commands
plug into.

Foundation packages (10 internal/):
- cmdutil: Factory(3 closure: Config/Client/Prompter) + Options + Errors
  with namespaced ErrorCode + typed predicates (IsAuthError / IsNotFound /
  IsTransient / IsAuthExpired) + Exporter + AddJSONFlags + ExitCode/PrintError
- iostreams: package singleton with TTY detection; ColorEnabled honors
  NO_COLOR / TERM=dumb on demand
- agent: mode + detect + annotations (Stripe pkg/useragent pattern;
  CLAUDECODE / CURSOR_AGENT / CODEX_* / AIDER_* / CONTINUE_* /
  OPENCODE_* / GEMINICODER env detection)
- build: ldflags-injected version/commit/date
- safepaths: separator-aware path-traversal protection
- config: yaml.v3 schema + atomic save 0600 (XDG-style)
- format: success/failure envelope (code/hint/request_id/risk/retryable/
  console_url) with typed RiskLevel
- prompt: Prompter interface + AgentPrompter (rejects in agent mode)
- secrets: file 0600 store with weknora:<context>:<key> namespace
- testutil: XDGTempDir(t) test helper

Smoke command: weknora version (-/--json). Build/test/vet matrix in
.github/workflows/cli.yml across linux x macos x windows x Go 1.24.

Factory.Client returns CodeLocalUnimplemented until PR-3 wires the SDK
adapter; PR-1 commands (version) don't need it.
2026-05-07 23:19:46 +08:00
wizardchen 6019c4b661 fix(frontend): polish model settings UI and fix disabled brand color
- Fix global `--td-brand-color-disabled` to use correct muted green instead of bright green
- Unify `variant="text"` buttons (refresh, download, check dimension) in model settings to inherit standard TDesign styles
- Adjust WeKnora Cloud credential status layout and change reconfigure button to outline
- Fix font size of external doc links in model settings hint blocks
- Increase font weight of KB chunking debug trigger for better visibility
2026-05-07 21:51:22 +08:00
wizardchen c77cef5a69 feat(frontend): unify external doc-link style and outline add buttons
- Add global .doc-link utility in theme.css (brand color text + link icon, underline on hover).

- Convert all external doc/console links across settings to use .doc-link with a trailing <t-icon name="link" />, matching the Ollama browse-library style.

- Switch primary section-toolbar 'Add X' buttons (Model/WebSearch/VectorStore/MCP/DataSource) to outline variant for a lighter, more consistent look.

- Drop redundant per-component CSS for builtin-hint-link, weknoracloud-hint-link, engine-doc-link, engine-link, hint-link, section-doc-link, ds-doc-link-inline, model-link, etc.
2026-05-07 21:51:22 +08:00
wizardchen cc6ac6ac50 fix(frontend): unify radio button theme and disabled selection
- Add global t-radio-button outline styles in theme.css (brand fill when checked).

- Align ModelEditor, share KB, datasource, agent/MCP settings with t-radio-button.

- Remove redundant :deep overrides in AgentEditorModal, KB editor, FAQEntryManager, etc.

- Use brand disabled tone for disabled+checked radio buttons so selection stays visible.
2026-05-07 21:51:22 +08:00
wizardchen 627e9e436d fix(frontend): render markdown tables in chat wiki drawer
The wiki page drawer opened from the chat view rendered tables without
borders or padding because the global `.wiki-graph-drawer .wiki-reader-body`
style block was missing GFM table rules. Add the same table styles already
used by `WikiBrowser.vue` so tables show consistent borders, header
background, and hover states across both entry points.
2026-05-07 18:29:36 +08:00
wizardchen 92e02c1d49 chore(llm): add fire-once OpenAI-layer stream diagnostic logs
Add per-stream once-only logs at the OpenAI-protocol layer so we can
triage streaming behavior (natural-stop vs tool-call, TTFC, ordering of
reasoning/content/tool_calls) without grepping through every delta.

streamState now tracks:
  - streamStartedAt    : baseline for elapsed_ms on each fire-once log
  - firstContentSeen   : delta.Content first appearance
  - firstReasoningSeen : reasoning_content first appearance
  - firstToolCallSeen / noToolCallStopLogged (existing flags retained)

Logs emitted at most once per stream:
  [LLM Stream] First reasoning_content at OpenAI layer (len, preview, elapsed_ms)
  [LLM Stream] First delta.Content at OpenAI layer (len, preview, tool_call_seen, thinking_seen, elapsed_ms)
  [LLM Stream] First tool_calls delta at OpenAI layer (count, first_id, first_name, first_content_seen, thinking_seen, elapsed_ms)
  [LLM Stream] Natural-stop at OpenAI layer (finish=stop, tool_calls field never observed, thinking_seen, first_content_seen, elapsed_ms)

Together with the existing agent-layer "Natural-stop candidate detected"
log this lets a single grep reconstruct the temporal layout of one stream
(reasoning -> content -> tool_call OR natural-stop) and tell apart:
  (A) tool_calls field truly absent (real natural-stop)
  (B) tool_calls field arrived but high-level marker not yet emitted

Pure logging change; no behavior change. Previews use the existing
truncateForDebug helper (rune-safe, capped at 80 chars).
2026-05-07 18:28:28 +08:00
wizardchen 1d3bd8bb13 chore(agent): log natural-stop candidate when no tool calls
Add explicit response logging for the branch where the model returns
finish=stop with zero tool calls. This makes it easier to identify the
natural-stop path early in logs and correlate rounds that are likely to
be treated as final-answer candidates by analyzeResponse.

- include `tool_calls=0` in the no-tool summary log
- emit a dedicated "Natural-stop candidate detected" info log when
  finish_reason is stop and no tool calls are present
2026-05-07 18:28:28 +08:00
wolfkill 450a5bd2dd fix(docreader): throttle heavy parser concurrency 2026-05-07 17:36:09 +08:00
wizardchen 56058287dd fix(agent): drop whitespace-only thinking events from the steps tree
Some LLMs (e.g. qwen3 series) emit blank lines like "\n\n" before tool
calls. The backend faithfully forwards these as thought_chunk events
because streamThinkingToEventBus only filters chunk.Content == "" and
treats whitespace as content. The frontend then merges them into a
thinking event whose content is non-empty in JS truthiness terms but
visually empty, producing a "思考" card with a chevron and no text.

Filter such events out at the end of buildFullEventList so neither the
intermediate-steps tree nor the displayEvents fallback path renders
them.
2026-05-07 17:33:03 +08:00
wizardchen 921e4c015a fix(agent): hide trailing thinking event when it duplicates final answer
The natural-stop path on the backend (observe.go Case 1) emits the same
content twice: once as streaming thinking chunks (from
streamThinkingToEventBus's per-chunk callback) which the frontend merges
into a single thinking event, and once as one big EventAgentFinalAnswer.
The result is that the intermediate-steps tree contains a thinking card
whose content is identical to the answer card rendered below it.

Detect this duplication and hide the offending thinking event from the
tree:

- New utils `normaliseForComparison` and `thinkingEqualsAnswer` perform
  a tolerant comparison: strip <think>…</think> blocks (which leak into
  raw thinking chunks but not into the post-StripThinkBlocks answer),
  strip final-answer wrappers, collapse whitespace, then exact-equal or
  ±2% length with matching head/tail.
- `hiddenThinkingEventIds` (replacing the previous single-id
  `promotedThinkingEventId`) now collects two cases of duplicate
  thinking: the legacy promotion fallback (no answer event present) and
  the natural-stop content-match case.

The final_answer-tool path is unaffected because reasoning thinking
events ("我找到了…让我整理…") and the answer payload have different
content and don't match the equivalence check.
2026-05-07 17:33:03 +08:00
wizardchen 4677caba01 fix(agent): unify answer rendering when model skips final_answer tool
Many models (especially smaller or non-OpenAI-style ones) do not reliably
call the structured final_answer tool, even when the system prompt
explicitly requires it. Two visible symptoms in the agent UI:

1. The natural-stop path emits the answer as a final_answer event whose
   content is wrapped in <answer>...</answer>, <final_answer>...</final_answer>,
   or prefixed with "Final Answer:" / "最终答案:". These wrappers were
   rendered verbatim into the answer card.

2. When no answer event arrives at all, the trailing thinking event was
   surfaced via the legacy fallback but kept its collapsed "思考" card
   styling instead of rendering as a proper answer.

This change adds a small unwrap helper and wires it through the answer
rendering path:

- New util `unwrapFinalAnswerWrappers` strips conservative outer wrappers
  (XML-style answer tags, fenced ```final_answer code blocks, and leading
  "Final Answer:" / "最终答案:" prefixes) only when they cover the entire
  trimmed content, so user markdown that happens to contain similar text
  is left alone.
- `renderAnswerContent` and `getActualContent` apply the unwrap before the
  markdown renderer / clipboard copy, so users see and copy clean text.
- The legacy "promote trailing thinking to final answer" fallback now
  synthesizes a virtual answer event so it renders with the answer card
  UI (expanded markdown + copy/add toolbar) instead of the collapsed
  thinking card. The promoted thinking event is hidden from the
  intermediate-steps tree to avoid duplication.
2026-05-07 17:33:03 +08:00
bingxiang.cheng a3c3ba5acf fix(markdown): 修复小概率情况下公式加载失败问题 2026-05-07 16:34:20 +08:00
bingxiang.cheng a232b12ff0 fix(agent): 修复WeKnora对话完前端未显示结束标记问题
fix(agent): 修复流式回答think内容与answer混淆问题
2026-05-07 16:34:20 +08:00
draix 7adf88766a fix(graph): recover JSON from malformed/truncated fences in extract_entity
Closes #1113

The graph extraction pipeline relies on a strict markdown fence regex to
pull JSON out of LLM responses. In production the regex misses ~84% of
real-world responses, in three ways:

  1. The LLM hits max_tokens mid-output and produces no closing fence.
  2. The opening fence is malformed or surrounded by prose, so the
     non-greedy regex fails to anchor a match.
  3. The model returns raw JSON with stray backticks but no real fence.

In every case extractContent fell through to returning the raw text,
which then failed json.Unmarshal with errors like
"invalid character '` + "`" + "' looking for beginning of value".

This change keeps the existing happy path untouched and adds a
conservative recovery step in the default branch of extractContent:

  - If an opening ``` is present, take everything after it, drop a
    likely language tag on the first line, cut at any trailing closing
    fence, and trim stray backticks/whitespace.
  - Otherwise, look for an outermost JSON object/array in the text using
    a small bracket-balanced scanner that respects string literals, so
    embedded {} or [] inside JSON strings don't confuse it.
  - Only fall back to the original raw-text behavior when neither
    strategy yields anything plausible.

The recovery helpers (stripFencesAndExtract, extractJSONLike,
isLikelyLanguageTag) are package-private and have table-driven tests in
extract_entity_test.go covering the three failure patterns described in
the issue, plus the previously-working fenced and bare-JSON shapes to
guard against regressions.
2026-05-07 15:40:19 +08:00
wizardchen 4a5caf54af fix(frontend): make general-settings font picker actually work
The font picker in 常规设置 silently failed in three distinct ways:

1. Font family changes were invisible on macOS because 6 of the 7 options
   (Inter / Roboto / Segoe UI / Cascadia Code / JetBrains Mono / Fira
   Code) aren't installed by default and fell back to system SF. Swap
   the picker to a platform-aware list — Mac / Windows / Linux each see
   only the fonts their OS actually ships. The "third sans slot" now
   always serves a serif face (Georgia / Times New Roman / DejaVu
   Serif) so the difference is unambiguously visible instead of being
   three near-identical sans families.

2. Font size flipped --app-font-scale on <html>, but that variable was
   only consumed by a handful of TDesign tokens and the body reset, so
   the ~1000+ hard-coded font-size: NNpx rules across the frontend
   stayed put. Apply the multiplier via `zoom` on <html> so the whole
   document scales, then fix the 15 `100vh` rules that would overflow
   under zoom (html / body / #app are already height: 100%, so the
   descendants can use 100% too).

3. The monospace picker had no visible effect even on a working stack
   because `<t-input style="font-family: ...">` is forwarded to the
   inner <input>, but TDesign applies `font: var(--td-font-body-medium)`
   — a shorthand that resets font-family. Reach the real <input> /
   <textarea> via `:deep()` instead of inline style.

Additional polish:
- Add a live preview row under each picker so users can see the effect
  without hunting for a code block or API Info screen
- Rename "等宽字体 / Monospace Font" → "代码字体 / Code Font" to match
  VSCode / GitHub / Notion convention; the old label was jargon
- Rewrite descriptions to explain the actual purpose and scope
2026-05-07 15:07:24 +08:00
wizardchen fb9ae11aa8 fix: 为前端静态资源设置正确的 Cache-Control 响应头
修复升级前端版本后,用户必须退出重新登录才能看到新版页面的问题。

根因:nginx 和 Lite 版 Go 静态服务均未设置 Cache-Control,浏览器按
启发式策略长期缓存 index.html,导致刷新仍加载旧版入口 HTML,旧版
引用的 hash 文件名也就不会变更。

改动:
- frontend/nginx.conf: index.html 及 SPA fallback 设 no-cache,
  must-revalidate;/assets/* (Vite 产物带 hash 文件名)设一年
  immutable 长缓存。location 内重复声明原有安全头,避免 nginx
  add_header 不继承导致安全头丢失。
- internal/router/router.go: Lite 版 serveFrontendStatic 按同样
  策略设置 Cache-Control。
2026-05-07 12:08:02 +08:00
wizardchen c577d0e474 fix(knowledge): polish chunking-settings debug trigger and strategy hint
The chunking-settings debug trigger and strategy picker had two UX
issues reported by real usage:

- The test button used theme=primary + variant=outline, a combo that
  appears nowhere else in the project. A green-bordered green-text
  pill in the corner of the section header read as a misplaced
  promotional element rather than a secondary action.
- The strategy select's placeholder was "选择策略(默认使用经典递归分块)"
  / "Select strategy (defaults to classic recursive splitting)". Naming
  the fallback strategy inside the placeholder made the empty select
  look as if the user had already picked "classic recursive" — even
  though the value is actually empty.

- Move the debug trigger from the section header into the strategy
  row's right column, stacked below the select. Users meet the test
  action exactly when they're deciding which strategy to use.
- Switch the trigger to a text-style button and override TDesign's
  default hover/focus/active surfaces so the button reads as an inline
  link (text colour shift only, no filled background block).
- Rename the trigger to "测试分块效果" / "Preview chunking" so it
  describes what will be inspected rather than the input source.
- Drop the reference to the fallback strategy inside the placeholder.
  "不填则按长度切分" / "splits by length if left empty" preserves the
  original information about empty-value behaviour without looking
  like a selected option.
- Remove the now-unused 80px right padding the section header reserved
  to keep the old inline trigger clear of the modal close button.
2026-05-07 11:08:29 +08:00