ae7cd02cd6735ffaf925ffec6fe3205cc6f92c9d
1696 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 wizardchen
|
ae7cd02cd6 |
fix(rbac): polish tenant member UI; add /leave endpoint
Self-review of PR 3's TenantMembers.vue surfaced a handful of real bugs
and a few rough edges. This commit fixes them and adds the self-leave
endpoint that PR 3's design called out as a follow-up.
Frontend bug fixes
- Read err.status (not err.response.status). The axios interceptor in
utils/request.ts already flattens errors into { status, message,
... }, so the original branch-by-status code never matched and every
4xx fell through to the generic toast. Fixes 404/409 surfacing as
"userNotFound" / "alreadyMember" / "lastOwner" instead of "Something
went wrong".
- t-dialog @confirm replaces :on-confirm so the dialog stays open on
validation failure and on API error. Users can fix and retry without
retyping the email.
- Drop the t-select revert hack; with :model-value the rendered value
is already source-of-truth-on-row.role, so no setTimeout dance is
needed.
- Don't toast "no active tenant" on cold mount; watch the activeTenantId
ref and load once the auth store hydrates.
Frontend UX additions
- Empty state via t-empty (also covers the search-no-results case).
- Compact role-permissions matrix above the table, mirroring
OrganizationSettingsModal so Owners can see what each role grants
before promoting/inviting.
- Search box filtering by username/email (client-side, sufficient for
expected per-tenant member counts).
- "Leave Tenant" button — visible to any active member except the
last-Owner case (server still enforces).
Backend
- New POST /tenants/:id/leave route. Calls TenantMemberService.RemoveMember
with the caller's own user id so non-Owners can quit on their own.
Gated Viewer+ in the matrix; the existing last-Owner invariant in the
service still rejects last-Owner self-leave with 409.
- 3 new handler tests cover the happy path, the last-Owner 409 path,
and the missing-caller-id 401 path (defensive — auth middleware
always attaches a user, but the test pins the contract).
i18n
- Add tenantMember.permissions / search / leave / empty strings in all
four locales (en/zh/ko/ru).
|
||
|
wizardchen
|
3f7150cb5c |
feat(rbac): tenant member management + invite_only gate (PR 3 of 3, #1303)
Wire the human-facing surface for the tenant-RBAC system that PR 1 introduced and PR 2 enforced: REST endpoints + UI for managing tenant members, plus the long-pending invite_only registration gate. Backend - internal/handler/auth.go: Register now rejects with 403 when auth.registration_mode=invite_only. PR 1 already exposes the mode via /auth/config so the frontend hides the sign-up link; this is the server-side enforcement for direct API calls. - internal/handler/tenant_member.go: new TenantMemberHandler with ListMembers / AddMember / UpdateMemberRole / RemoveMember. Maps service-layer sentinels onto HTTP semantics (ErrLastOwner -> 409, ErrMembershipAlreadyExists -> 409, ErrUserNotFound on AddMember -> 404, ErrInvalidTenantRole -> 400). AddMember requires the invitee to already have an account; SMTP-driven email invites are deferred. - internal/types/tenant_member.go: new TenantMemberResponse projection carrying the fields the management UI renders (email, username, avatar, role, joined_at). - internal/router/router.go: four routes under /tenants/:id/members gated per the matrix (Viewer+ for list, Owner+ for every mutation). - internal/container/container.go: dig provider for the new handler. Tests - handler/tenant_member_test.go: 12 table-driven cases covering happy path, dangling-user tolerance, ErrLastOwner -> 409, ErrMembershipNotFound -> 404, ErrMembershipAlreadyExists -> 409, unknown email -> 404, invalid role rejected before user lookup, bad tenant id -> 400. - handler/auth_register_invite_only_test.go: invite_only -> 403, self_serve -> 201, nil Auth config falls through. Frontend - src/api/tenant/members.ts: typed client (TenantRole/TenantMember/...) for the four endpoints. - src/views/settings/TenantMembers.vue: members table with add dialog, inline role select, confirm-remove modal. All mutation widgets gated behind currentTenantRole === 'owner'; server still enforces. - src/views/settings/Settings.vue: register the Members tab; show only when currentTenantRole is loaded (avoids tab flash on cold nav). - i18n in en-US/zh-CN/ko-KR/ru-RU: new tenantMember namespace covering all UI strings and structured error messages (lastOwner / userNotFound / alreadyMember / invalidRole etc). This completes the #1303 RBAC rollout. With cfg.Tenant.EnableRBAC=true operators now have a full management surface; with EnableRBAC=false the routes still exist but mutations log-and-pass per PR 2 contract. |
||
|
wizardchen
|
0f0f7e4339 |
test(rbac): cover review gaps from PR 2 self-review
- Add TestRequireRole_CrossTenantSuperuserBypass: pin the CanAccessAllTenants -> Owner-gate bypass behaviour for RequireRole, matching the existing coverage on RequireOwnershipOrRole. Catches reorderings of the fast paths that would re-lock org-level superusers out of foreign-tenant Owner-only routes. - Add TestAgentCreatorLookup_CrossTenantIsHiddenAsNotFound: lock down the defence-in-depth tenant re-check inside AgentCreatorLookup so a future refactor that loosens the service-layer scope (or wires a cross-tenant variant by mistake) cannot leak access via the ownership shortcut. - Reword RequireOwnershipOrRole step 7-8 doc comment so "role check already failed" is unambiguous (it refers to step 1 above). - Annotate the synthetic-system-user construction in middleware/auth.go with a back-pointer to types.IsSyntheticUserID so anyone changing the prefix or numeric suffix knows to update the helper too. |
||
|
wizardchen
|
8b71def60b |
fix(rbac): address PR 2 review feedback
Middleware - RequireOwnershipOrRole now skips the creator lookup entirely when EnableRBAC is off, eliminating a hidden per-request DB roundtrip in the rollout window. - New ErrResourceNotFound sentinel lets lookups signal "not visible to this tenant"; the middleware passes the request through so handlers can return a real 404 instead of a misleading 403. - Transient lookup failures now surface as 503 (was 403) so monitoring can tell denied-by-policy from cannot-verify-right-now. - Cross-tenant superusers (CanAccessAllTenants) bypass tenant role gates, restoring DELETE /tenants/:id and ResetAPIKey for org operators that resolve to Admin in foreign tenants. - Construction-time warning fires once when cfg is nil/incomplete, so a wiring bug that disables enforcement globally is visible at boot. Route coverage - Add role guards to FAQ, KnowledgeTag, WikiPage, Skill, WebSearch, WebSearchProvider, WeKnoraCloud, Session, Chat, and Message route groups so the matrix actually matches the PR description. Service layer - KB and agent creators are no longer recorded as the synthetic system-<tenantID> user used by the X-API-Key path; resources stay tenant-owned. New types.IsSyntheticUserID helper centralises the prefix check. - KBCreatorLookup and AgentCreatorLookup re-check tenant scope so a same-ID resource in another tenant cannot piggyback through the ownership shortcut. Docs - CustomAgent.RunnableByViewer comment now matches enforcement: it only blocks Viewer-driven agent-mode runs, not plain RAG/Wiki QA. Tests - New cases for the 503/404 paths, lookup-skip behaviour, cross-tenant superuser bypass, the lookup helpers (not-found, cross-tenant, built-in agent, missing tenant context), and IsSyntheticUserID. |
||
|
wizardchen
|
9e3c92ca95 |
feat(rbac): enforce tenant role matrix on routes (PR 2 of 3, #1303)
Build on PR 1's tenant_members schema by enforcing the role matrix on
every authenticated route and at agent run time. Enforcement is gated
behind cfg.Tenant.EnableRBAC so the change ships dormant — the same
code paths log the would-be rejection during the rollout window and
start blocking once operators flip the flag.
Middleware (internal/middleware/rbac.go):
- RequireRole(min, cfg): role-only gate using TenantRole.HasPermission.
- RequireOwnershipOrRole(min, lookup, cfg): allows the original creator
through even when their role is below min, otherwise falls back to
the role check. Lookup errors fail closed; an empty creator_id is
treated as tenant-owned and requires the role.
- Both honour cfg.Tenant.EnableRBAC for the rollout window.
Creator lookups (internal/handler/rbac_lookups.go):
- KBCreatorLookup resolves :id -> KnowledgeBase.CreatorID.
- AgentCreatorLookup resolves :id -> CustomAgent.CreatedBy; built-in
agents (IsBuiltin=true) return ("", nil) so they remain Admin+ only.
- Compile-time guards keep the methods aligned with CreatorLookup.
Route guards (internal/router/rbac.go + router.go):
- rbacGuards bundles the matrix into Viewer/Contributor/Admin/Owner
helpers plus OwnedKBOrAdmin / OwnedAgentOrAdmin. Each Register*
function takes *rbacGuards so route lines remain readable.
- KB / Agent / Knowledge / Chunk / Tenant / Model / MCPService /
VectorStore / DataSource / IMChannel routes annotated per the PR 1
design doc matrix. Tenant-wide infra (models, vector stores, MCP,
data sources, IM channels) reads as Viewer+ and writes as Admin+;
per-resource mutations on KB / Agent route through the
ownership-or-Admin guard.
Resource ownership (services):
- CreateKnowledgeBase / CopyKnowledgeBase populate CreatorID from the
authenticated user, so the OwnedKBOrAdmin guard has a creator to
match.
- CreateAgent / CopyAgent populate CreatedBy similarly. Built-in
agents stay tenant-owned.
Agent run-time gate (internal/handler/session/qa.go):
- AgentQA refuses to launch the agent engine for Viewers when the
target agent has RunnableByViewer=false. Same EnableRBAC gating as
the middleware so it's a no-op until enforcement is turned on.
Tests:
- 12 unit tests cover RequireRole / RequireOwnershipOrRole, including
the fail-open and fail-closed branches and the lookup-error path.
|
||
|
wizardchen
|
af23483433 |
fix(auth): honour JWT tenant_id claim in middleware
POST /auth/switch-tenant mints an access token whose `tenant_id` claim
points to the target tenant, but the middleware previously discarded
that claim entirely:
user, err := userService.ValidateToken(ctx, token) // returned only *User
targetTenantID := user.TenantID // always the home tenant
The session therefore stayed scoped to the user's home tenant despite
the supposedly fresh token, making the new endpoint a no-op once
the request hit any tenant-scoped query.
This change widens the contract:
- UserService.ValidateToken now returns (user, activeTenantID, error)
where activeTenantID is the JWT's `tenant_id` claim, falling back to
user.TenantID when the claim is absent (backward compatibility with
pre-RBAC tokens still in the wild).
- middleware/auth.go uses activeTenantID as the default target before
applying any X-Tenant-ID header override, and recomputes
crossTenantSwitch from `activeTenantID != user.TenantID` so the
resolveTenantRole branches stay correct for switch-tenant sessions.
- The non-middleware /auth/validate handler in handler/auth.go absorbs
the new return value (it only cares about user identity).
Tests:
- internal/application/service/user_jwt_test.go pins the JWT->tenant-id
projection (`tenantIDFromClaims`) — claim present beats fallback,
zero/negative/string claims fall back, int64/uint64 are accepted for
test-built tokens.
Required for PR 2 multi-tenant integration tests, where a single user
needs to switch between Owner-in-A and Viewer-in-B sessions and observe
distinct authority on each side.
|
||
|
wizardchen
|
f3d1210577 |
refactor(rbac): address PR 1 review feedback
Tighten correctness, security and observability of the tenant RBAC foundation introduced in 3c4cdd70 in response to internal review. Bugfixes - custom_agents.runnable_by_viewer: drop the GORM `default:true` tag. GORM treats a plain bool's zero value as "unspecified" and silently rewrites it to the column default on insert, which would prevent admins from ever creating an agent with viewer-runtime disabled. The column-level DEFAULT TRUE on the database side still backstops rows inserted by raw SQL / migrations. - TenantMemberRepository.HasAnyMembers: replace `Limit(1).Count(&n)` with `Take(&probe)`. GORM's Count ignores Limit and scans the whole table, which matters because this runs on the auth middleware's hot path for users without a cached membership. - service/user.LoginWithOIDC: include User, Tenant and Memberships in the OIDCCallbackResponse so the OIDC flow returns the same shape as password login (previously User and friends were left nil). Security - middleware/auth.resolveTenantRole: restrict orphan-tenant auto-promotion to the caller's home tenant. Cross-tenant superuser visits and JWTs minted for foreign tenants no longer trigger silent Owner grants. Two regression tests pin the new behaviour. - service/user.synthFallbackMembership: default the synthesised role to Viewer instead of Owner. The login response only feeds UI; the backend re-derives the real role from tenant_members on every request, so showing a Viewer UI is preferable to a misleading Owner UI that would surface admin controls the backend will then 403. - migrations/versioned/000043_tenant_rbac.down.sql: gate the destructive drop behind a `weknora.allow_destructive_migration` GUC so an accidental `migrate down` cannot wipe role data in production. - frontend/src/stores/auth.ts: comment currentTenantRole as rendering-only so future PRs cannot mistake the localStorage-backed value for an authorisation signal. API contract - Memberships in LoginResponse and OIDCCallbackResponse now serialises without omitempty, and buildMembershipsForUser / synthFallbackMembership always return a non-nil slice — frontend can rely on JSON `[]`. - handler/auth.GetCurrentUser includes memberships so /auth/me refreshes recover currentTenantRole after a page reload. Performance - Add TenantService.GetTenantsByIDs (with the matching repo method) and use it inside buildMembershipsForUser to batch tenant-name lookups instead of issuing one query per membership. Observability - Replace the remaining `log.Printf` calls in middleware/auth with contextual logger.Warnf / logger.Infof so role resolution events ride the standard request-id / Langfuse trace plumbing. The successful auto-promote line is tagged `[audit]` for log mining. Tests - internal/application/service/tenant_member_test.go: hand-rolled fakeTenantMemberRepo plus 11 cases covering AddMember validation, EnsureOwner idempotence, "cannot demote/remove the last Owner", same-role no-op, ErrInvalidTenantRole / ErrMembershipNotFound, and the TenantRole.HasPermission lattice. - internal/middleware/auth_role_test.go: fakeMemberService plus 8 cases pinning every branch of resolveTenantRole, including the two H1 regressions (cross-tenant superuser must not auto-promote; orphan auto-promote requires the home tenant). |
||
|
wizardchen
|
7f7a48880f |
feat(rbac): add tenant_members schema and auth wiring (PR 1 of 3, #1303)
Lay the foundation for tenant-level RBAC without enforcing it yet. Schema, types, middleware lookup, and login-flow shape changes ship now; RequireRole/route annotations and member-management endpoints follow in PR 2 and PR 3. Highlights: - New tenant_members table (Postgres + SQLite) with (user_id, tenant_id) unique key and (owner|admin|contributor|viewer) role values. Migration 000043 backfills one row per existing user, marking the earliest-created active user in each tenant as Owner and the rest as Contributor. - KnowledgeBase.creator_id and CustomAgent.runnable_by_viewer added so PR 2 can wire ownership-based and Viewer-runtime checks. - Auth middleware loads the active TenantMember row, sets a new TenantRoleContextKey, and falls back via three escape hatches: cross-tenant superusers (CanAccessAllTenants) get Admin in the target tenant; orphan tenants (zero members) auto-promote the first authenticating human to Owner; and when tenant.enable_rbac is false (the v1 default) unmatched lookups fail open with Admin to preserve current behaviour. - Register / OIDC provisioning now insert an Owner membership for the registrant via TenantMemberService.EnsureOwner. - LoginResponse renamed tenant -> active_tenant and gained a memberships array. POST /auth/switch-tenant issues a fresh token pair scoped to a target membership; GET /auth/config exposes auth.registration_mode for the frontend register-tab gating. - New config keys: auth.registration_mode (self_serve | invite_only, defaults to self_serve) and tenant.enable_rbac (defaults to false). Both have WEKNORA_* env overrides. - Frontend store persists memberships and exposes currentTenantRole; Login.vue calls /auth/config and hides the Register tab in invite_only mode. Go SDK types updated to match the new LoginResponse shape. |
||
|
wolfkill
|
3214e3d952 | fix(agent): exclude processing docs from prompt | ||
|
ochan.kwon
|
0e8de6192c |
feat(knowledge-base): validate vector store bindings on create, copy, and delete
Wires KnowledgeBase.VectorStoreID and the ownership-aware retrieve factory into the user-facing knowledge-base lifecycle: - POST /knowledge-bases validates the requested vector_store_id against the caller's tenant scope and the engine registry. New error codes ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable (2201) distinguish the typed branches without echoing UUIDs to the client. - GET / POST / PUT / PUT-pin responses embed the bound store's display metadata (name, source, engine_type, status) without exposing any connection credentials. Cross-tenant shared KBs receive a suppressed payload (vector_store_id stripped, source="shared") so operator-chosen store names cannot be enumerated across tenants. - POST /knowledge-bases/copy synchronously rejects clones whose target has a different embedding model or vector store, before the async clone task is enqueued. The async clone worker re-applies the same checks for defense in depth. - DELETE /vector-stores/:id refuses to remove a store with bound KBs, inside a transaction that row-locks the store on PostgreSQL and serializes via WAL on SQLite. unregister-from-registry is wrapped in defer/recover so a panic surfaces as a structured warning instead of silently leaking a stale engine. - vector_store_id is immutable after creation. The GORM <-:create tag blocks every ORM update path; the service-layer DTO omits the field entirely; a reflection-based regression test catches any future maintainer who adds it back to either layer. - Empty-string vector_store_id is normalized to nil at both the create path and inside SharesStoreWith, so rows persisted by callers that did not run Normalize first cannot trip false same-store comparisons. Part of #993. Depends on #994 and #1310. |
||
|
nullkey
|
7611d59d71 |
docs(cli): README / AGENTS.md / CHANGELOG + CI parity test
Wire-contract documentation and the CI check that keeps it honest. * cli/README.md gains a verbatim --help block (top-level + subtrees), an Exit codes table covering 0/1/2/3/4/5/6/7/10/124/130, a "Status vs check" verb-pair subtable, and a "doc wait" paragraph spelling out the four exit codes (0 / 1 / 124 / 130). The api passthrough note trims storage provider out of the deep-config list now that kb create --storage-provider is a polished flag. * cli/AGENTS.md becomes the contributor guide: build/test, CRUD flag conventions, the status/check verb pattern, long-poll wait commands, the SetAgentHelp pattern, and a full Error code reference with 35 typed codes mapped to namespaces, exit codes, retryable / hint guidance. Reference section is bracketed by HTML markers so a CI parity test can keep it in sync with AllCodes(). * cli/internal/cmdutil/errors_doc_test.go enforces parity: every code in AllCodes() must appear in AGENTS.md inside the markers, and AGENTS.md must not reference codes that no longer exist. Fails CI if a new typed code is added without documentation. * CHANGELOG.md gets the v0.6 entry: BREAKING (--json / --no-stream / WEKNORA_SDK_DEBUG / kb create --name), Added (--format / --jq / doc wait / --log-level / kb-and-agent status & check / multi-id delete / api --paginate / MCP schema extension / SetAgentHelp / signal-aware ctx / kb create --storage-provider / new operation.* namespace), Changed (multi-id partial-failure exit code, doc upload FlagError, --log-level FlagError, multi-id stdout cleanup, README / AGENTS.md changes), with a Migration from v0.5 section walking every BREAKING through its v0.6 replacement. |
||
|
nullkey
|
34bb0b5096 |
feat(cli): doc delete multi-id + api --paginate + paginate fixes + batch deletes
Batch-write surface and pagination consistency:
* weknora doc delete <doc-id> [<doc-id>...] — positional multi-id, default
keep-going on failure. Single -y confirms entire batch.
* weknora session delete <session-id> [<session-id>...] — same shape.
* weknora chunk delete <chunk-id> [<chunk-id>...] --doc <doc-id> — multi-id
with shared --doc parent.
* Multi-id partial-failure rolls up as operation.failed (exit 1), not
server.error (exit 7) — failures are operation outcomes, not transient
transport issues, and the retry-with-backoff hint for server.* would
mislead callers.
* weknora api <path> --paginate — auto-walks offset pagination and merges
pages into a single {data, total} JSON response.
* Paginate truncation fix across 6 list/follower call sites.
* All doc / search / chunk / session / kb list commands migrated to
FormatOptions API.
Multi-id RunE only emits the {ok, failed} envelope when the operation
actually ran — pre-flight failures (e.g. confirmation_required) leave
stdout empty per the wire contract.
doc upload's missing-positional-or-flag check is wrapped as FlagError so
the exit code (2) matches the convention used by other commands that
require a positional argument directly.
|
||
|
nullkey
|
0e081aec5c |
feat(cli): --log-level + kb/agent status & check + cross-cutting refactor
Operability surface and the bulk of the jopts→fopts migration: * --log-level error|warn|info|debug + WEKNORA_LOG_LEVEL env, wired to the SDK via client.SetDebugLevel. Invalid --log-level returns FlagError (exit 2). * kb status <kb-id> / kb check <kb-id> verb split (1 HTTP vs 1+N for failed_count aggregation). * agent status <agent-id> / agent check <agent-id> verb split (probes kb_scope_all_reachable via 1+N HTTP). * kb create <name> positional (matches agent create). * Positional id help strings namespaced (<kb-id> / <agent-id>). * All auth / context / link / doctor / kb / agent CRUD commands migrated to the FormatOptions API. * root.go Execute(ctx) takes a context so signal-cancellation propagates via cmd.Context() into long-running commands. * Pagination termination uses len(accum) >= total (not page*pageSize) so server-capped page sizes do not truncate aggregations. |
||
|
nullkey
|
7eeb3bec5d |
feat(cli): doc wait command (multi-target wait-all)
weknora doc wait <doc-id> [<doc-id>...] blocks until every given document reaches a terminal parse_status (completed / failed), --timeout expires, or the user interrupts (SIGINT). * --timeout DURATION (default 10m; exit 124 on timeout, matches GNU timeout(1) convention) * --interval DURATION (default 2s; exponential backoff to 15s + jitter) * Multi-id polled concurrently (max 5 parallel) * Exit code priority 1 > 124 > 0 (failed > timeout > completed) New typed errors: * operation.timeout → exit 124 * operation.failed → exit 1 * operation.cancelled → exit 1 (main raises to 130 on signal) server.session_create_failed gets a special case in ExitCode to map to exit 1 (workflow failure, not transient retry). doc view and doc download positional id namespaced to <doc-id>. |
||
|
nullkey
|
567d7ac74e |
feat(cli): --format / NDJSON / chat & agent invoke / MCP / SetAgentHelp / signal-aware ctx / kb create --storage-provider
Adds the structured-output and agent-help surface plus root-level signal
handling so AI agents (and humans working through pipes) get a stable
wire contract.
* --format text|json|ndjson flag, registered per-command on outputs that
need it; default text on TTY, json on pipe. --jq <expr> pairs with json
/ ndjson to filter or project. FormatOptions absorbs JQ; WantsJSON()
helper for the JSON dispatch.
* WriteNDJSON helper in internal/format/ (per ndjson.org: one JSON value
per line, arrays split element-per-line, empty slice → zero bytes).
* chat / agent invoke wire --format ndjson via SDK StreamResponse /
AgentStreamResponse 1:1 passthrough. Both commands detect ctx.Cancelled
in every stream + session-create path and emit a stable
"operation.cancelled" code on Ctrl-C / SIGTERM.
* main.go wires signal.NotifyContext(SIGINT, SIGTERM) into the root
context so long-running commands run their cancellation cleanup
(re-emit auto-created session id, etc); the process exits 130 when
the context was signal-cancelled, matching Unix convention.
* MCP chat / agent_invoke output schemas extended with thinking /
tool_calls / assistant_message_id (server-side accumulated; MCP
tools/call has no standard partial-response). doc_view and doc_download
now use doc_id (not knowledge_id) so agents see a single id naming
convention across all tools — matches the chunk_list / search_chunks
schemas and the CLI's <doc-id> positional.
* SetAgentHelp(cmd, AgentHelp{...}) — opt-in machine-friendly --help
payload activated by WEKNORA_AGENT_HELP=1. Applied to chat / kb list.
* kb create --storage-provider <local|minio|cos|tos|s3|oss|ks3> — sets
the new KB's storage_provider_config.provider at creation time (server
does not expose it on update). Required on self-hosted deployments
where the server-side default doesn't pre-populate a provider —
without it, subsequent doc upload returns a misleading "kb not found".
|
||
|
Your Name
|
5deffbd769 |
fix: guard against empty choices and message=None in LLM response
response.choices[0].message.content raises IndexError (empty choices) or AttributeError (message=None on filtered content) during dataset generation. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
taoyang
|
b3004f73d4 |
test(repo): add wiki_config + indexing_strategy to sqlite test DDL
The in-memory SQLite DDL in `knowledgebase_sqlite_test.go` mirrors the
`knowledge_bases` columns the tests need to insert against. It missed two
columns that the `KnowledgeBase` struct has been carrying for a while:
- `wiki_config` (gorm:"column:wiki_config;type:json")
- `indexing_strategy` (gorm:"column:indexing_strategy;type:json")
Without those columns the `INSERT INTO knowledge_bases (...)` statement
GORM emits fails with:
table knowledge_bases has no column named wiki_config
…which cascades to every TestKnowledgeBase_VectorStoreID_* test
failing on a fresh `go test ./internal/application/repository/`.
Reproduce on main (before this commit):
go test ./internal/application/repository/ -count=1
...
--- FAIL: TestKnowledgeBase_VectorStoreID_Save_Immutable (0.00s)
knowledgebase_sqlite_test.go:91:
Received unexpected error:
table knowledge_bases has no column named wiki_config
FAIL github.com/Tencent/WeKnora/internal/application/repository
After this commit:
ok github.com/Tencent/WeKnora/internal/application/repository
The fix is two `TEXT NULL` column additions to the inline DDL. No
production code, no migration, no schema change — strictly aligning the
test scaffold with the model fields that already exist.
|
||
|
wizardchen
|
5141a1230c |
fix(embedding): repair broken comment in Zhipu embedder
A stray `/` line split what should have been `//`, which prevented `go build` from succeeding in CI. |
||
|
Wellbek
|
4fef6826cf | feat(embedding): add Zhipu AI embedder implementation | ||
|
wizardchen
|
ce8969ab14 |
fix(initialization): surface upstream and SSRF errors verbatim in test-connection responses
Test-connection's "连接失败" was nearly useless for debugging because two layers conspired to swallow the actual upstream error: Backend swallow --------------- checkChatModelConnection and CheckASRModel pattern-matched on the upstream error string (401 / 404 / timeout / ...) and replaced it with a fixed Chinese hint. The original SDK message (request URL, response body, provider's error code description) was discarded. Refactor: extract classifyConnectionError(errMsg) — returns just the short Chinese hint. Callers combine "<hint>：<raw err>" so the hint indicates "where to start looking" and the raw error shows "what actually happened". The status-400 special-case (endpoint reachable, parameter mismatch only) still returns success. Frontend swallow ---------------- ModelEditorDialog.testConnection ignored result.message on success paths (fine), but also overwrote it with a generic i18n string on failure paths and 4xx rejections — even though axios's interceptor already lifts the backend's error.message to error.message. Failure branch now uses error?.message || result.message as the toast content; i18n fallback only when neither is present. SSRF allowlist hint ------------------- SSRF validation failures previously returned "Base URL 未通过安全校验: <reason>" with no mention of how to recover (the allowlist is server-side, not configurable in the UI). Operators hit it and were stuck. New utils.FormatSSRFError(label, rawURL, err) wraps the error with guidance to add the host to SSRF_WHITELIST_EXTRA (the operator-facing append-only escape hatch — SSRF_WHITELIST itself is the project's baseline). The hint echoes the failed host so it can be copy-pasted straight into the env var: Base URL 未通过安全校验：DNS resolution failed for hostname test.local.com ... 如该地址确实可信，请联系运维在服务端环境变量 SSRF_WHITELIST_EXTRA 中加入该主机（支持精确域名 / *.example.com 通配 / IP / CIDR），示例：SSRF_WHITELIST_EXTRA=test.local.com, *.example.com,10.0.0.0/8 All SSRF call sites switch to the helper: initialization (CheckRemoteModel / TestEmbedding / CheckRerank / CheckASR / TestMultimodal / batch validation), model CRUD, mcp_service CRUD, knowledge URL ingest, system ReconnectDocReader. |
||
|
wizardchen
|
31d5566a50 |
chore(runtime): silence gin per-route logs and emit env config banner at startup
Two unrelated startup-noise fixes bundled because they share one file (internal/runtime/startup.go) and one motivation (make the boot log useful again). Gin per-route spam ------------------ With ~150 registered routes, DebugMode prints one "[GIN-debug] METHOD path --> handler" line per route, drowning out everything else. Override gin.DebugPrintRouteFunc to a counter and gin.DebugPrintFunc to route through the structured logger. A single "[gin] registered N routes" summary prints once router build is complete. DebugMode behaviour otherwise (panic recovery, runtime warnings) is preserved. Env config banner ----------------- Operators had no easy way to confirm SYSTEM_AES_KEY (or any other critical env) was actually loaded — getting the byte length wrong is particularly silent: utils.GetAESKey() returns nil for anything other than exactly 32 bytes, which simply disables encryption. LogStartupEnv prints one line per curated env var (security + DB + storage + redis), with sensitive values redacted as "set (N chars)" and unset values as "<unset>". Followed by targeted footgun warnings — the AES-256 length check fires loudly when SYSTEM_AES_KEY is set but wrong-length. Both helpers are called from cmd/server and cmd/desktop. The env banner runs before container.BuildContainer so it prints even if DB / storage init fails. |
||
|
wizardchen
|
dc634e863a |
feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling
Two encryption gaps and one fail-loud regression, fixed together because
they are inseparable in practice.
At-rest encryption gaps
-----------------------
Before this change, only Tenant.APIKey, Tenant.Credentials.WeKnoraCloud.AppSecret,
Model.Parameters.APIKey/AppSecret, and WebSearchProviderParameters.APIKey
were encrypted with SYSTEM_AES_KEY. MCP and DataSource were missed:
- MCPAuthConfig.Value/Scan went through bare json.Marshal/Unmarshal, so
api_key and token were written to mcp_services.auth_config (json) in
plaintext regardless of whether SYSTEM_AES_KEY was set.
- DataSource.Config (types.JSON) is a byte passthrough, so connector
secrets (Notion token, GitHub PAT, Feishu app_secret, ...) inside the
Credentials map were stored in plaintext too.
Now both follow the ModelParameters / WebSearchProviderParameters pattern.
Encryption operates on a local copy to avoid mutating the caller's
in-memory struct (otherwise the next read of the same *MCPAuthConfig
would surface ciphertext as if it were the API key). For DataSource the
ToJSON ↔ ParseConfig pair is the only application boundary that
materializes credentials, so encrypting there is sufficient and avoids
having to fork the generic types.JSON.
Backward compatibility
----------------------
DecryptStoredSecret returns rows without the enc:v1: prefix verbatim, so
historical plaintext records keep working. Next write re-saves them
encrypted. No migration step.
Graceful Scan-time degradation
------------------------------
The strict "fail loud on key rotation" behaviour of DecryptStoredSecret
turned into a footgun once it ran from GORM Scan: a single broken row
caused ListModels / ListMCPServices / ListDataSources to return zero
results, hiding all other rows and giving the operator no way to find
which credential needed re-entering.
New utils.DecryptStoredSecretLenient: same logic, but on failure returns
("", false). The strict variant is kept for hot paths that are about to
actually USE the credential (so a rotated key surfaces loudly there
rather than silently degrading to "missing api key").
All Scan paths switch to lenient + per-failure log line:
ModelParameters.Scan, WebSearchProviderParameters.Scan,
MCPAuthConfig.Scan, DataSource.ParseConfig, CredentialsConfig.Scan.
On lenient failure the field blanks out (never ciphertext), the row
still loads, the DTO reports configured=false so the UI shows "not
configured", and any upstream call attempted with the empty value fails
with a clear "missing api key" rather than sending ciphertext as the key.
Tenant.AfterFind (Tenant.APIKey) intentionally stays strict — it is the
primary HTTP auth credential and a silent empty value would change auth
semantics ambiguously.
|
||
|
wizardchen
|
a9d37970ae |
fix(initialization): fall back to stored apiKey when test-connecting an existing model
Direct regression from the previous credential subresource refactor: the edit-mode ModelEditorDialog stopped binding apiKey into formData (the field now lives in <CredentialResource>), so "Test connection" sent apiKey="" and every upstream call (DeepSeek / lkeap / OpenAI etc.) 401'd. Fix in the backend so the frontend stays simple: - ModelTestRequest gains an optional ModelID field. - New fillSecretsFromStoredModel helper: when ModelID is set AND a secret field on the request is empty, copy it from the stored (decrypted) model. Non-empty request values still win, so users typing a new key to verify it before saving behave the same as before. - The four test entrypoints (chat / embedding / rerank / asr) call the helper right after ShouldBindJSON. Frontend: ModelEditorDialog.testConnection forwards modelId when isEdit is true. The four API client signatures accept the optional field. WebSearch and MCP are unaffected: WebSearch already has a TestProviderByID ID-based path, MCP has no standalone raw test endpoint. |
||
|
wizardchen
|
7643dd457e |
refactor(credentials): introduce /credentials subresource pattern
Replace the legacy "redacted placeholder + Clear* boolean" pattern with
dedicated per-resource credential subresources across MCP services,
Models, Web search providers, and Data sources.
Why
---
The previous design had three problems:
1. Main PUT body carried secret fields. The frontend echoed back a
redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty
defense in the service layer tried to detect "user did not change this"
vs "user wants to clear this". A regression in that defense (or a new
frontend forgetting it) silently overwrites the stored secret with the
placeholder.
2. The "remove this credential" UX was a red checkbox under a pre-filled
password input. Three intents (preserve / replace / clear) collapsed
onto one field, and credential changes were bundled with unrelated
config edits in the same submit. Users wiped working keys by mistake.
3. Secret presence was inferred from "did the response come back with a
'***' placeholder", which couples the contract to a magic string.
Design
------
Each of the four resources now exposes:
PUT /{resource}/{id}/credentials # write one or more fields
DELETE /{resource}/{id}/credentials/{field} # clear a single field
"Is this configured?" metadata travels on the main resource response as
a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET
endpoint. The frontend reads the configured boolean from the main GET
and never sees secret values at all.
Main PUT endpoints now ignore any secret fields in the request body and
log a deprecation warning if they appear, so legacy clients fail loudly
rather than overwriting silently.
Frontend
--------
- New reusable <CredentialResource> component renders a three-state card
(unconfigured / configured / editing) and drives the dedicated
endpoints. Used by MCP, Model, Web search; DataSource has a bespoke
card with the same behaviour because its credentials are a single
atomic per-connector map.
- Cancel from edit mode now restores state synchronously from the meta
prop. The previous async refresh() was a no-op while state was still
'editing', leaving the input frozen open.
- Remove is single-click + toast. The danger-themed button is the
deterrent; a modal confirm adds friction without adding safety (the
plaintext is irrecoverable client-side either way — recovery means
re-typing).
DTOs (internal/handler/dto/) are deliberately separate from the GORM
models so "no secret in response" is a compile-time invariant: a future
contributor cannot leak a secret without explicitly adding the field to
the DTO, which is review-able in a single diff.
Storage is unchanged — credentials still live in the existing jsonb /
parameters columns. No schema migration.
Cleanup
-------
- types.MCPService / types.Model / types.WebSearchProviderEntity /
types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret
/ ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData().
- utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty.
RedactedSecretPlaceholder constant is retained because VectorStore
still uses the old pattern and is out of scope here.
- Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat
redaction handling removed; i18n keys renamed secret -> credential.
|
||
|
taoyang
|
f30e9ccd0f |
fix(docreader): bump grpcio floor to 1.78.0 to match generated proto
docreader fails to start on a fresh `uv sync` of main:
$ uv sync --frozen
$ uv run python -m docreader.main
Traceback (most recent call last):
...
File ".../docreader/proto/docreader_pb2_grpc.py", line 19, in <module>
raise RuntimeError(...)
RuntimeError: The grpc package installed is at version 1.76.0, but the
generated code in docreader_pb2_grpc.py depends on grpcio>=1.78.0.
Please upgrade your grpc module to grpcio>=1.78.0 or downgrade your
generated code using grpcio-tools<=1.76.0.
`docreader_pb2_grpc.py:8` was regenerated with grpcio-tools 1.78+ and
hardcodes `GRPC_GENERATED_VERSION = '1.78.0'`, then at import time runs
`first_version_is_lower(GRPC_VERSION, GRPC_GENERATED_VERSION)` and
raises if the installed runtime is older. But `pyproject.toml` still
pins:
"grpcio>=1.76.0",
"grpcio-health-checking>=1.76.0",
"grpcio-tools>=1.76.0",
…and `uv.lock` resolves to grpcio 1.76.0. So every fresh clone +
`uv sync` ends up with a docreader that crashes on import, and a
WeKnora backend that thinks docreader is "not connected" — silently
falling back to the Go-side static engine registry only.
Downstream user-visible symptom: `/api/v1/system/parser-engines`
returns just the Go-side engines. Python-only engines (e.g.
markitdown, plus any custom Python parsers) never appear in the KB
create/edit form. Document parsing falls back to mineru/builtin even
when the user intended to use docreader-backed engines.
Reproduce on main (before this commit):
cd docreader
uv sync --frozen
uv run python -m docreader.main
# → RuntimeError at import, server never starts
After this commit:
Resolved 79 packages
Updated grpcio v1.76.0 -> v1.80.0
Updated grpcio-health-checking v1.76.0 -> v1.80.0
Updated grpcio-tools v1.76.0 -> v1.80.0
...
$ uv run python -c "from docreader.proto import docreader_pb2_grpc; print('ok')"
ok
Fix is the minimum-floor bump on the three grpcio packages, plus the
lockfile update uv emits as a result (all 1.76.0 → 1.80.0, newest
stable matching the proto's floor). No proto regen, no code change —
just aligning the dep declaration with what the generated client
already requires.
The next regen of docreader_pb2_grpc.py (e.g. with grpcio-tools 1.80+)
would raise the floor again; pinning at the proto's declared floor is
the conservative choice that keeps this in sync.
|
||
|
ochan.kwon
|
d869cc1ff6 |
fix: redact sensitive fields and prevent credential loss on edit
Closes #988 Unify the "write-only secrets" pattern across MCP services, Models, WebSearch providers, and DataSources. Fix two related defects: 1. GET responses leaked plaintext credentials — GetMCPServiceByID while List masked, all Model / WebSearchProvider / DataSource GET responses unconditionally. 2. When the frontend echoed a masked placeholder back on save, the update path overwrote the stored secret with the mask — silently wiping the real credential. MCP surfaced this most visibly: ListMCPServices masked Token to "first4****last4" and McpServiceDialog round-tripped that masked string back into the DB. - internal/types/secret.go introduces the shared sentinel RedactedSecretPlaceholder = "***" and two helpers: - IsRedactedOrEmpty(s): true for "" or the sentinel - PreserveIfRedacted(incoming, existing): merge predicate - Each resource adds a RedactSensitiveData (mutates-in-place) or Redacted (returns-a-copy) method and a MergeUpdate contract: empty or placeholder preserves, any other value replaces, and a write-only Clear* bool flag (gorm:"-", json omitempty) wipes with precedence over preserve / replace. Clear* never persists. - Every GET / List / Create / Update response path now redacts. VectorStore's pre-existing "***" literal is refactored onto the shared constant for consistency. All four dialogs — McpServiceDialog, ModelEditorDialog, WebSearchSettings, DataSourceEditorDialog — never pre-fill secret inputs. The stored-state signal is collapsed into the input placeholder: "•••••••• (Enter new value to replace)" when the server reports a stored value, falling back to the connector-specific placeholder otherwise. The input itself carries the "something is there" cue, matching how password dots render. An opt-in "Remove this credential" checkbox appears in edit mode when a value is stored; checking it disables the input and triggers a danger-themed confirmation dialog before submit. DataSource rolls the clear up to a connector-level "Remove all credentials" flag since its credentials are per-connector atomic bundles (e.g. Feishu app_id + app_secret). i18n updates across en-US, zh-CN, ko-KR, ru-RU: secret.storedPlaceholder / clearHint / confirmClearTitle / confirmClearBody. - MCP update handler's manual map-to-struct projection was silently dropping clear_api_key / clear_token / custom_headers on partial updates. Pull them off the map explicitly. - MCP configChanged detection now snapshots URL / StdioConfig / TransportType before the merge so connection recycling still triggers on real transport changes (not just whenever AuthConfig is present). - MCP CustomHeaders: nil preserves existing, non-nil replaces. Was always overwriting, silently wiping headers on any auth-bearing update. - GetMCPServiceByID now redacts (was plaintext while List masked). - WebSearchProvider update preserves Name / Description when empty instead of clobbering them. - WebSearchProvider TestProviderRaw rejects the "***" placeholder up front with an actionable message rather than forwarding it to the upstream provider API as a literal key. - Model builtin hideSensitiveInfo now preserves non-secret metadata (SupportsVision, Provider, ExtraConfig, InterfaceType, ParameterSize, EmbeddingParameters) that were previously zeroed out for builtin model cards. - Replace local equalStringSlices / equalStringMaps helpers with stdlib slices.Equal / maps.Equal (Go 1.21+). - internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go: table-driven coverage of merge, redact, clear, and write-only flag invariants for every resource. - internal/application/service/mcp_service_test.go: full service layer with a fake repo; covers ListMCPServices / GetMCPServiceByID redaction, UpdateMCPService partial-update honoring of Clear*, CustomHeaders preserve-on-nil / replace-on-non-nil, and clear-flag precedence over submitted values. Additive only. No changes to request or response JSON shape beyond the write-only Clear* flags (omitted by default via omitempty). No DB schema changes. No SDK struct field removals. Existing clients that don't know about the new flags continue to function unchanged. - DataSource.Config jsonb at-rest encryption. Pre-existing residual risk; tracked as a follow-up. - Renaming VectorStore.MaskSensitiveFields to match the new RedactSensitiveData / Redacted naming convention. Left alone to minimize diff; a follow-up rename PR can unify. |
||
|
wizardchen
|
a3411899cf |
fix(docreader/auth): harden gRPC TLS/Token rollout from #1359
Follow-up to #1359. Addresses a set of correctness and security gaps in the initial docreader auth implementation. - docker-compose: inject GRPC_TLS_*/GRPC_TLS_SERVER_NAME/GRPC_AUTH_TOKEN into the WeKnora-app service. Without this the Go client never saw the knobs, so enabling token auth on the server broke every RPC. - client: bind tokenAuth.RequireTransportSecurity() to TLSEnabled so a bearer token cannot be sent over an insecure channel once TLS is on. - server: load_tls_credentials now raises TLSConfigError on misconfig (cert/key missing, file unreadable, mTLS without CA); main.py exits 1 instead of silently downgrading to insecure. - server: replace endswith("/Check"|"/Watch") health bypass with exact match against /grpc.health.v1.Health/{Check,Watch}. - server: compare tokens with hmac.compare_digest, warn on tokens < 16B. - server: AuthInterceptor now returns an abort handler matching the original RPC kind (unary/stream) and uses context.abort, so streaming RPCs surface UNAUTHENTICATED instead of INTERNAL. - internal/infrastructure/docparser/grpc_parser.go: drop the duplicated TLS/tokenAuth block and reuse docreader/client.LoadAuthConfigFromEnv + BuildDialOptions. Single source of truth for client-side auth. - Add GRPC_TLS_SERVER_NAME (client SNI override) and GRPC_MTLS_REQUIRE_CLIENT_CERT (server explicit mTLS toggle); document the differing CA semantics between client and server in .env*.example. - Reject half-configured client mTLS (cert XOR key) loudly. - Fix missing trailing newline in .env.lite.example. Verified locally: go build ./... and go vet ./... clean; auth.py fail-fast / token paths smoke-tested. |
||
|
Li Xianggang
|
5a02e22f52 | feat(auth): 连接docreader支持auth | ||
|
nullkey
|
c87e35b34b |
chore(cli): polish + docs sync + pre-PR audit fixes
Code-reuse polish (post-implementation review pass): - Extract text.OneLine(maxWidth, s) helper combining preview-row normalization (newline/CR/tab → space) with text.Truncate's UTF-8-safe truncation. Replaces agent/view.go truncate1Line (ASCII '...' + byte-slice CJK-unsafe) and chunk/list.go singleLine. - Lift cmdutil.OpenInput(path) for the '-' = stdin / else os.Open pattern shared across agent create/edit and the api command. Replaces agent/create.go's private openInput. - Strip inline doc-spec parentheticals from source comments — those belong in commit messages and project docs, not in source where they rot. Pre-PR audit fixes: - doc upload: reject `--metadata` paired with `--from-url` as input.invalid_argument up-front (the URL-ingest request type has no metadata field server-side, so the pair would otherwise silently drop). Long help and CHANGELOG updated to call out the asymmetry. - doc upload (file path): map sdk.ErrDuplicateFile sentinel to resource.already_exists. The sentinel arrives with no "HTTP error <n>:" prefix because the SDK short-circuits on file-hash before reading the HTTP status, so the previous WrapHTTP fall-through misclassified it as network.error with a misleading "check base URL reachability" hint. The --from-url branch already handled ErrDuplicateURL this way; this closes the asymmetry. Caught by e2e re-upload of an already-ingested file; regression test added. - README exit-10 enumeration adds `agent delete` and `chunk delete` (these were missing alongside the v0.5 destructive verbs they were meant to gate). Docs sync: - cli/README.md: command tree now includes the chunk subtree; adds agent / chunk lines to the 5-minute quickstart; adds a "Contributing / Reporting issues" section pointing at the repo's SECURITY.md and AGENTS.md; drops third-party CLI parallels from the surface description. - cli/AGENTS.md: "Command surface design SOP" gains the flag-vs-escape-hatch step. "CRUD command flag canon" renamed to the hard-required-flags pattern with the contrast (TTY-prompts-fill) defined inline rather than via opaque shorthand. - cli/CHANGELOG.md: search docs case-sensitivity shift promoted to its own #### Breaking changes subsection. MCP doc_list filter count corrected from 5 to 6. Drops the bogus go.mod yaml.v3 entry (yaml.v3 was already a dependency on main; v0.5 added zero go.mod lines). Replaces internal-Go identifiers (fuzzyTime, NoOptDefVal) with user-language and drops the § section-symbol jargon. |
||
|
nullkey
|
f89d54362d |
feat(cli): doc/kb resource expansion — upload flags + list filters + view fields
Closes deep-tuning gaps in v0.4-shipped doc / kb / session commands. Each command had multiple SDK fields the CLI silently hardcoded or omitted; this commit threads them through. doc upload: - --enable-multimodel (tri-state via NoOptDefVal): toggle multimodal extraction (PDF/DOCX image-to-text) per upload. Empty explicit value (e.g. --enable-multimodel="" from uninterpolated shell var) rejected as input.invalid_argument rather than silently coerced. - --metadata key=value (repeatable): attach arbitrary metadata - --channel <name> (default api, override for browser / wechat ingests) - URL mode gains --title / --file-type / --tag-id; URL-only flags rejected with input.invalid_argument when used without --from-url doc list filter flags: - --keyword (server-side LIKE — case-sensitive per PG) - --file-type / --source / --tag-id - --start-time / --end-time (RFC3339) search docs switches from client-side substring to server-side Keyword via ListKnowledgeWithFilter — smaller wire payload, but case-sensitivity shifts (documented in CHANGELOG + help text + Long). MCP doc_list schema gains the same 6 filter fields (parity with CLI). session view --full + --limit: loads chat history via LoadMessages SDK method. kb view human KV expanded: type / pinned / temporary / processing state + count / summary model / created timestamp. All omit-empty. doc view human KV expanded: title (when distinct from filename) / description / source / channel / summary_status / enable_status / tag_id / storage (human bytes) / file_hash (12-char prefix). |
||
|
nullkey
|
7bccd72ba3 |
feat(cli): search --all-pages canon catch-up + AGENTS.md SOP / CRUD canon
Brings search docs and search sessions to v0.4 pagination canon (--limit / --page-size / --all-pages, matching session list / doc list). Both default --all-pages=true to preserve prior silent walk-all behavior; explicit knobs added for users who want one-page fetch. cli/AGENTS.md gains two new sections: - Command surface design SOP — a 5-step SDK-schema-first pre-design checklist for future contributors. Earlier spec drafts produced schema-error classes (missing/mismatched fields, missing pagination flags) when commands were designed from convention rather than from the SDK; the SOP makes the SDK the ground truth. - CRUD command flag canon — Mode A (hard-required + flag error, no interactive prompts), the established pattern for non-auth CRUD. Also fixes the agent invoke rationale source: the CLI-layer precedent for invoke being a separate verb (not a chat mode) is documented inline rather than referencing other vendor CLI behavior. |
||
|
nullkey
|
26fa43e2cc |
fix(cli): post-audit fixes (MCP MatchCount + sessions + auth + view)
Four unrelated shipped-code drifts found during v0.5 audit cycles: 1. MCP search_chunks tool omitted MatchCount from SearchParams. Server fell back to its default cap; agents asking for limit:50 silently got fewer results. Adds MatchCount: limit to the struct literal. 2. search sessions printed UpdatedAt as raw RFC3339 while session list used a fuzzy "X hours ago" render — same SDK field, two human renderings. Switches to the shared text.FuzzyAgoStr helper for parity. 3. auth status --json omitted three operationally-meaningful AuthUser fields (username, is_active, can_access_all_tenants). Agents branching on can_access_all_tenants previously needed a second round-trip. 4. session view Long help claimed the SDK doesn't wrap session_messages; it does (LoadMessages / GetMessagesBefore / GetRecentMessages all exist in client/message.go). Rewrites the comment to be accurate. |
||
|
nullkey
|
5b07c9ab87 |
feat(cli): chunk subtree + MCP chunk_list tool + curation rationale
New subtree (chunk list / view / delete) exposes RAG retrieval debugging primitives with SDK-grounded field set (23 Chunk fields). Pagination follows v0.4 canon: --limit / --page-size (1..1000) / --all-pages. - chunk list --doc <id>: enumerate by ChunkIndex (separate from search chunks which is hybrid retrieval; Long help documents the distinction) - chunk view <id>: scope-less render via /chunks/by-id route; full content verbatim - chunk delete <id> --doc <id>: scope-flag + scope-id; L-13 destructive; 404 NOT idempotent; resource.not_found / auth.forbidden / input.confirmation_required typed exit codes documented in Long help MCP server gains chunk_list as 10th curated tool. Schema deliberately exposes only doc_id + limit (no pagination workflow on MCP); response includes truncated_at_limit flag when total > limit. cli/AGENTS.md MCP curation rationale rewritten: curated read-only is a deliberate product call because the server side does not yet enforce per-token scope. When server scope ships, mutation tools can land in the MCP surface. Shared helper cli/internal/text/timeago_string.go (FuzzyAgoStr) extracted from session list during the C2 quality-review pass. |
||
|
nullkey
|
59132a56f6 |
feat(cli): agent CRUD + view full config rendering
Adds the three management verbs missing from v0.4's agent subtree (create / edit / delete) and expands v0.4-shipped agent view to render all 34 AgentConfig fields in human output (was 7). Surface: hot-path flags (--model required + 7 optional) + --config-file YAML/JSON tail + --generate-skeleton template emit. Flag > file > server-default precedence for hybrid invocation. - agent create <name> --model <id> [flags] + --from <agent-id> for copy-then-overlay (CopyAgent + UpdateAgent); preserves source config except for fields explicitly overridden - agent edit <id> with --add-kb / --remove-kb idempotent pair, L-2 fetch-then-update, at-least-one-flag validation, --description "" clearing via Flags().Changed(). --config-file fully replaces the AgentConfig baseline (use surgical flags for partial edits; the Long help spells this out + a test pins the contract). - agent delete <id> with ConfirmDestructive + exit-10 protocol; 404 propagates resource.not_found (not idempotent) - agent view: 10 grouped sections (Identity / LLM / KB attachment / Retrieval / Query rewrite / Tools / FAQ / Web search / Multi-turn / Fallback / Templates); --json field discovery includes all config.* keys Shared helper cli/internal/cmdutil/agentconfig.go handles YAML/JSON parsing, flag-overlay-file fusion, and skeleton emission. |
||
|
Claude
|
b7f9f155b3 |
fix(knowledge): prevent infinite scroll race condition causing missing documents
The knowledge base file list infinite scroll had a race condition: - page++ executed before the condition check, causing page numbers to be skipped when scroll events fired rapidly - No loading guard meant concurrent API requests could overlap, with later responses overwriting earlier ones This caused documents to not load when scrolling to the bottom of the file list, especially noticeable with 100+ files where only the first page would reliably appear. Changes: - Add scrollLoading flag to prevent concurrent scroll-triggered fetches - Move page++ inside the condition check (was unconditional before) - Use resetPage() helper to consistently reset both page and loading state - Change scroll trigger threshold from >= scrollHeight to >= scrollHeight - 10 for better trigger sensitivity |
||
|
wolfkill
|
5a81441637 | fix(model): hide Ollama tip for remote models | ||
|
weitong
|
a27f12a6cb | 验证完毕 | ||
|
hjz
|
06feee0fee | feat: modify the skills registration directory | ||
|
nullkey
|
69fbbfc252 |
chore(cli): gofmt sweep
`gofmt -w cli/` — trailing newlines, minor whitespace alignment. Caught during PR-readiness review (`gofmt -l .` had been non-clean since the em-dash → ASCII bulk replace earlier in the v0.4 series). |
||
|
nullkey
|
f2e8e3f56c |
refactor(cli): drop aiclient package; align AGENTS.md with mainstream
Survey of 10 mainstream CLIs (gh, lark, stripe, vercel, supabase, aws, azure, gcloud, openai/codex, github-copilot-cli) showed env-gated per-command --help blurbs are a Stripe-only pattern; gh uses env detect for telemetry only, and lark relies on installed agent Skills + MCP. Our cmd/mcp/serve already covers the dominant 2025/26 path, so internal/aiclient/ (136 LOC + 38 callsites) is net maintenance burden without precedent. - Drop internal/aiclient/ entirely (annotations + detect + tests) - Remove 38 SetAgentHelp callsites + agentAwareHelpFunc / SetHelpFunc wiring in cmd/root.go - Migrate 4 command-level rules to standard Long help (visible to all, not env-gated): doc upload mode mutex, kb edit at-least-one, kb pin idempotent, search chunks channel mutex - Rewrite AGENTS.md as a developer guide (gh-style 6 H2 / 167 lines): audience preamble + Build / Architecture / Command Structure / Testing / Code Style / Error Handling. Drops sections absent in surveyed projects (Commit & PR Conventions, Who Uses This CLI) - Clean 14 internal doc refs (ADR-N, spec §X, v0.X) in source comments and docs that pointed at docs/superpowers/ — that directory is local-only / uncommitted, so refs are dead for outside readers - Drop forward-looking "once v0.2 ships" from README |
||
|
nullkey
|
a0dd989c81 |
refactor(cli): auth security audit — gh CLI parity hardening
Compared the auth subtree (login/logout/list/status/refresh/token)
against gh CLI's auth implementation. Three gaps closed:
1. `auth login --with-token` validates the API key against `/auth/me`
before persisting (mirrors gh's pre-persist GetCurrentLogin probe).
A typo'd / expired / wrong-host key fails fast with
`auth.bad_credential` (exit 3) and nothing is written to the
keyring. Side benefit: api-key contexts now carry the resolved
`user` + `tenant_id` at rest, so `auth list` reflects who owns
the key — previously these columns were blank for `--with-token`
contexts because we never queried the server.
2. `auth login` prints a stderr advisory when the secrets store falls
back to the 0600 plaintext file (keychain unavailable — typical on
headless CI, WSL without DBus, agent containers). `weknora doctor`
carried the same info in its credential_storage check, but users
who go straight to `auth login` could miss it. gh has the same
silent-fallback gap; we're stricter here.
3. AGENTS.md adds an "Auth security contract" section documenting:
- Credential storage (keychain primary, 0600 file fallback)
- `--with-token` reads stdin (not flag value), pre-validated
- No env-var token bypass — by design, to avoid the
`/proc/<pid>/environ` / `ps -E` leak surface that
`GH_TOKEN`-style env vars expose
- `auth status` / `auth list` never emit token values
- `auth refresh --json` returns only `{context}` (never the
new tokens)
- `auth token` stdout has no trailing newline + TTY stderr hint
- `auth logout` is local-only (no server-side revocation)
Verified against gh CLI behavior (cli.github.com manual + cli/cli
trunk source):
| dimension | gh | weknora v0.4 |
|---------------------------------|----------------|--------------|
| pre-persist token validation | ✓ | ✓ (new) |
| OS keychain primary | go-keyring | go-keyring |
| stderr warning on file fallback | ✗ silent | ✓ (new) |
| `auth status` default token | masked prefix | not shown |
| `auth token` TTY warning | ✗ | ✓ |
| env-var token bypass | ✓ (GH_TOKEN) | ✗ by design |
| process-args / `ps` leak surface| ✗ stdin only | ✗ stdin only |
|
||
|
nullkey
|
e623e8208f |
refactor(cli): delete envelope infrastructure, errors to stderr
Removes the entire envelope machinery now that every success path
emits bare JSON:
- cli/internal/format/envelope.go (Envelope, Success, Failure,
SuccessWithRisk, WriteEnvelope, Meta, Notice, UpdateNotice,
VersionSkewNotice, Risk, RiskLevel, ErrorBody) + tests.
- cli/internal/format/filter.go envelope-specific helpers
(WriteEnvelopeFiltered, marshalEnvelope, applyFieldFilter,
filterDataPayload, filterObjectData); the reusable
filterArrayItems / filterObjectKeys / writeJQ stay for bare.go.
- cli/internal/cmdutil/exporter.go + tests (envelope-only).
- cli/internal/cmdutil/PrintErrorEnvelope + ToErrorBody +
operationRiskOf + Error.OperationRisk field + OperationRisk struct.
Error path: all errors now go to stderr via cmdutil.PrintError in
`code: message\nhint: ...` form, regardless of --json. Stdout stays
empty (or holds the partial-success the command already wrote) so
downstream `--json | jq` pipelines never have to filter error shapes
out of the success stream. Typed exit codes (3 auth.* / 4
resource.not_found / 5 input.* / 6 server.rate_limited / 7 server.*
+ network.* / 10 input.confirmation_required) carry the failure
class for agents that branch on it.
Acceptance contract:
- envelope_test.go → wire_test.go (TestEnvelopeGolden → TestWireGolden).
- testdata/envelopes/ → testdata/wire/.
- Error-path cases assert the typed code substring on stderr.
- Orphan whoami.*.json goldens deleted.
AGENTS.md + README.md rewritten for the bare-data contract:
- Drop envelope schema section + dry-run rule.
- Document bare JSON on stdout + `code: msg\nhint: …` on stderr.
- ADR-3 reframed around bare data and why error separation matters
for `--json | jq` pipelines.
WriteJSONFiltered short-circuits to WriteJSON when both filters are
empty (skip the marshal-buffer round-trip for the common case).
Final review pass:
- Fix wire-contract bug: `--json id,name` (space form) is broken by
pflag's NoOptDefVal; AGENTS.md / README.md / SetAgentHelp + the
field-discovery help text all switched to `--json=id,name`.
- Fix `weknora api --jq` silently ignored: api.go now routes through
WriteJSONFiltered with jopts.JQ.
- AGENTS.md: drop the false claim that `auth logout` honors `-y`
(logout is local-only with no ConfirmDestructive guard); list the
actual destructive commands instead.
- Rewrite cli/acceptance/e2e/e2e_test.go for the bare-data wire shape
(was still parsing `out["data"]` / `env["ok"]`).
- Add `JSONOptions.Emit(w, v)` helper; collapse ~33 repeated
`format.WriteJSONFiltered(iostreams.IO.Out, X, jopts.Fields,
jopts.JQ)` sites to `jopts.Emit(iostreams.IO.Out, X)` — drops the
format import from 22 cmd/* files.
- Delete single-caller `cmdutil.MustRequireFlag`; inline as
`_ = cmd.MarkFlagRequired(...)` everywhere.
- Add `_ = cmd.MarkFlagRequired("name")` to `kb create`; it was the
only write command relying on runtime --name validation while
`context add` already used the cobra-level mark.
- `context use`: register `--json` / `--jq` (was always emitting JSON
unconditionally with no human path and no flag — diverged from
every other write command); human mode now prints
`✓ Switched context to X (was Y)`.
- Replace per-package `confirmPrompter` / `scriptedConfirm` /
`errPrompter` test doubles with `testutil.ConfirmPrompter`.
- Rename `chatService` → `ChatService` (export to match siblings
`ListService` / `ViewService`); rename `printUploadSuccess` →
`renderUploadSuccess` (siblings use `render*`).
- `defaultHint(CodeResourceNotFound)`: drop the hardcoded
"list available with `weknora kb list`" — misleading on agent /
doc / session 404. Replaced with "verify the resource ID and try
again".
- Strip stale `v0.2/v0.3` / "envelope" / "v0.0/v0.1 supports only"
historical tags from production comments and a few test
descriptions.
|
||
|
nullkey
|
cc8254f862 |
refactor(cli): drop --dry-run + introduce bare-JSON output path
Two intertwined mainstream-alignment moves bundled because they share
the migration target (every command's --json path):
1. Drop --dry-run entirely. Survey of comparable API-wrapper CLIs
(gh, aws, stripe, lark): none expose --dry-run. The mainstream that
does (kubectl/git/helm/ansible) operates on declarative manifests
or local state where the preview is materially different from the
executed action. WeKnora's CLI just echoed the same parameters
that would have gone on the wire — the preview added no real
signal over `--help` + reading the call site. Removes:
- root --dry-run persistent flag + cmdutil/dryrun.go
- DryRun fields + EmitDryRun calls in 12 write commands
- format.Envelope.DryRun field
- 8 corresponding *_test.go cases
- --dry-run mention from README.md and CHANGELOG.md
- "dry_run":false from 16 golden envelopes
2. Migrate every --json output to bare data:
- New format.WriteJSON / WriteJSONFiltered helpers
(cli/internal/format/bare.go) share filterArrayItems /
filterObjectKeys / writeJQ with the (still-live for now) envelope
filter helpers.
- Read commands (kb/doc/session list+view, search chunks/docs/
sessions/kb, auth list/status, agent list/view, context list,
doctor) emit bare arrays / objects on stdout.
- Write commands (kb create/edit/delete/pin/empty, doc upload/
upload_recursive/delete, session delete, auth login/logout/
refresh/token, link/unlink, context add/use/remove, agent
invoke, chat, api, version) emit bare result objects. Risk
classification dropped — the resource + exit code already
convey the action.
Per-command shape changes:
list / search → []T (was {ok, data:{items:[…]}})
view → T (was {ok, data:T, _meta:…})
create / edit → T
delete / pin / etc. → {id, …action result…}
doctor → {summary, checks}
api → {status, headers, body}
_meta dropped on the read path:
pagination (page/page_size/total/has_more) — agents iterate with
--all-pages or accept --limit (gh CLI parity);
kb_id / context echo — caller already knows what it asked for.
Acceptance contract goldens regenerated for the new bare shape.
Error envelope on stdout (PrintErrorEnvelope) stays live for now —
the envelope-infra deletion lands in the next commit.
|
||
|
nullkey
|
bdc589e1c0 |
refactor(cli): --limit/--all-pages, Go 1.26, internal/agent → aiclient
Cross-cutting cleanup that lands alongside the new feature surface: - `--limit / -L` and `--all-pages` on every list command. Default --limit 30 (gh-parity); --all-pages drains every server page client-side, capped by --limit. Closes the audit finding that the old "1000 max per call" implicit cap was undiscoverable. - `auth token` emits a TTY-only stderr advisory when stdout is a terminal (the credential just got displayed in scrollback) plus an api-key-mode rotation hint. - Comment + doc discipline pass: drop external project name references from in-code comments (we reference them in design notes, not inline). - Bump `go` directive to 1.26.0 and CI matrix to 1.26.x to align with the main module's go.mod. - Rename `cli/internal/agent` → `cli/internal/aiclient` to disambiguate from the new `cli/cmd/agent` resource subtree. The package handles AI coding-agent env detection + per-command --help annotations; the new name reflects that more precisely. |
||
|
nullkey
|
9bb83b47fd |
feat(cli): mcp serve curated stdio MCP server
`weknora mcp serve` — long-lived stdio MCP (Model Context Protocol) transport that exposes a fixed, curated tool surface to MCP-aware agents (Claude Desktop, Claude Code, custom MCP clients). Curated tool set (readonly by default): - whoami — active context + tenant - search (hybrid retrieval against a KB) - kb list / view - doc list / view - agent list / view / invoke - session list / view The list is intentionally narrow to the read + agent-invoke surface; destructive verbs (`delete` / `empty` / `upload`) are gated behind `--write`. Schema is built from each leaf cobra command's flags so adding a new tool is a single registry entry plus a Service interface. Includes the simplify post-review polish + a second simplify pass to fold the resulting feedback (typed schemas, agent_help wording, unify chat / agent invoke option names). |
||
|
nullkey
|
493fc41e98 |
feat(cli): agent subtree (list/view/invoke)
Manages WeKnora's first-class Custom Agent resources — server-side records (system prompt + model + allowed tools + KB scope) that the user authored in the web UI. Commands: - `weknora agent list` — tenant-visible agents (built-in + custom), sorted updated_at desc; `--limit`/`-L` caps the slice client-side. - `weknora agent view <id>` — full sdk.Agent including nested AgentConfig (mode / model / allowed_tools / KB scope). Human mode prints a compact KV layout + Config: block. - `weknora agent invoke <agent-id> "<text>"` — streams the agent's configured workflow against a query over SSE. Auto-creates a fresh session unless `--session` is passed. Streaming defaults to TTY + no-stream/no-json; agent-friendly buffered single-object output with `--json` (or `--no-stream`). Decoupled from the existing `chat` subtree: agents bring their own system prompt / tool surface / KB selection, so the chat / agent split matches the server-side resource boundary. |
||
|
nullkey
|
3b67986863 |
feat(cli): per-resource filter flags on list commands
Adds the filter flags users were reaching for via `--jq` post-filter: - `kb list --pinned` — client-side filter to KBs with `IsPinned`. - `doc list --status <pending|processing|completed|failed>` — server-side query-param filter; `failed` surfaces ingestion errors immediately for triage. - `session list --since <duration>` — client-side filter to sessions updated within the past duration. Accepts time.ParseDuration forms (24h, 1h30m, 30m) plus a `<N>d` suffix for whole days (7d, 0.5d). Server-side filters are forwarded as query params (where the API supports them) to avoid pulling the full list into memory; client-side filters apply after the fetch so they compose with `--limit`. |
||
|
nullkey
|
1b20b06f5e |
feat(cli): --json field-select, --jq, auth token, doc --from-url
Output ergonomics:
- `--json` accepts a comma-separated field list (gh-parity); selects
named keys from the per-command payload. Bare `--json` keeps the
full shape.
- `--jq <expr>` evaluates a gojq expression over the JSON; pairs with
`--json field-list` so projection runs before jq.
- `--version` is a global cobra flag in addition to the `version`
subcommand; both render the same line.
- Per-command `--help` now renders the available JSON field list under
"JSON fields available via `--json id,name,...`" (field-discovery
parity with gh / kubectl `-o jsonpath`).
New commands:
- `auth token` — print the active context's credential to stdout for
shell command substitution (`WEKNORA_TOKEN=$(weknora auth token)`).
Default: raw secret, no trailing newline. `--json` emits
`{token, mode, context}`.
- `doc upload --from-url <URL>` — ingest a remote URL via the SDK
`CreateKnowledgeFromURL`. `--name` forwarded as `FileName` so the
server's known-extension heuristic upgrades crawl-mode to
file-download-mode where appropriate.
Includes the simplify post-review polish pass (field-filter unit
tests, --json/--jq compose check, agent_help copy fixes).
|
||
|
wizardchen
|
4fb089d4d7 |
fix(kb): map ErrKnowledgeBaseNotFound to 404 across handler helpers
Five handler helpers (validateAndGetKnowledgeBase in knowledgebase.go,
validateKnowledgeBaseAccessWithKBID in knowledge.go, the kbService
guards in faq.go and tag.go, and getKnowledgeBaseForInitialization +
the per-kb config getter in initialization.go) wrapped every
GetKnowledgeBaseByID error — including the well-known
repository.ErrKnowledgeBaseNotFound sentinel — as
NewInternalServerError. The result was that every probe of a stale or
cross-tenant kb id surfaced as a 500 instead of the 404 it should have
been, both confusing clients ("real 5xx vs. wrong URL") and burning
ops attention on monitoring alerts.
The mapping pattern is the same as PR #1336 for sessions: detect the
sentinel via stderrors.Is and emit NewNotFoundError; everything else
still surfaces as a 500 so genuine DB / repo failures keep firing the
alerts that matter. Caught during the RBAC e2e smoke run on
feat/rbac, where a deliberate cross-tenant kb-id probe produced
HTTP 500 + body "knowledge base not found" — the smoking gun.
Tests: new internal/handler/knowledgebase_not_found_test.go covers
three cases — bare sentinel, fmt.Errorf("%w") wrapped sentinel
(regression guard against a future revert to `==`), and a non-sentinel
infrastructure error that must still 500. All three pass.
The full handler test package is green.
|
||
|
wizardchen
|
3a0a0cff9c |
fix(migration): ensure pg_trgm is created before trigram index in 000041
Migration 000002 creates the pg_trgm extension only inside the conditional embeddings block guarded by app.skip_embedding. Deployments that use a non-postgres retrieve driver (Qdrant, Elasticsearch, Milvus, etc.) skip that block entirely, so pg_trgm is never installed. When migration 000041 then runs CREATE INDEX ... USING GIN (lower(title) gin_trgm_ops) the statement fails and the whole migration is rolled back, leaving task_pending_ops and task_dead_letters uncreated. The application keeps starting (migrations are best-effort), but wiki ingest enqueue silently fails and wiki pages are never produced. Re-issue CREATE EXTENSION IF NOT EXISTS pg_trgm at the top of 000041 so the extension is guaranteed present at the moment the trigram index is created. The statement is idempotent on environments where 000002 already installed it, and surfaces a clear, early error on environments where the extension genuinely cannot be loaded. Fixes #1319. Co-authored-by: Cursor <cursoragent@cursor.com> |