Commit Graph

1715 Commits

Author SHA1 Message Date
wizardchen 854cb0c2a1 feat(ui): redirect tenant switch away from tenant-scoped routes
Switching the active tenant triggers window.location.reload() to
re-key every cached store / open SSE stream / in-flight request.
The catch: when the user is currently sitting on a route whose URL
embeds a tenant-scoped resource id, the new tenant lands on a
404/403 and the user sees a blank screen.

Three risky route shapes today:

  /platform/knowledge-bases/:kbId
  /platform/knowledge-bases/:kbId/creatChat
  /platform/chat/:chatid

Add a tiny tenantSwitch utility (navigateAfterTenantSwitch) that
matches the current path against this list and redirects to
/platform/knowledge-bases — the platform default landing, always
renderable regardless of role. Anything else (settings, agent
list, KB list, etc.) continues to reload its current path because
those views handle empty/forbidden data gracefully.

Both tenant-switch call sites (TenantSelector and UserMenu) use
the helper so the matrix lives in one place; future tenant-scoped
routes only need to add a regex to the list.
2026-05-18 17:28:58 +08:00
wizardchen 8c0510f2e1 fix(api): reject /agent-chat early when agent_id is missing
AgentQA used to trust request.AgentEnabled even when
resolveAgent() returned nil (no agent_id, or unresolvable id) — the
request reached the async goroutine, AgentQA service errored with
'custom agent configuration is required', and the user saw a broken
SSE stream with no actionable error.

Most likely root cause is a stale frontend settings store where
isAgentEnabled stayed true after selectedAgentId got blanked
(typically a cross-tenant switch where the previously selected
agent isn't visible in the new tenant). The frontend fix lives in
selectAgent() syncing both flags, but the handler shouldn't trust
that contract.

Add an early 400 with a clear message before executeQA dispatches.
Logs the actual request.AgentID (sanitized) so future stuck-state
reports are diagnosable from a single log line.
2026-05-18 17:28:58 +08:00
wizardchen 619e14a2eb fix(rbac): wire chunk-id ownership chain for generated-question delete
DELETE /chunks/by-id/:id/questions used to be gated at flat
Contributor+ as a deliberate carve-out: the URL only carries the
chunk's own id, so the chunk -> knowledge -> KB ownership chain
wasn't reachable from request params alone. The carve-out had two
problems:

  - Inconsistent with every other chunk mutation. PUT
    /chunks/:knowledge_id/:id is OwnedKBOrAdmin (KB creator OR
    Admin+); a Contributor with no KB ownership can't update a
    chunk but COULD delete a generated question on the same chunk.
  - User-visible: a Contributor in tenant A who never created any
    KB still saw the 'delete question' button work, while every
    other chunk control 403'd.

Wire the missing hop:

  KBCreatorLookupFromChunkIDParam (handler) reads :id, fetches the
  chunk via the existing GetChunkByIDOnly (unscoped repo call),
  re-checks chunk.TenantID against the caller's tenant context to
  block cross-tenant chunk-id probing, then defers to the existing
  resolveKBCreatorByKnowledgeID helper for the rest of the chain.

  OwnedChunkKBOrAdminFromChunkID (router) wraps it in
  RequireOwnershipOrRole(Admin, ...) — same matrix as every other
  chunk/knowledge/wiki mutation guard.

  chunks.DELETE switches from g.Contributor() to the new guard.

4 unit tests cover happy path, chunk-not-found, cross-tenant
chunk-id probing, and KB-not-found upstream — same edge-case shape
as the existing KBCreatorLookup_* tests so future refactors get
caught loudly.
2026-05-18 17:28:58 +08:00
wizardchen f7a264ca93 fix(rbac): hide UI mutation surfaces for Viewer / non-creator
Several mutation entry points stayed visible to users who lacked the
backend-required role, so Viewers (and non-creator Contributors) hit
403 on click instead of seeing the disabled state. Surgical fixes,
all matching the existing backend matrix:

  - KnowledgeBaseList: gate the entire 3-dot popup (both the 'all'
    and 'mine' tabs) behind canManageKBCard. Pin/unpin used to be
    unguarded; settings/delete were per-item gated but the trigger
    still showed. Now non-managers don't see the popup at all.

  - AgentList: the popup trigger now requires at least one menu item
    to be visible (edit OR copy OR disable OR delete), via a compound
    v-if on the popup itself. Disable/enable on own custom agents
    and shared agents both gain a hasRole('admin') guard to match
    POST /shared-agents/disabled, which writes a tenant-level
    preference and is Admin-gated.

  - AgentEditorModal: new readOnly prop hides the '确定' (save)
    button and replaces 'Cancel' with 'Close'. AgentList passes
    readOnly when the user can't manage the agent (built-in agents
    and others' custom agents for Viewers). Form-input disabling is
    intentionally NOT wired here yet — the modal has 3000+ lines of
    inputs and the only mutation surface that 403s is the save
    button. Hiding it removes the broken path without rewiring the
    whole form.

  - doc-content: generated-question delete button gated by a new
    canEditKB prop driven by KnowledgeBase.vue's existing canEdit
    computed (KB creator OR Admin OR org-share-edit). Defaults to
    Admin-only when the prop is missing — fail-closed since the
    backend matrix is now stricter (see follow-up commit).

The server still enforces the same matrix on every mutation; these
gates only stop the user from staring at a 'failed' toast.
2026-05-18 17:28:58 +08:00
wizardchen 1e8b5c8d40 refactor(api): drop deprecated tenant ConversationConfig field and KV write path
The Tenant.ConversationConfig field was marked deprecated when
CustomAgent (builtin-quick-answer) replaced the per-tenant defaults
path, but the JSONB column, GET/PUT KV handlers, write helpers and
read fall-throughs all stayed wired. This had two concrete failure
modes:

  - Input-field.vue silently PUT-ed per-user model picks back into
    /tenants/kv/conversation-config, which requires Admin+ — Viewers
    and Contributors got a 403 every time they switched chat models.
  - The same write also overwrote the tenant-shared default for every
    other user, turning a per-user UX into a per-tenant footgun.

Remove the field, the struct, the validate helper, the PUT case and
the two dead Summary helpers in session/helpers.go. KnowledgeSearch
loses its tenant-conversation fallback (one chunk-tool reader); the
global config + hardcoded defaults still cover it. session_agent_qa
already required agents to declare their own rerank_model_id, so no
behaviour change there.

Frontend follows:
  - Input-field.vue: per-user 'last selected chat model' moves to
    localStorage, no API call.
  - AgentEditorModal: defaults for builtin-quick-answer creation now
    come from prompt-templates (system/context/rewrite/fallback) and
    retrieval-config (top_k / threshold), the two endpoints that
    actually own those settings now.
  - AgentSettings.vue: orphan settings page with no router/import
    referencing it; deleted.
  - api/system: getConversationConfig / updateConversationConfig and
    the ConversationConfig interface go away.

Net: 137 insertions, 2761 deletions.
2026-05-18 17:28:58 +08:00
wizardchen b0b7cfa351 refactor(settings): group sidebar nav items into labeled sections
The settings sidebar grew to 13 entries — flat scroll made related
settings (e.g. model + parser + storage) hard to find. Group them by
intent (basic / AI / data / workspace / system) with section titles
above each cluster, mirroring how popular SaaS settings pages are
organised. Behaviour and routing unchanged.
2026-05-18 17:28:58 +08:00
wizardchen 0e1482bc4c fix(rbac): propagate TenantRole through logger.CloneContext
logger.CloneContext rebuilds a fresh context.Background() and only
copies a hand-curated allow-list of context keys (LoggerContextKey,
TenantIDContextKey, UserContextKey, etc.) before passing it to
downstream handlers — the chat / agent QA pipelines all do
`ctx := logger.CloneContext(c.Request.Context())` at the top of
parseQARequest.

TenantRoleContextKey was missing from that allow-list. Any call site
that read the role via TenantRoleFromContext(reqCtx.ctx) saw the
type-zero TenantRole and the helper's least-privilege fallback
returned Viewer. The auth middleware had set role=Owner correctly
on the parent context, but the cloned context lost it.

Symptom: an Owner of their home tenant got 403 on POST /agent-chat
when the agent had runnable_by_viewer=false, because the gate at
session/qa.go:520 saw role=viewer despite middleware logging
role=owner two ms earlier.

Diagnostic log lines added in the previous commits (auth.go:145
"[auth] resolved role=...", auth.go:283 "[auth] resolveTenantRole
step1 hit row_role=...") made this immediately visible — a request
showed role=owner at the middleware layer and role=viewer at the
handler layer, which can only mean ctx propagation lost it. Keeping
the diagnostic logs in place so similar regressions surface fast.

Fix is one line: add TenantRoleContextKey to the CloneContext
allow-list, with a comment pinning it to the bug it fixes so a
future drive-by removal has to weigh that loss.
2026-05-18 17:28:58 +08:00
wizardchen f832bce16d docs(rbac): document Contributor vs OwnedXxxOrAdmin selection rule
Adds a checklist comment at the top of internal/router/rbac.go that
explains how to pick between the four role-only guards
(Viewer / Contributor / Admin / Owner) and the ownership-or-role
guards (OwnedKBOrAdmin and friends) when wiring a new route.

The two questions encoded here:

  Q1. Does the resource have a creator?
        YES  → mutating routes use OwnedXxxOrAdmin
        NO   → tenant-wide infra → Admin()
        ENTRY POINT (creating a brand-new owned resource) → Contributor()

  Q2. Is the side effect "private to me" or "visible to others"?
        PRIVATE → Contributor() is fine (e.g. POST /agents/:id/copy)
        PUBLIC  → OwnedXxxOrAdmin or Admin (sharing, transferring,
                  tenant-wide toggles)

The recurring failure mode caught in this PR — FAQ entries, tag CRUD,
KB share, agent share, /shared-agents/disabled all gated to plain
Contributor while the matrix demanded OwnedXxxOrAdmin — is the
"sub-resource drift" anti-pattern: someone wires a new sub-resource
with a freshly-invented gate instead of inheriting from the parent
KB/Agent's matrix.

Also documents the user-facing invariant the guards together produce
("Contributor in their own KB acts like Owner; Contributor in someone
else's KB acts like Viewer") so reviewers have a UX-level sanity check
beyond the role labels.
2026-05-18 17:28:58 +08:00
wizardchen fb947205b7 fix(rbac): align FAQ and tag mutating routes with KB ownership matrix
FAQ entries (CRUD on /knowledge-bases/:id/faq/entries) and KB tags
(/knowledge-bases/:id/tags) were gated to plain Contributor+, while
the KB main routes plus chunks/wiki sub-resources require
OwnedKBOrAdmin (creator OR Admin+). The mismatch let any Contributor
mutate FAQ entries and tags inside someone else's KB even though
they could not edit the KB's other content.

Both are KB child resources by design: FAQ entries are the body of
FAQ-type KBs, and tags categorise KB content. Touching them affects
the KB owner's organisation of their KB, so they belong on the same
matrix as chunks and wiki pages — Contributor with KB ownership can
manage their own, others need Admin+.

Sweep checked router gates and handler/service tenant scoping for
the rest of the surface (Models / MCP / WebSearchProvider /
VectorStore / DataSource / WeKnoraCloud / IM channels). All Admin+
routes correctly load tenant_id from context, and per-id mutating
service calls go through (id, tenantID) primary keys, so a Viewer
cannot reach those even with a guessed cross-tenant id.
2026-05-18 17:28:58 +08:00
wizardchen 96de7e3976 fix(rbac): gate sharing routes; tenant-disable shared agent → Admin+
Three more mutating routes were ungated:

- POST /shared-agents/disabled — sets the "disabled by me" preference
  for a shared agent. Despite the "by me" naming this writes a
  tenant-level row (tenant_disabled_shared_agents) that hides the
  agent from every member of the tenant's chat dropdown. So a Viewer
  could silently nuke an agent for the entire tenant. Now Admin+
  only.
- POST /knowledge-bases/:id/shares — sharing a KB to an organization
  exposes its content to every member of that org. The service layer
  only checked "is this KB inside the caller's tenant", so any Viewer
  could share any tenant KB. Gate matches the KB-update matrix:
  OwnedKBOrAdmin (creator OR Admin+).
- POST /agents/:id/shares — same problem, same fix:
  OwnedAgentOrAdmin. PUT /shares/:share_id and DELETE /shares/:share_id
  for both KB and agent share routes get the same matrix — changing
  permission or revoking a share is at least as sensitive as
  creating one.

GET /knowledge-bases/:id/shares and GET /agents/:id/shares stay
Viewer+ since they're read-only.

The /organizations/* routes are intentionally left unchanged: they
implement organization-level RBAC (org owner / admin / editor /
viewer) inside the service layer, which is a different matrix from
tenant RBAC and not in scope for this PR.
2026-05-18 17:28:58 +08:00
wizardchen 0a44077498 fix(rbac): plug ungated mutating routes; log RBAC state at startup
Sweep over the route table found four classes of mutating endpoints
that had no role guard:

- POST /tenants — creating a brand-new tenant. Now g.CrossTenant():
  only org-level superusers may create tenants, matching /tenants/all
  and /tenants/search.
- POST /initialization/initialize/:kbId, PUT /initialization/config/:kbId
  — these change a KB's embedding/parser/storage configuration, which
  is at least as sensitive as PUT /knowledge-bases/:id. New
  KBCreatorLookupFromKbIDParam (sister of KBCreatorLookup but reading
  :kbId) lets us reuse the OwnedKBOrAdmin matrix; routes now use the
  new g.OwnedKBOrAdminFromKbIDParam() guard.
- POST /initialization/{ollama,remote,embedding,rerank,asr,multimodal,
  extract}/* — system-level model probes / downloads / extraction
  pipelines. These trigger LLM calls and outbound network fanout
  with tenant credentials; gated to Admin+. The corresponding GETs
  (status / list / progress) drop to Viewer+ since they're read-only
  observability.
- POST /system/{parser-engines,docreader,storage-engine}/* — same
  pattern: GETs that report state are Viewer+, the *-check /
  /reconnect actions are Admin+.

POST /evaluation/ runs the eval harness which fans out LLM calls
across the tenant — gated to Admin+.

Operational improvement: log the resolved tenant RBAC state at
LoadConfig completion. air's hot-reload only rebuilds the binary on
Go-source changes; it does NOT re-source .env, so flipping
WEKNORA_TENANT_ENABLE_RBAC=true while the dev loop is already
running has no effect until the parent dev-app process restarts.
This was a real diagnostic loop in PR review — printing the value at
startup makes the "I edited .env but the gates still aren't firing"
trap obvious from the first console line.

The Printf includes both the resolved cfg.Tenant.EnableRBAC value
and the raw env var, so a mismatch between "what the .env file
says" and "what this process actually inherited" is one glance.
2026-05-18 17:28:58 +08:00
wizardchen b9a1ce7617 fix(rbac): drop X-Tenant-ID short-circuit in SSE / fetch paths
The previous PR removed the
"selectedTenantId === weknora_tenant.id → skip header" short-circuit
from the axios interceptor, but the same copy-pasted logic lived in
three more places that bypass axios:

- `api/chat/streame.ts` — `fetchEventSource` for the streaming chat
  endpoint. Without this fix the chat reply would 404 with
  "流式连接失败:HTTP 404" on the first message after switching to
  a non-home tenant: the SSE call landed on the home tenant where
  the freshly-created session did not exist.
- `api/initialization/index.ts` — native `fetch` for multimodal
  test (FormData) requests. Misroutes the test to the home tenant.
- `utils/security.ts` — `getProtectedFileRequestHeaders` used by the
  Markdown image proxy. Authenticated file fetches landed on the
  home tenant, breaking image rendering after a tenant switch.

All three now mirror the simplified rule: if `weknora_selected_tenant_id`
is set, attach `X-Tenant-ID` unconditionally. The server's
`IsTenantAccessible` already accepts a header pointing at the user's
home tenant, so always attaching is safe.

Same root cause as the axios fix: any code path that writes the
active tenant into `weknora_tenant` (UserMenu's `loadUserInfo`, OIDC
sync, router hydrate) made the two values equal, after which these
non-axios paths silently dropped the header and re-routed to home.
2026-05-18 17:28:58 +08:00
wizardchen 1a46fcf8ca fix(rbac): role-aware UI gates leaked write affordances after tenant switch
When an Owner of tenant A invited user B as a Viewer, B could log in,
switch the active tenant from B's home to A via the user menu, and
still see every write button — Create KB, Upload Document, Edit/Delete
agent — because the role-gate plumbing on the client was wired to the
wrong tenant. The server's RBAC enforcement (PR 2 / PR 5) still blocked
the actual mutations with 403, so no data leaked, but the UI presented
authority the user did not have, which is a confusing and embarrassing
defect.

Five distinct root causes, all on the client / server boundary:

1. `currentTenantRole` (frontend/src/stores/auth.ts) read
   `tenant.value.id` — the user's *home* tenant — instead of the active
   tenant id. After switching to A, the helper still returned B's home
   role (Owner), so every `hasRole(...)` gate in the codebase silently
   short-circuited. Fix: read `selectedTenantId` first, fall back to
   `tenant.id`.

2. `setMemberships` was only called once, on /auth/login. Hard reloads
   (which the tenant switcher triggers) re-ran `getCurrentUser()` via
   `App.vue` and `router/index.ts`, but those callers discarded the
   `memberships` field of the response, so the auth store kept the
   login-time snapshot forever. Role changes between sessions stayed
   invisible. Fix: extract memberships in both /auth/me consumers and
   feed them into `setMemberships`.

3. `KnowledgeBase.vue`'s `isOwner` compared `kb.tenant_id` to the
   active tenant id, so any KB inside the current tenant was treated
   as "owned by the caller" — bypassing every role gate below. Fix:
   compare against `kb.creator_id` (the field PR 5 introduced) and
   require an actual creator match. `canEdit` / `canManage` now layer
   `hasRole('admin')` on top so a non-creator with sufficient role can
   still manage.

4. `KnowledgeBaseList.vue` and `AgentList.vue` rendered the per-card
   more-menu (Settings, Delete, Edit) unconditionally. With the
   ownership column from PR 5 this is straightforward to gate on the
   client: Settings/Delete now require creator-match or `hasRole('admin')`
   per the server's `OwnedKBOrAdmin` / `OwnedAgentOrAdmin` matrix.

5. The axios interceptor (frontend/src/utils/request.ts) had a
   "selectedTenantId === defaultTenantId → don't attach X-Tenant-ID"
   short-circuit. Any code that wrote the active tenant into
   `weknora_tenant` (UserMenu's loadUserInfo, OIDC sync, router
   hydrate) made the two values equal, after which the header silently
   stopped attaching and subsequent navigation requests landed on the
   home tenant. Fix: drop the short-circuit; always attach when
   selectedTenantId is set.

Server-side, /auth/me read `user.TenantID` (the immutable home tenant
id stamped at signup) when fetching the tenant for the response, so
the `tenant` field always pointed at home even after the auth
middleware had already resolved an active tenant from X-Tenant-ID.
The frontend then re-keyed `authStore.tenant` to the home tenant on
every refresh, undoing the switch. Fix: read the active tenant id from
context (`types.TenantIDFromContext(ctx)`) and only fall back to
user.TenantID when context is unset.

The server-side enforcement was correct throughout — every fix here is
purely about not lying to the user with the rendered UI.
2026-05-18 17:28:58 +08:00
wizardchen 766a6c482f feat(rbac): per-tenant audit log for RBAC events (PR 6, #1303)
PR 1-5 of the multi-tenant RBAC series enforce the role matrix; what
they don't do is keep a durable record of who did what. Today the
RBAC middleware logs "[rbac] role insufficient ..." to stderr -
perishable, not searchable. Operators cannot answer "who removed
Alice from tenant T-foo on Tuesday" or "did we 403 a probing user
200 times last hour" after the fact.

This PR adds a generic per-tenant audit log that:

- Captures mutating RBAC actions (member add/remove/role-change/leave)
  durably from the tenant_member service after the repo write succeeds.
- Captures enforcement rejections (RequireRole / RequireOwnershipOrRole
  denies under EnableRBAC=true) via a gin-context-injected hook.
- Sliding-window dedup (1 minute) on (tenant, actor, action, path)
  for denied events so a probing client cannot flood audit_logs.
- Default-on - no extra env flag. Schema is generic enough for future
  KB / agent / datasource events to reuse via new action namespaces
  without another migration.

Schema (migration 000044) mirrors wiki_log_entries' shape: a single
audit_logs table with action, target_type/id, request path/method,
outcome, and a JSONB details column. Three indexes: (tenant_id, id
DESC) for the per-tenant cursor feed, actor_user_id for "what did
Alice do", (tenant_id, action) for both the action filter and the
dedup lookup. Sqlite mirror added for Lite mode.

Wiring is nil-safe: the audit field on tenantMemberService and the
gin-context lookup in rbac.go both tolerate a nil service so existing
tests construct without a stub and dormant deployments degrade
gracefully if the provider is misconfigured.

The query API is GET /tenants/:id/audit-log, gated by Admin+ on top
of the existing PathTenantMatch (denial histories should not surface
to ordinary members). Cursor-paginated by descending id (sidesteps
the duplicate-timestamp tie-breaking that BIGSERIAL trivially solves).

Frontend: TenantMembers.vue gets a t-tabs wrapper with Members
(existing) and Audit log (new) tabs; the audit tab is gated to
Admin+ to mirror the server, lazy-loads on first open, paginates via
the next_cursor pattern, and renders coloured chips per
action/outcome so an operator can scan for anomalies. i18n keys
added in all four locales.

Tests: 14 new tests across service (6), middleware (4), and handler
(4). Service tests pin the dedup window behaviour (writes again
after expiry, dedup is per-actor-per-path, degrades on lookup
error). Middleware tests pin both reject sites' audit hook and the
"dormant mode does not fire" + "nil audit does not panic"
invariants. Handler tests pin the cursor envelope and that an empty
page returns next_cursor=0.

Out of scope: per-action ACL on the audit log, retention/TTL job,
KB/agent/datasource event wiring (schema supports it - just no
emitters yet), async writes, frontend filter UI.
2026-05-18 17:28:58 +08:00
wizardchen d42a41b784 feat(rbac): per-KB ownership for knowledge / chunks / wiki pages (#1303)
PR 1–4 of the multi-tenant RBAC series gated KB-level and Agent-level
mutations behind a "creator OR Admin+" check via OwnedKBOrAdmin /
OwnedAgentOrAdmin. The layer underneath — knowledge entries, chunks,
wiki pages — was still gated only by Contributor, which is wider than
intended: any tenant Contributor could edit/delete documents in a
private KB they didn't own. PR 2 / wiki author left explicit follow-up
breadcrumbs at the three call sites in router.go acknowledging this gap.

This PR closes those breadcrumbs by reusing the existing scaffolding:

- KnowledgeService.GetOwningKBCreatorID walks knowledge_id -> kb_id ->
  KB.CreatorID, tenant-scoped through GetKnowledgeByID. The chain lives
  in the service layer so the handler-side lookups stay pure adapters.
- Three new CreatorLookup closures (KBCreatorLookupFromKnowledgeID,
  KBCreatorLookupFromKnowledgeIDParam, KBCreatorLookupFromKBPath) are
  added in handler/rbac_lookups.go alongside the existing
  KBCreatorLookup / AgentCreatorLookup. Each translates the repository
  sentinels into middleware.ErrResourceNotFound and re-checks tenant
  scope as defence-in-depth, mirroring KBCreatorLookup exactly.
- router/rbac.go gains three guard factories — OwnedKnowledgeKBOrAdmin,
  OwnedChunkKBOrAdmin, OwnedWikiKBOrAdmin — wired through the same
  rbacGuards bundle.
- Mutating routes in RegisterKnowledgeRoutes / RegisterChunkRoutes /
  RegisterWikiPageRoutes switch from g.Contributor() to the new
  ownership guards. Cross-KB batch operations (POST /knowledge/move,
  POST /knowledge/batch-delete, PUT /knowledge/tags) stay
  Contributor-gated since there's no single owning KB to check
  against. KB-scoped uploads (POST /knowledge-bases/:id/knowledge/...)
  reuse the existing OwnedKBOrAdmin because the URL :id is the KB id
  directly. The chunks.DELETE("/by-id/:id/questions") route addresses
  chunks by their own id with no knowledge id in the URL, so it stays
  Contributor-gated as a documented carve-out.

No schema changes: documents inside a shared KB are KB-owned content,
not personal uploads — that's how the product is positioned and matches
the original "per-KB-ownership granularity" wording from the PR 2
breadcrumbs. Legacy/built-in KBs with empty CreatorID surface as
tenant-owned (Admin+ only), same rule as built-in agents.

Tests:
- 9 new test cases in handler/rbac_lookups_test.go covering the three
  new lookups: NotFoundMapsToSentinel (each repository sentinel),
  CrossTenantIsHiddenAsNotFound, OwnerMatchReturnsCreatorID,
  MissingTenantContext (fail-closed), and ChunkRouteShape (param-name
  regression guard).
- 5 new test cases in service/knowledge_owner_chain_test.go covering
  the chain helper: knowledge-not-found, kb-not-found, nil-kb-without-
  err, happy path, and legacy KB with empty CreatorID.

Build, vet, and the targeted ./internal/middleware/...
./internal/handler/... ./internal/router/...
./internal/application/service test runs are clean. Pre-existing
TestCreateStore_* / TestOssEnsureBucket_CreateFails failures on main
are unrelated and reproduce on a clean checkout.
2026-05-18 17:28:58 +08:00
wizardchen 466182c8d6 feat(frontend): tenant switcher in user menu + role-aware UI gates
PR 4 of #1303 wired the backend X-Tenant-ID gate to accept active
tenant_members rows (not just superusers), but the frontend never
exposed a switcher to ordinary multi-tenant members. The only
existing component, the sidebar TenantSelector, is gated to
canAccessAllTenants and queries /tenants/search — both wrong for a
Contributor that an Owner just invited to a second tenant.

This PR closes both gaps:

1. Switcher entry inside the user-avatar dropdown
   - New "Switch Tenant" submenu in UserMenu.vue, hover-triggered, with
     a teleported floating panel (same pattern as the existing IM
     overview submenu so the layout/animation read consistently).
   - Data source is authStore.memberships, populated by /auth/login.
     No new API call: the JWT-issued claim already enumerates every
     tenant the user is an active member of plus their role.
   - Visibility: only shown when memberships.length > 1. Cluster
     superusers keep using the sidebar TenantSelector for "any tenant
     in the system" — the avatar entry is the curated "tenants I'm a
     member of" list.
   - Click writes via authStore.setSelectedTenant(...), then
     window.location.reload() so every cached store / open SSE
     stream / in-flight request is re-keyed under the new tenant.
     Same approach as the existing TenantSelector.

2. Role-aware UI gates on high-friction surfaces
   - authStore exposes a new hasRole(min: 'viewer'|'contributor'|
     'admin'|'owner') helper (numeric ordering matches
     middleware/rbac.go) so v-if gates line up with the server-side
     matrix instead of duplicating ['admin','owner'].includes(...).
   - Gates applied per the PR 4 audit:
     * KB list "Create" button — Contributor+
     * Agent list "Create" button — Contributor+
     * Model / VectorStore / WebSearch / MCP — Add+Edit+Delete+Test
       are Admin+ (and the per-card action menus return [] for
       non-Admins so the dropdowns disappear, not just disable).
     * Parser / Storage engine drawers — Save button Admin+
     * Retrieval settings — slider/selector inputs disabled for
       non-Admins (the form auto-saves on change, no dedicated Save).
     * API key reset — Owner+ (Reset is the highest-impact tenant op).
     * IM channel toggle/edit/delete + Add — Admin+. Same in the
       avatar-menu IM submenu (toggle disabled for non-Admins).
   - KB / agent per-card edit/delete already key off kb.isMine /
     agent.isMine + (for builtins) is_builtin; that path is left as-is.

3. i18n
   - Three new keys under tenant.switcher.* in en-US/zh-CN/ko-KR/ru-RU.
   - Role labels reuse the existing tenantMember.role.* bundle from
     PR 3, with a "show raw role" fallback if a locale lacks the key.

The server is the source of truth for every gated action — these
changes only hide buttons that would 403, never grant new authority.
authStore.hasRole and currentTenantRole both carry a SECURITY note
saying they read from localStorage and are UI-rendering-only.

vue-tsc --build error count unchanged from baseline (90, all
pre-existing issues unrelated to this PR).
2026-05-18 17:28:58 +08:00
wizardchen 9e1c2c2546 refactor(tenant_member): simplify tenant ID resolution in handlers
Removed the resolveTenantIDFromPath method from TenantMemberHandler, consolidating tenant ID parsing directly into the ListMembers, AddMember, UpdateMemberRole, and RemoveMember methods. This change streamlines the code by utilizing the existing parseTenantIDFromPath function, enhancing readability and reducing redundancy. No functional changes were introduced, maintaining the existing behavior of tenant ID resolution.
2026-05-18 17:28:58 +08:00
wizardchen 2082b63951 refactor(rbac): centralise tenant-access guards in middleware
Tenant-access decisions previously lived in four places with subtly
different shapes: canAccessTenant in middleware/auth.go, an unexported
isCrossTenantSuperuser in middleware/rbac.go, authorizeTenantAccess +
two opening if-blocks in handler/tenant.go, and resolveTenantIDFromPath
in handler/tenant_member.go. Each variant had its own logging, its
own treatment of the cluster-wide EnableCrossTenantAccess flag, and
its own subtle difference in what "URL :id matches active tenant"
meant.

Collapse all four into middleware/access.go:

- IsCrossTenantSuperuser(ctx, cfg): exported, flag-aware. Both the
  per-user CanAccessAllTenants attribute AND
  cfg.Tenant.EnableCrossTenantAccess must be true; either alone
  rejects, so flipping the cluster flag off operationally revokes
  every superuser without a token re-issue.
- IsTenantAccessible(ctx, user, target, memberService, cfg): home
  tenant + superuser path + active tenant_members row. Replaces the
  old superuser-only canAccessTenant; the X-Tenant-ID gate now also
  honours ordinary multi-tenant memberships.
- RequireCrossTenantAccess(cfg): route guard for /tenants/all and
  /tenants/search, replacing the 12-line if-blocks at the top of
  ListAllTenants and SearchTenants.
- RequirePathTenantMatch(cfg): route guard for /tenants/:id/...,
  replacing authorizeTenantAccess (4 callers in tenant.go) and
  resolveTenantIDFromPath's superuser carve-out (5 endpoints in
  tenant_member.go). The router declares it once on the /tenants/:id
  group so the rule lives next to the route definitions.

router/rbac.go grows two factory methods (CrossTenant,
PathTenantMatch) so RegisterTenantRoutes stays declarative and reads
the same as the existing g.Viewer() / g.Owner() / g.OwnedKBOrAdmin()
guards. The /tenants/:id endpoints now group under one
PathTenantMatch() middleware instead of each handler re-deriving the
check.

Handler cleanups follow naturally: tenant.go drops
authorizeTenantAccess and the cross-tenant if-blocks; tenant_member.go
drops the cfg field and shrinks resolveTenantIDFromPath to a thin
parser. NewTenantMemberHandler loses its *config.Config argument;
container/container.go's dig wiring picks up the simpler signature
automatically.

middleware/access_test.go covers all four exports with truth-table
cases (flag/attribute/membership combinations, lookup errors, missing
context). tenant_member_test.go's existing handler-level cross-tenant
assertions still pass because memberTestRouter now mounts
RequirePathTenantMatch the same way the production router does.

Net -41 lines across 8 changed files + 2 new files; no behaviour
change other than the X-Tenant-ID gate accepting active memberships
(which was the original PR 4 goal that motivated this refactor).
2026-05-18 17:28:58 +08:00
wizardchen 543e9709be fix(rbac): address PR 3 + leave-endpoint review feedback
Security
- Reject URL :id when it does not match the caller's active tenant on
  every /tenants/:id/members/* and /tenants/:id/leave endpoint. Without
  this an Owner of tenant A could drive operations on tenant B simply
  by changing the URL, because RequireRole resolves the caller's role
  against the active tenant from context, not the URL param. The
  cross-tenant superuser carve-out mirrors middleware/rbac.go
  (CanAccessAllTenants + EnableCrossTenantAccess required).
- AddMember no longer records the synthetic "system-<tenantID>" caller
  (X-API-Key auth path) as invited_by, matching the same filter
  KB.CreatorID got in PR 2.
- AuthHandler now logs a loud one-shot error when constructed with a
  nil/incomplete config so a wiring bug that silently disables the
  invite_only gate is visible at boot.

Race fixes
- New repo helpers DemoteOwnerAtomically / RemoveOwnerAtomically wrap
  the count-other-owners + write step in a transaction that holds
  SELECT … FOR UPDATE locks on every other active Owner row. The
  service routes Owner demotion / Owner removal through them, closing
  the TOCTOU window where two concurrent demotes of two different
  Owners could each observe count=2 and both commit, leaving the
  tenant ownerless.

Performance
- ListMembers hydration now batches user lookups via a new
  GetUsersByIDs interface method (one SELECT … WHERE id IN (…)
  instead of N round-trips). Failure degrades to "no email/username"
  rather than dropping rows.

Frontend
- canManage also fires for cross-tenant superusers
  (authStore.canAccessAllTenants), mirroring the backend carve-out so
  the UI controls match what the server actually allows.

Tests
- Cross-tenant URL guard regression cases for List / Add / Update /
  Remove / Leave.
- Superuser bypass tests (flag on -> service reached, flag off -> 403).
- Batched lookup test ensures GetUsersByIDs is called exactly once and
  GetUserByID is never used as a fallback.
- Synthetic caller invited_by filter regression test.
- Service-layer atomic last-Owner regression tests for both
  UpdateRole and RemoveMember.
2026-05-18 17:28:58 +08:00
wizardchen ae7cd02cd6 fix(rbac): polish tenant member UI; add /leave endpoint
Self-review of PR 3's TenantMembers.vue surfaced a handful of real bugs
and a few rough edges. This commit fixes them and adds the self-leave
endpoint that PR 3's design called out as a follow-up.

Frontend bug fixes
- Read err.status (not err.response.status). The axios interceptor in
  utils/request.ts already flattens errors into { status, message,
  ... }, so the original branch-by-status code never matched and every
  4xx fell through to the generic toast. Fixes 404/409 surfacing as
  "userNotFound" / "alreadyMember" / "lastOwner" instead of "Something
  went wrong".
- t-dialog @confirm replaces :on-confirm so the dialog stays open on
  validation failure and on API error. Users can fix and retry without
  retyping the email.
- Drop the t-select revert hack; with :model-value the rendered value
  is already source-of-truth-on-row.role, so no setTimeout dance is
  needed.
- Don't toast "no active tenant" on cold mount; watch the activeTenantId
  ref and load once the auth store hydrates.

Frontend UX additions
- Empty state via t-empty (also covers the search-no-results case).
- Compact role-permissions matrix above the table, mirroring
  OrganizationSettingsModal so Owners can see what each role grants
  before promoting/inviting.
- Search box filtering by username/email (client-side, sufficient for
  expected per-tenant member counts).
- "Leave Tenant" button — visible to any active member except the
  last-Owner case (server still enforces).

Backend
- New POST /tenants/:id/leave route. Calls TenantMemberService.RemoveMember
  with the caller's own user id so non-Owners can quit on their own.
  Gated Viewer+ in the matrix; the existing last-Owner invariant in the
  service still rejects last-Owner self-leave with 409.
- 3 new handler tests cover the happy path, the last-Owner 409 path,
  and the missing-caller-id 401 path (defensive — auth middleware
  always attaches a user, but the test pins the contract).

i18n
- Add tenantMember.permissions / search / leave / empty strings in all
  four locales (en/zh/ko/ru).
2026-05-18 17:28:58 +08:00
wizardchen 3f7150cb5c feat(rbac): tenant member management + invite_only gate (PR 3 of 3, #1303)
Wire the human-facing surface for the tenant-RBAC system that PR 1
introduced and PR 2 enforced: REST endpoints + UI for managing tenant
members, plus the long-pending invite_only registration gate.

Backend
- internal/handler/auth.go: Register now rejects with 403 when
  auth.registration_mode=invite_only. PR 1 already exposes the mode via
  /auth/config so the frontend hides the sign-up link; this is the
  server-side enforcement for direct API calls.
- internal/handler/tenant_member.go: new TenantMemberHandler with
  ListMembers / AddMember / UpdateMemberRole / RemoveMember. Maps
  service-layer sentinels onto HTTP semantics (ErrLastOwner -> 409,
  ErrMembershipAlreadyExists -> 409, ErrUserNotFound on AddMember -> 404,
  ErrInvalidTenantRole -> 400). AddMember requires the invitee to
  already have an account; SMTP-driven email invites are deferred.
- internal/types/tenant_member.go: new TenantMemberResponse projection
  carrying the fields the management UI renders (email, username,
  avatar, role, joined_at).
- internal/router/router.go: four routes under /tenants/:id/members
  gated per the matrix (Viewer+ for list, Owner+ for every mutation).
- internal/container/container.go: dig provider for the new handler.

Tests
- handler/tenant_member_test.go: 12 table-driven cases covering happy
  path, dangling-user tolerance, ErrLastOwner -> 409, ErrMembershipNotFound
  -> 404, ErrMembershipAlreadyExists -> 409, unknown email -> 404,
  invalid role rejected before user lookup, bad tenant id -> 400.
- handler/auth_register_invite_only_test.go: invite_only -> 403,
  self_serve -> 201, nil Auth config falls through.

Frontend
- src/api/tenant/members.ts: typed client (TenantRole/TenantMember/...)
  for the four endpoints.
- src/views/settings/TenantMembers.vue: members table with add dialog,
  inline role select, confirm-remove modal. All mutation widgets gated
  behind currentTenantRole === 'owner'; server still enforces.
- src/views/settings/Settings.vue: register the Members tab; show only
  when currentTenantRole is loaded (avoids tab flash on cold nav).
- i18n in en-US/zh-CN/ko-KR/ru-RU: new tenantMember namespace covering
  all UI strings and structured error messages (lastOwner / userNotFound
  / alreadyMember / invalidRole etc).

This completes the #1303 RBAC rollout. With cfg.Tenant.EnableRBAC=true
operators now have a full management surface; with EnableRBAC=false
the routes still exist but mutations log-and-pass per PR 2 contract.
2026-05-18 17:28:58 +08:00
wizardchen 0f0f7e4339 test(rbac): cover review gaps from PR 2 self-review
- Add TestRequireRole_CrossTenantSuperuserBypass: pin the
  CanAccessAllTenants -> Owner-gate bypass behaviour for RequireRole,
  matching the existing coverage on RequireOwnershipOrRole. Catches
  reorderings of the fast paths that would re-lock org-level
  superusers out of foreign-tenant Owner-only routes.

- Add TestAgentCreatorLookup_CrossTenantIsHiddenAsNotFound: lock down
  the defence-in-depth tenant re-check inside AgentCreatorLookup so a
  future refactor that loosens the service-layer scope (or wires a
  cross-tenant variant by mistake) cannot leak access via the
  ownership shortcut.

- Reword RequireOwnershipOrRole step 7-8 doc comment so "role check
  already failed" is unambiguous (it refers to step 1 above).

- Annotate the synthetic-system-user construction in middleware/auth.go
  with a back-pointer to types.IsSyntheticUserID so anyone changing
  the prefix or numeric suffix knows to update the helper too.
2026-05-18 17:28:58 +08:00
wizardchen 8b71def60b fix(rbac): address PR 2 review feedback
Middleware
- RequireOwnershipOrRole now skips the creator lookup entirely when
  EnableRBAC is off, eliminating a hidden per-request DB roundtrip in
  the rollout window.
- New ErrResourceNotFound sentinel lets lookups signal "not visible to
  this tenant"; the middleware passes the request through so handlers
  can return a real 404 instead of a misleading 403.
- Transient lookup failures now surface as 503 (was 403) so monitoring
  can tell denied-by-policy from cannot-verify-right-now.
- Cross-tenant superusers (CanAccessAllTenants) bypass tenant role gates,
  restoring DELETE /tenants/:id and ResetAPIKey for org operators that
  resolve to Admin in foreign tenants.
- Construction-time warning fires once when cfg is nil/incomplete, so a
  wiring bug that disables enforcement globally is visible at boot.

Route coverage
- Add role guards to FAQ, KnowledgeTag, WikiPage, Skill, WebSearch,
  WebSearchProvider, WeKnoraCloud, Session, Chat, and Message route
  groups so the matrix actually matches the PR description.

Service layer
- KB and agent creators are no longer recorded as the synthetic
  system-<tenantID> user used by the X-API-Key path; resources stay
  tenant-owned. New types.IsSyntheticUserID helper centralises the
  prefix check.
- KBCreatorLookup and AgentCreatorLookup re-check tenant scope so a
  same-ID resource in another tenant cannot piggyback through the
  ownership shortcut.

Docs
- CustomAgent.RunnableByViewer comment now matches enforcement: it only
  blocks Viewer-driven agent-mode runs, not plain RAG/Wiki QA.

Tests
- New cases for the 503/404 paths, lookup-skip behaviour, cross-tenant
  superuser bypass, the lookup helpers (not-found, cross-tenant,
  built-in agent, missing tenant context), and IsSyntheticUserID.
2026-05-18 17:28:58 +08:00
wizardchen 9e3c92ca95 feat(rbac): enforce tenant role matrix on routes (PR 2 of 3, #1303)
Build on PR 1's tenant_members schema by enforcing the role matrix on
every authenticated route and at agent run time. Enforcement is gated
behind cfg.Tenant.EnableRBAC so the change ships dormant — the same
code paths log the would-be rejection during the rollout window and
start blocking once operators flip the flag.

Middleware (internal/middleware/rbac.go):
- RequireRole(min, cfg): role-only gate using TenantRole.HasPermission.
- RequireOwnershipOrRole(min, lookup, cfg): allows the original creator
  through even when their role is below min, otherwise falls back to
  the role check. Lookup errors fail closed; an empty creator_id is
  treated as tenant-owned and requires the role.
- Both honour cfg.Tenant.EnableRBAC for the rollout window.

Creator lookups (internal/handler/rbac_lookups.go):
- KBCreatorLookup resolves :id -> KnowledgeBase.CreatorID.
- AgentCreatorLookup resolves :id -> CustomAgent.CreatedBy; built-in
  agents (IsBuiltin=true) return ("", nil) so they remain Admin+ only.
- Compile-time guards keep the methods aligned with CreatorLookup.

Route guards (internal/router/rbac.go + router.go):
- rbacGuards bundles the matrix into Viewer/Contributor/Admin/Owner
  helpers plus OwnedKBOrAdmin / OwnedAgentOrAdmin. Each Register*
  function takes *rbacGuards so route lines remain readable.
- KB / Agent / Knowledge / Chunk / Tenant / Model / MCPService /
  VectorStore / DataSource / IMChannel routes annotated per the PR 1
  design doc matrix. Tenant-wide infra (models, vector stores, MCP,
  data sources, IM channels) reads as Viewer+ and writes as Admin+;
  per-resource mutations on KB / Agent route through the
  ownership-or-Admin guard.

Resource ownership (services):
- CreateKnowledgeBase / CopyKnowledgeBase populate CreatorID from the
  authenticated user, so the OwnedKBOrAdmin guard has a creator to
  match.
- CreateAgent / CopyAgent populate CreatedBy similarly. Built-in
  agents stay tenant-owned.

Agent run-time gate (internal/handler/session/qa.go):
- AgentQA refuses to launch the agent engine for Viewers when the
  target agent has RunnableByViewer=false. Same EnableRBAC gating as
  the middleware so it's a no-op until enforcement is turned on.

Tests:
- 12 unit tests cover RequireRole / RequireOwnershipOrRole, including
  the fail-open and fail-closed branches and the lookup-error path.
2026-05-18 17:28:58 +08:00
wizardchen af23483433 fix(auth): honour JWT tenant_id claim in middleware
POST /auth/switch-tenant mints an access token whose `tenant_id` claim
points to the target tenant, but the middleware previously discarded
that claim entirely:

    user, err := userService.ValidateToken(ctx, token)  // returned only *User
    targetTenantID := user.TenantID                     // always the home tenant

The session therefore stayed scoped to the user's home tenant despite
the supposedly fresh token, making the new endpoint a no-op once
the request hit any tenant-scoped query.

This change widens the contract:
- UserService.ValidateToken now returns (user, activeTenantID, error)
  where activeTenantID is the JWT's `tenant_id` claim, falling back to
  user.TenantID when the claim is absent (backward compatibility with
  pre-RBAC tokens still in the wild).
- middleware/auth.go uses activeTenantID as the default target before
  applying any X-Tenant-ID header override, and recomputes
  crossTenantSwitch from `activeTenantID != user.TenantID` so the
  resolveTenantRole branches stay correct for switch-tenant sessions.
- The non-middleware /auth/validate handler in handler/auth.go absorbs
  the new return value (it only cares about user identity).

Tests:
- internal/application/service/user_jwt_test.go pins the JWT->tenant-id
  projection (`tenantIDFromClaims`) — claim present beats fallback,
  zero/negative/string claims fall back, int64/uint64 are accepted for
  test-built tokens.

Required for PR 2 multi-tenant integration tests, where a single user
needs to switch between Owner-in-A and Viewer-in-B sessions and observe
distinct authority on each side.
2026-05-18 17:28:58 +08:00
wizardchen f3d1210577 refactor(rbac): address PR 1 review feedback
Tighten correctness, security and observability of the tenant RBAC
foundation introduced in 3c4cdd70 in response to internal review.

Bugfixes
- custom_agents.runnable_by_viewer: drop the GORM `default:true` tag.
  GORM treats a plain bool's zero value as "unspecified" and silently
  rewrites it to the column default on insert, which would prevent
  admins from ever creating an agent with viewer-runtime disabled. The
  column-level DEFAULT TRUE on the database side still backstops rows
  inserted by raw SQL / migrations.
- TenantMemberRepository.HasAnyMembers: replace `Limit(1).Count(&n)`
  with `Take(&probe)`. GORM's Count ignores Limit and scans the whole
  table, which matters because this runs on the auth middleware's
  hot path for users without a cached membership.
- service/user.LoginWithOIDC: include User, Tenant and Memberships in
  the OIDCCallbackResponse so the OIDC flow returns the same shape as
  password login (previously User and friends were left nil).

Security
- middleware/auth.resolveTenantRole: restrict orphan-tenant
  auto-promotion to the caller's home tenant. Cross-tenant superuser
  visits and JWTs minted for foreign tenants no longer trigger silent
  Owner grants. Two regression tests pin the new behaviour.
- service/user.synthFallbackMembership: default the synthesised role
  to Viewer instead of Owner. The login response only feeds UI; the
  backend re-derives the real role from tenant_members on every
  request, so showing a Viewer UI is preferable to a misleading Owner
  UI that would surface admin controls the backend will then 403.
- migrations/versioned/000043_tenant_rbac.down.sql: gate the destructive
  drop behind a `weknora.allow_destructive_migration` GUC so an
  accidental `migrate down` cannot wipe role data in production.
- frontend/src/stores/auth.ts: comment currentTenantRole as
  rendering-only so future PRs cannot mistake the localStorage-backed
  value for an authorisation signal.

API contract
- Memberships in LoginResponse and OIDCCallbackResponse now serialises
  without omitempty, and buildMembershipsForUser / synthFallbackMembership
  always return a non-nil slice — frontend can rely on JSON `[]`.
- handler/auth.GetCurrentUser includes memberships so /auth/me refreshes
  recover currentTenantRole after a page reload.

Performance
- Add TenantService.GetTenantsByIDs (with the matching repo method) and
  use it inside buildMembershipsForUser to batch tenant-name lookups
  instead of issuing one query per membership.

Observability
- Replace the remaining `log.Printf` calls in middleware/auth with
  contextual logger.Warnf / logger.Infof so role resolution events ride
  the standard request-id / Langfuse trace plumbing. The successful
  auto-promote line is tagged `[audit]` for log mining.

Tests
- internal/application/service/tenant_member_test.go: hand-rolled
  fakeTenantMemberRepo plus 11 cases covering AddMember validation,
  EnsureOwner idempotence, "cannot demote/remove the last Owner",
  same-role no-op, ErrInvalidTenantRole / ErrMembershipNotFound, and
  the TenantRole.HasPermission lattice.
- internal/middleware/auth_role_test.go: fakeMemberService plus 8 cases
  pinning every branch of resolveTenantRole, including the two H1
  regressions (cross-tenant superuser must not auto-promote; orphan
  auto-promote requires the home tenant).
2026-05-18 17:28:58 +08:00
wizardchen 7f7a48880f feat(rbac): add tenant_members schema and auth wiring (PR 1 of 3, #1303)
Lay the foundation for tenant-level RBAC without enforcing it yet.
Schema, types, middleware lookup, and login-flow shape changes ship now;
RequireRole/route annotations and member-management endpoints follow in
PR 2 and PR 3.

Highlights:
- New tenant_members table (Postgres + SQLite) with (user_id, tenant_id)
  unique key and (owner|admin|contributor|viewer) role values. Migration
  000043 backfills one row per existing user, marking the earliest-created
  active user in each tenant as Owner and the rest as Contributor.
- KnowledgeBase.creator_id and CustomAgent.runnable_by_viewer added so
  PR 2 can wire ownership-based and Viewer-runtime checks.
- Auth middleware loads the active TenantMember row, sets a new
  TenantRoleContextKey, and falls back via three escape hatches:
  cross-tenant superusers (CanAccessAllTenants) get Admin in the target
  tenant; orphan tenants (zero members) auto-promote the first authenticating
  human to Owner; and when tenant.enable_rbac is false (the v1 default)
  unmatched lookups fail open with Admin to preserve current behaviour.
- Register / OIDC provisioning now insert an Owner membership for the
  registrant via TenantMemberService.EnsureOwner.
- LoginResponse renamed tenant -> active_tenant and gained a memberships
  array. POST /auth/switch-tenant issues a fresh token pair scoped to a
  target membership; GET /auth/config exposes auth.registration_mode for
  the frontend register-tab gating.
- New config keys: auth.registration_mode (self_serve | invite_only,
  defaults to self_serve) and tenant.enable_rbac (defaults to false).
  Both have WEKNORA_* env overrides.
- Frontend store persists memberships and exposes currentTenantRole;
  Login.vue calls /auth/config and hides the Register tab in invite_only
  mode. Go SDK types updated to match the new LoginResponse shape.
2026-05-18 17:28:58 +08:00
wolfkill 3214e3d952 fix(agent): exclude processing docs from prompt 2026-05-18 16:49:14 +08:00
ochan.kwon 0e8de6192c feat(knowledge-base): validate vector store bindings on create, copy, and delete
Wires KnowledgeBase.VectorStoreID and the ownership-aware retrieve factory
into the user-facing knowledge-base lifecycle:

- POST /knowledge-bases validates the requested vector_store_id against
  the caller's tenant scope and the engine registry. New error codes
  ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable (2201)
  distinguish the typed branches without echoing UUIDs to the client.
- GET / POST / PUT / PUT-pin responses embed the bound store's display
  metadata (name, source, engine_type, status) without exposing any
  connection credentials. Cross-tenant shared KBs receive a suppressed
  payload (vector_store_id stripped, source="shared") so operator-chosen
  store names cannot be enumerated across tenants.
- POST /knowledge-bases/copy synchronously rejects clones whose target
  has a different embedding model or vector store, before the async
  clone task is enqueued. The async clone worker re-applies the same
  checks for defense in depth.
- DELETE /vector-stores/:id refuses to remove a store with bound KBs,
  inside a transaction that row-locks the store on PostgreSQL and
  serializes via WAL on SQLite. unregister-from-registry is wrapped in
  defer/recover so a panic surfaces as a structured warning instead of
  silently leaking a stale engine.
- vector_store_id is immutable after creation. The GORM <-:create tag
  blocks every ORM update path; the service-layer DTO omits the field
  entirely; a reflection-based regression test catches any future
  maintainer who adds it back to either layer.
- Empty-string vector_store_id is normalized to nil at both the create
  path and inside SharesStoreWith, so rows persisted by callers that
  did not run Normalize first cannot trip false same-store comparisons.

Part of #993. Depends on #994 and #1310.
2026-05-18 15:58:46 +08:00
nullkey 7611d59d71 docs(cli): README / AGENTS.md / CHANGELOG + CI parity test
Wire-contract documentation and the CI check that keeps it honest.

* cli/README.md gains a verbatim --help block (top-level + subtrees),
  an Exit codes table covering 0/1/2/3/4/5/6/7/10/124/130, a "Status
  vs check" verb-pair subtable, and a "doc wait" paragraph spelling out
  the four exit codes (0 / 1 / 124 / 130). The api passthrough note
  trims storage provider out of the deep-config list now that
  kb create --storage-provider is a polished flag.
* cli/AGENTS.md becomes the contributor guide: build/test, CRUD flag
  conventions, the status/check verb pattern, long-poll wait commands,
  the SetAgentHelp pattern, and a full Error code reference with 35
  typed codes mapped to namespaces, exit codes, retryable / hint
  guidance. Reference section is bracketed by HTML markers so a CI
  parity test can keep it in sync with AllCodes().
* cli/internal/cmdutil/errors_doc_test.go enforces parity: every code
  in AllCodes() must appear in AGENTS.md inside the markers, and
  AGENTS.md must not reference codes that no longer exist. Fails CI
  if a new typed code is added without documentation.
* CHANGELOG.md gets the v0.6 entry: BREAKING (--json / --no-stream /
  WEKNORA_SDK_DEBUG / kb create --name), Added (--format / --jq /
  doc wait / --log-level / kb-and-agent status & check / multi-id
  delete / api --paginate / MCP schema extension / SetAgentHelp /
  signal-aware ctx / kb create --storage-provider / new operation.*
  namespace), Changed (multi-id partial-failure exit code, doc upload
  FlagError, --log-level FlagError, multi-id stdout cleanup, README /
  AGENTS.md changes), with a Migration from v0.5 section walking
  every BREAKING through its v0.6 replacement.
2026-05-18 11:10:19 +08:00
nullkey 34bb0b5096 feat(cli): doc delete multi-id + api --paginate + paginate fixes + batch deletes
Batch-write surface and pagination consistency:

* weknora doc delete <doc-id> [<doc-id>...] — positional multi-id, default
  keep-going on failure. Single -y confirms entire batch.
* weknora session delete <session-id> [<session-id>...] — same shape.
* weknora chunk delete <chunk-id> [<chunk-id>...] --doc <doc-id> — multi-id
  with shared --doc parent.
* Multi-id partial-failure rolls up as operation.failed (exit 1), not
  server.error (exit 7) — failures are operation outcomes, not transient
  transport issues, and the retry-with-backoff hint for server.* would
  mislead callers.
* weknora api <path> --paginate — auto-walks offset pagination and merges
  pages into a single {data, total} JSON response.
* Paginate truncation fix across 6 list/follower call sites.
* All doc / search / chunk / session / kb list commands migrated to
  FormatOptions API.

Multi-id RunE only emits the {ok, failed} envelope when the operation
actually ran — pre-flight failures (e.g. confirmation_required) leave
stdout empty per the wire contract.

doc upload's missing-positional-or-flag check is wrapped as FlagError so
the exit code (2) matches the convention used by other commands that
require a positional argument directly.
2026-05-18 11:10:19 +08:00
nullkey 0e081aec5c feat(cli): --log-level + kb/agent status & check + cross-cutting refactor
Operability surface and the bulk of the jopts→fopts migration:

* --log-level error|warn|info|debug + WEKNORA_LOG_LEVEL env, wired to
  the SDK via client.SetDebugLevel. Invalid --log-level returns FlagError
  (exit 2).
* kb status <kb-id> / kb check <kb-id> verb split (1 HTTP vs 1+N for
  failed_count aggregation).
* agent status <agent-id> / agent check <agent-id> verb split
  (probes kb_scope_all_reachable via 1+N HTTP).
* kb create <name> positional (matches agent create).
* Positional id help strings namespaced (<kb-id> / <agent-id>).
* All auth / context / link / doctor / kb / agent CRUD commands migrated
  to the FormatOptions API.
* root.go Execute(ctx) takes a context so signal-cancellation propagates
  via cmd.Context() into long-running commands.
* Pagination termination uses len(accum) >= total (not page*pageSize)
  so server-capped page sizes do not truncate aggregations.
2026-05-18 11:10:19 +08:00
nullkey 7eeb3bec5d feat(cli): doc wait command (multi-target wait-all)
weknora doc wait <doc-id> [<doc-id>...] blocks until every given document
reaches a terminal parse_status (completed / failed), --timeout expires,
or the user interrupts (SIGINT).

* --timeout DURATION (default 10m; exit 124 on timeout, matches GNU
  timeout(1) convention)
* --interval DURATION (default 2s; exponential backoff to 15s + jitter)
* Multi-id polled concurrently (max 5 parallel)
* Exit code priority 1 > 124 > 0 (failed > timeout > completed)

New typed errors:
* operation.timeout → exit 124
* operation.failed → exit 1
* operation.cancelled → exit 1 (main raises to 130 on signal)

server.session_create_failed gets a special case in ExitCode to map to
exit 1 (workflow failure, not transient retry).

doc view and doc download positional id namespaced to <doc-id>.
2026-05-18 11:10:19 +08:00
nullkey 567d7ac74e feat(cli): --format / NDJSON / chat & agent invoke / MCP / SetAgentHelp / signal-aware ctx / kb create --storage-provider
Adds the structured-output and agent-help surface plus root-level signal
handling so AI agents (and humans working through pipes) get a stable
wire contract.

* --format text|json|ndjson flag, registered per-command on outputs that
  need it; default text on TTY, json on pipe. --jq <expr> pairs with json
  / ndjson to filter or project. FormatOptions absorbs JQ; WantsJSON()
  helper for the JSON dispatch.
* WriteNDJSON helper in internal/format/ (per ndjson.org: one JSON value
  per line, arrays split element-per-line, empty slice → zero bytes).
* chat / agent invoke wire --format ndjson via SDK StreamResponse /
  AgentStreamResponse 1:1 passthrough. Both commands detect ctx.Cancelled
  in every stream + session-create path and emit a stable
  "operation.cancelled" code on Ctrl-C / SIGTERM.
* main.go wires signal.NotifyContext(SIGINT, SIGTERM) into the root
  context so long-running commands run their cancellation cleanup
  (re-emit auto-created session id, etc); the process exits 130 when
  the context was signal-cancelled, matching Unix convention.
* MCP chat / agent_invoke output schemas extended with thinking /
  tool_calls / assistant_message_id (server-side accumulated; MCP
  tools/call has no standard partial-response). doc_view and doc_download
  now use doc_id (not knowledge_id) so agents see a single id naming
  convention across all tools — matches the chunk_list / search_chunks
  schemas and the CLI's <doc-id> positional.
* SetAgentHelp(cmd, AgentHelp{...}) — opt-in machine-friendly --help
  payload activated by WEKNORA_AGENT_HELP=1. Applied to chat / kb list.
* kb create --storage-provider <local|minio|cos|tos|s3|oss|ks3> — sets
  the new KB's storage_provider_config.provider at creation time (server
  does not expose it on update). Required on self-hosted deployments
  where the server-side default doesn't pre-populate a provider —
  without it, subsequent doc upload returns a misleading "kb not found".
2026-05-18 11:10:19 +08:00
Your Name 5deffbd769 fix: guard against empty choices and message=None in LLM response
response.choices[0].message.content raises IndexError (empty choices) or
AttributeError (message=None on filtered content) during dataset generation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-18 11:02:37 +08:00
taoyang b3004f73d4 test(repo): add wiki_config + indexing_strategy to sqlite test DDL
The in-memory SQLite DDL in `knowledgebase_sqlite_test.go` mirrors the
`knowledge_bases` columns the tests need to insert against. It missed two
columns that the `KnowledgeBase` struct has been carrying for a while:

- `wiki_config`        (gorm:"column:wiki_config;type:json")
- `indexing_strategy`  (gorm:"column:indexing_strategy;type:json")

Without those columns the `INSERT INTO knowledge_bases (...)` statement
GORM emits fails with:

    table knowledge_bases has no column named wiki_config

…which cascades to every TestKnowledgeBase_VectorStoreID_* test
failing on a fresh `go test ./internal/application/repository/`.

Reproduce on main (before this commit):

    go test ./internal/application/repository/ -count=1
    ...
    --- FAIL: TestKnowledgeBase_VectorStoreID_Save_Immutable (0.00s)
        knowledgebase_sqlite_test.go:91:
            Received unexpected error:
            table knowledge_bases has no column named wiki_config
    FAIL    github.com/Tencent/WeKnora/internal/application/repository

After this commit:

    ok      github.com/Tencent/WeKnora/internal/application/repository

The fix is two `TEXT NULL` column additions to the inline DDL. No
production code, no migration, no schema change — strictly aligning the
test scaffold with the model fields that already exist.
2026-05-17 21:34:17 +08:00
wizardchen 5141a1230c fix(embedding): repair broken comment in Zhipu embedder
A stray `/` line split what should have been `//`, which prevented `go build` from succeeding in CI.
2026-05-17 21:31:54 +08:00
Wellbek 4fef6826cf feat(embedding): add Zhipu AI embedder implementation 2026-05-17 20:14:51 +08:00
wizardchen ce8969ab14 fix(initialization): surface upstream and SSRF errors verbatim in test-connection responses
Test-connection's "连接失败" was nearly useless for debugging because
two layers conspired to swallow the actual upstream error:

Backend swallow
---------------
checkChatModelConnection and CheckASRModel pattern-matched on the
upstream error string (401 / 404 / timeout / ...) and replaced it with
a fixed Chinese hint. The original SDK message (request URL, response
body, provider's error code description) was discarded.

Refactor: extract classifyConnectionError(errMsg) — returns just the
short Chinese hint. Callers combine "<hint>:<raw err>" so the hint
indicates "where to start looking" and the raw error shows "what
actually happened". The status-400 special-case (endpoint reachable,
parameter mismatch only) still returns success.

Frontend swallow
----------------
ModelEditorDialog.testConnection ignored result.message on success
paths (fine), but also overwrote it with a generic i18n string on
failure paths and 4xx rejections — even though axios's interceptor
already lifts the backend's error.message to error.message. Failure
branch now uses error?.message || result.message as the toast content;
i18n fallback only when neither is present.

SSRF allowlist hint
-------------------
SSRF validation failures previously returned
"Base URL 未通过安全校验: <reason>" with no mention of how to recover
(the allowlist is server-side, not configurable in the UI). Operators
hit it and were stuck.

New utils.FormatSSRFError(label, rawURL, err) wraps the error with
guidance to add the host to SSRF_WHITELIST_EXTRA (the operator-facing
append-only escape hatch — SSRF_WHITELIST itself is the project's
baseline). The hint echoes the failed host so it can be copy-pasted
straight into the env var:

  Base URL 未通过安全校验:DNS resolution failed for hostname
  test.local.com ... 如该地址确实可信,请联系运维在服务端环境变量
  SSRF_WHITELIST_EXTRA 中加入该主机(支持精确域名 / *.example.com
  通配 / IP / CIDR),示例:SSRF_WHITELIST_EXTRA=test.local.com,
  *.example.com,10.0.0.0/8

All SSRF call sites switch to the helper:
  initialization (CheckRemoteModel / TestEmbedding / CheckRerank /
  CheckASR / TestMultimodal / batch validation), model CRUD,
  mcp_service CRUD, knowledge URL ingest, system ReconnectDocReader.
2026-05-17 15:27:52 +08:00
wizardchen 31d5566a50 chore(runtime): silence gin per-route logs and emit env config banner at startup
Two unrelated startup-noise fixes bundled because they share one file
(internal/runtime/startup.go) and one motivation (make the boot log
useful again).

Gin per-route spam
------------------
With ~150 registered routes, DebugMode prints one "[GIN-debug] METHOD
path --> handler" line per route, drowning out everything else. Override
gin.DebugPrintRouteFunc to a counter and gin.DebugPrintFunc to route
through the structured logger. A single "[gin] registered N routes"
summary prints once router build is complete. DebugMode behaviour
otherwise (panic recovery, runtime warnings) is preserved.

Env config banner
-----------------
Operators had no easy way to confirm SYSTEM_AES_KEY (or any other
critical env) was actually loaded — getting the byte length wrong is
particularly silent: utils.GetAESKey() returns nil for anything other
than exactly 32 bytes, which simply disables encryption.

LogStartupEnv prints one line per curated env var (security + DB +
storage + redis), with sensitive values redacted as "set (N chars)"
and unset values as "<unset>". Followed by targeted footgun warnings —
the AES-256 length check fires loudly when SYSTEM_AES_KEY is set but
wrong-length.

Both helpers are called from cmd/server and cmd/desktop. The env banner
runs before container.BuildContainer so it prints even if DB / storage
init fails.
2026-05-17 15:27:52 +08:00
wizardchen dc634e863a feat(crypto): AES-GCM at-rest for MCP and DataSource credentials with graceful key-rotation handling
Two encryption gaps and one fail-loud regression, fixed together because
they are inseparable in practice.

At-rest encryption gaps
-----------------------
Before this change, only Tenant.APIKey, Tenant.Credentials.WeKnoraCloud.AppSecret,
Model.Parameters.APIKey/AppSecret, and WebSearchProviderParameters.APIKey
were encrypted with SYSTEM_AES_KEY. MCP and DataSource were missed:

- MCPAuthConfig.Value/Scan went through bare json.Marshal/Unmarshal, so
  api_key and token were written to mcp_services.auth_config (json) in
  plaintext regardless of whether SYSTEM_AES_KEY was set.
- DataSource.Config (types.JSON) is a byte passthrough, so connector
  secrets (Notion token, GitHub PAT, Feishu app_secret, ...) inside the
  Credentials map were stored in plaintext too.

Now both follow the ModelParameters / WebSearchProviderParameters pattern.
Encryption operates on a local copy to avoid mutating the caller's
in-memory struct (otherwise the next read of the same *MCPAuthConfig
would surface ciphertext as if it were the API key). For DataSource the
ToJSON ↔ ParseConfig pair is the only application boundary that
materializes credentials, so encrypting there is sufficient and avoids
having to fork the generic types.JSON.

Backward compatibility
----------------------
DecryptStoredSecret returns rows without the enc:v1: prefix verbatim, so
historical plaintext records keep working. Next write re-saves them
encrypted. No migration step.

Graceful Scan-time degradation
------------------------------
The strict "fail loud on key rotation" behaviour of DecryptStoredSecret
turned into a footgun once it ran from GORM Scan: a single broken row
caused ListModels / ListMCPServices / ListDataSources to return zero
results, hiding all other rows and giving the operator no way to find
which credential needed re-entering.

New utils.DecryptStoredSecretLenient: same logic, but on failure returns
("", false). The strict variant is kept for hot paths that are about to
actually USE the credential (so a rotated key surfaces loudly there
rather than silently degrading to "missing api key").

All Scan paths switch to lenient + per-failure log line:
  ModelParameters.Scan, WebSearchProviderParameters.Scan,
  MCPAuthConfig.Scan, DataSource.ParseConfig, CredentialsConfig.Scan.

On lenient failure the field blanks out (never ciphertext), the row
still loads, the DTO reports configured=false so the UI shows "not
configured", and any upstream call attempted with the empty value fails
with a clear "missing api key" rather than sending ciphertext as the key.

Tenant.AfterFind (Tenant.APIKey) intentionally stays strict — it is the
primary HTTP auth credential and a silent empty value would change auth
semantics ambiguously.
2026-05-17 15:27:52 +08:00
wizardchen a9d37970ae fix(initialization): fall back to stored apiKey when test-connecting an existing model
Direct regression from the previous credential subresource refactor: the
edit-mode ModelEditorDialog stopped binding apiKey into formData (the
field now lives in <CredentialResource>), so "Test connection" sent
apiKey="" and every upstream call (DeepSeek / lkeap / OpenAI etc.) 401'd.

Fix in the backend so the frontend stays simple:

- ModelTestRequest gains an optional ModelID field.
- New fillSecretsFromStoredModel helper: when ModelID is set AND a secret
  field on the request is empty, copy it from the stored (decrypted) model.
  Non-empty request values still win, so users typing a new key to verify
  it before saving behave the same as before.
- The four test entrypoints (chat / embedding / rerank / asr) call the
  helper right after ShouldBindJSON.

Frontend: ModelEditorDialog.testConnection forwards modelId when isEdit
is true. The four API client signatures accept the optional field.

WebSearch and MCP are unaffected: WebSearch already has a TestProviderByID
ID-based path, MCP has no standalone raw test endpoint.
2026-05-17 15:27:52 +08:00
wizardchen 7643dd457e refactor(credentials): introduce /credentials subresource pattern
Replace the legacy "redacted placeholder + Clear* boolean" pattern with
dedicated per-resource credential subresources across MCP services,
Models, Web search providers, and Data sources.

Why
---
The previous design had three problems:

1. Main PUT body carried secret fields. The frontend echoed back a
   redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty
   defense in the service layer tried to detect "user did not change this"
   vs "user wants to clear this". A regression in that defense (or a new
   frontend forgetting it) silently overwrites the stored secret with the
   placeholder.

2. The "remove this credential" UX was a red checkbox under a pre-filled
   password input. Three intents (preserve / replace / clear) collapsed
   onto one field, and credential changes were bundled with unrelated
   config edits in the same submit. Users wiped working keys by mistake.

3. Secret presence was inferred from "did the response come back with a
   '***' placeholder", which couples the contract to a magic string.

Design
------
Each of the four resources now exposes:

  PUT    /{resource}/{id}/credentials          # write one or more fields
  DELETE /{resource}/{id}/credentials/{field}  # clear a single field

"Is this configured?" metadata travels on the main resource response as
a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET
endpoint. The frontend reads the configured boolean from the main GET
and never sees secret values at all.

Main PUT endpoints now ignore any secret fields in the request body and
log a deprecation warning if they appear, so legacy clients fail loudly
rather than overwriting silently.

Frontend
--------
- New reusable <CredentialResource> component renders a three-state card
  (unconfigured / configured / editing) and drives the dedicated
  endpoints. Used by MCP, Model, Web search; DataSource has a bespoke
  card with the same behaviour because its credentials are a single
  atomic per-connector map.
- Cancel from edit mode now restores state synchronously from the meta
  prop. The previous async refresh() was a no-op while state was still
  'editing', leaving the input frozen open.
- Remove is single-click + toast. The danger-themed button is the
  deterrent; a modal confirm adds friction without adding safety (the
  plaintext is irrecoverable client-side either way — recovery means
  re-typing).

DTOs (internal/handler/dto/) are deliberately separate from the GORM
models so "no secret in response" is a compile-time invariant: a future
contributor cannot leak a secret without explicitly adding the field to
the DTO, which is review-able in a single diff.

Storage is unchanged — credentials still live in the existing jsonb /
parameters columns. No schema migration.

Cleanup
-------
- types.MCPService / types.Model / types.WebSearchProviderEntity /
  types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret
  / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData().
- utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty.
  RedactedSecretPlaceholder constant is retained because VectorStore
  still uses the old pattern and is out of scope here.
- Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat
  redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 15:27:52 +08:00
taoyang f30e9ccd0f fix(docreader): bump grpcio floor to 1.78.0 to match generated proto
docreader fails to start on a fresh `uv sync` of main:

    $ uv sync --frozen
    $ uv run python -m docreader.main
    Traceback (most recent call last):
      ...
      File ".../docreader/proto/docreader_pb2_grpc.py", line 19, in <module>
        raise RuntimeError(...)
    RuntimeError: The grpc package installed is at version 1.76.0, but the
    generated code in docreader_pb2_grpc.py depends on grpcio>=1.78.0.
    Please upgrade your grpc module to grpcio>=1.78.0 or downgrade your
    generated code using grpcio-tools<=1.76.0.

`docreader_pb2_grpc.py:8` was regenerated with grpcio-tools 1.78+ and
hardcodes `GRPC_GENERATED_VERSION = '1.78.0'`, then at import time runs
`first_version_is_lower(GRPC_VERSION, GRPC_GENERATED_VERSION)` and
raises if the installed runtime is older. But `pyproject.toml` still
pins:

    "grpcio>=1.76.0",
    "grpcio-health-checking>=1.76.0",
    "grpcio-tools>=1.76.0",

…and `uv.lock` resolves to grpcio 1.76.0. So every fresh clone +
`uv sync` ends up with a docreader that crashes on import, and a
WeKnora backend that thinks docreader is "not connected" — silently
falling back to the Go-side static engine registry only.

Downstream user-visible symptom: `/api/v1/system/parser-engines`
returns just the Go-side engines. Python-only engines (e.g.
markitdown, plus any custom Python parsers) never appear in the KB
create/edit form. Document parsing falls back to mineru/builtin even
when the user intended to use docreader-backed engines.

Reproduce on main (before this commit):

    cd docreader
    uv sync --frozen
    uv run python -m docreader.main
    # → RuntimeError at import, server never starts

After this commit:

    Resolved 79 packages
    Updated grpcio v1.76.0 -> v1.80.0
    Updated grpcio-health-checking v1.76.0 -> v1.80.0
    Updated grpcio-tools v1.76.0 -> v1.80.0
    ...
    $ uv run python -c "from docreader.proto import docreader_pb2_grpc; print('ok')"
    ok

Fix is the minimum-floor bump on the three grpcio packages, plus the
lockfile update uv emits as a result (all 1.76.0 → 1.80.0, newest
stable matching the proto's floor). No proto regen, no code change —
just aligning the dep declaration with what the generated client
already requires.

The next regen of docreader_pb2_grpc.py (e.g. with grpcio-tools 1.80+)
would raise the floor again; pinning at the proto's declared floor is
the conservative choice that keeps this in sync.
2026-05-17 15:17:09 +08:00
ochan.kwon d869cc1ff6 fix: redact sensitive fields and prevent credential loss on edit
Closes #988

Unify the "write-only secrets" pattern across MCP services, Models,
WebSearch providers, and DataSources. Fix two related defects:

1. GET responses leaked plaintext credentials — GetMCPServiceByID
   while List masked, all Model / WebSearchProvider / DataSource GET
   responses unconditionally.
2. When the frontend echoed a masked placeholder back on save, the
   update path overwrote the stored secret with the mask — silently
   wiping the real credential. MCP surfaced this most visibly:
   ListMCPServices masked Token to "first4****last4" and
   McpServiceDialog round-tripped that masked string back into the DB.

- internal/types/secret.go introduces the shared sentinel
  RedactedSecretPlaceholder = "***" and two helpers:
  - IsRedactedOrEmpty(s): true for "" or the sentinel
  - PreserveIfRedacted(incoming, existing): merge predicate
- Each resource adds a RedactSensitiveData (mutates-in-place) or
  Redacted (returns-a-copy) method and a MergeUpdate contract:
  empty or placeholder preserves, any other value replaces, and a
  write-only Clear* bool flag (gorm:"-", json omitempty) wipes with
  precedence over preserve / replace. Clear* never persists.
- Every GET / List / Create / Update response path now redacts.
  VectorStore's pre-existing "***" literal is refactored onto the
  shared constant for consistency.

All four dialogs — McpServiceDialog, ModelEditorDialog,
WebSearchSettings, DataSourceEditorDialog — never pre-fill secret
inputs. The stored-state signal is collapsed into the input
placeholder: "•••••••• (Enter new value to replace)" when the server
reports a stored value, falling back to the connector-specific
placeholder otherwise. The input itself carries the "something is
there" cue, matching how password dots render.

An opt-in "Remove this credential" checkbox appears in edit mode when
a value is stored; checking it disables the input and triggers a
danger-themed confirmation dialog before submit. DataSource rolls the
clear up to a connector-level "Remove all credentials" flag since its
credentials are per-connector atomic bundles (e.g. Feishu app_id +
app_secret).

i18n updates across en-US, zh-CN, ko-KR, ru-RU:
secret.storedPlaceholder / clearHint / confirmClearTitle /
confirmClearBody.

- MCP update handler's manual map-to-struct projection was silently
  dropping clear_api_key / clear_token / custom_headers on partial
  updates. Pull them off the map explicitly.
- MCP configChanged detection now snapshots URL / StdioConfig /
  TransportType before the merge so connection recycling still
  triggers on real transport changes (not just whenever AuthConfig
  is present).
- MCP CustomHeaders: nil preserves existing, non-nil replaces. Was
  always overwriting, silently wiping headers on any auth-bearing
  update.
- GetMCPServiceByID now redacts (was plaintext while List masked).
- WebSearchProvider update preserves Name / Description when empty
  instead of clobbering them.
- WebSearchProvider TestProviderRaw rejects the "***" placeholder up
  front with an actionable message rather than forwarding it to the
  upstream provider API as a literal key.
- Model builtin hideSensitiveInfo now preserves non-secret metadata
  (SupportsVision, Provider, ExtraConfig, InterfaceType,
  ParameterSize, EmbeddingParameters) that were previously zeroed
  out for builtin model cards.
- Replace local equalStringSlices / equalStringMaps helpers with
  stdlib slices.Equal / maps.Equal (Go 1.21+).

- internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go:
  table-driven coverage of merge, redact, clear, and write-only flag
  invariants for every resource.
- internal/application/service/mcp_service_test.go: full service layer
  with a fake repo; covers ListMCPServices / GetMCPServiceByID
  redaction, UpdateMCPService partial-update honoring of Clear*,
  CustomHeaders preserve-on-nil / replace-on-non-nil, and
  clear-flag precedence over submitted values.

Additive only. No changes to request or response JSON shape beyond
the write-only Clear* flags (omitted by default via omitempty). No DB
schema changes. No SDK struct field removals. Existing clients that
don't know about the new flags continue to function unchanged.

- DataSource.Config jsonb at-rest encryption. Pre-existing residual
  risk; tracked as a follow-up.
- Renaming VectorStore.MaskSensitiveFields to match the new
  RedactSensitiveData / Redacted naming convention. Left alone to
  minimize diff; a follow-up rename PR can unify.
2026-05-17 11:37:53 +08:00
wizardchen a3411899cf fix(docreader/auth): harden gRPC TLS/Token rollout from #1359
Follow-up to #1359. Addresses a set of correctness and security gaps in
the initial docreader auth implementation.

- docker-compose: inject GRPC_TLS_*/GRPC_TLS_SERVER_NAME/GRPC_AUTH_TOKEN
  into the WeKnora-app service. Without this the Go client never saw the
  knobs, so enabling token auth on the server broke every RPC.
- client: bind tokenAuth.RequireTransportSecurity() to TLSEnabled so a
  bearer token cannot be sent over an insecure channel once TLS is on.
- server: load_tls_credentials now raises TLSConfigError on misconfig
  (cert/key missing, file unreadable, mTLS without CA); main.py exits 1
  instead of silently downgrading to insecure.
- server: replace endswith("/Check"|"/Watch") health bypass with exact
  match against /grpc.health.v1.Health/{Check,Watch}.
- server: compare tokens with hmac.compare_digest, warn on tokens < 16B.
- server: AuthInterceptor now returns an abort handler matching the
  original RPC kind (unary/stream) and uses context.abort, so streaming
  RPCs surface UNAUTHENTICATED instead of INTERNAL.
- internal/infrastructure/docparser/grpc_parser.go: drop the duplicated
  TLS/tokenAuth block and reuse docreader/client.LoadAuthConfigFromEnv +
  BuildDialOptions. Single source of truth for client-side auth.
- Add GRPC_TLS_SERVER_NAME (client SNI override) and
  GRPC_MTLS_REQUIRE_CLIENT_CERT (server explicit mTLS toggle); document
  the differing CA semantics between client and server in .env*.example.
- Reject half-configured client mTLS (cert XOR key) loudly.
- Fix missing trailing newline in .env.lite.example.

Verified locally: go build ./... and go vet ./... clean; auth.py
fail-fast / token paths smoke-tested.
2026-05-16 21:45:56 +08:00
Li Xianggang 5a02e22f52 feat(auth): 连接docreader支持auth 2026-05-16 21:31:17 +08:00
nullkey c87e35b34b chore(cli): polish + docs sync + pre-PR audit fixes
Code-reuse polish (post-implementation review pass):
- Extract text.OneLine(maxWidth, s) helper combining preview-row
  normalization (newline/CR/tab → space) with text.Truncate's
  UTF-8-safe truncation. Replaces agent/view.go truncate1Line (ASCII
  '...' + byte-slice CJK-unsafe) and chunk/list.go singleLine.
- Lift cmdutil.OpenInput(path) for the '-' = stdin / else os.Open
  pattern shared across agent create/edit and the api command.
  Replaces agent/create.go's private openInput.
- Strip inline doc-spec parentheticals from source comments — those
  belong in commit messages and project docs, not in source where
  they rot.

Pre-PR audit fixes:
- doc upload: reject `--metadata` paired with `--from-url` as
  input.invalid_argument up-front (the URL-ingest request type has
  no metadata field server-side, so the pair would otherwise silently
  drop). Long help and CHANGELOG updated to call out the asymmetry.
- doc upload (file path): map sdk.ErrDuplicateFile sentinel to
  resource.already_exists. The sentinel arrives with no "HTTP error <n>:"
  prefix because the SDK short-circuits on file-hash before reading the
  HTTP status, so the previous WrapHTTP fall-through misclassified it
  as network.error with a misleading "check base URL reachability" hint.
  The --from-url branch already handled ErrDuplicateURL this way; this
  closes the asymmetry. Caught by e2e re-upload of an already-ingested
  file; regression test added.
- README exit-10 enumeration adds `agent delete` and `chunk delete`
  (these were missing alongside the v0.5 destructive verbs they were
  meant to gate).

Docs sync:
- cli/README.md: command tree now includes the chunk subtree; adds
  agent / chunk lines to the 5-minute quickstart; adds a "Contributing
  / Reporting issues" section pointing at the repo's SECURITY.md and
  AGENTS.md; drops third-party CLI parallels from the surface
  description.
- cli/AGENTS.md: "Command surface design SOP" gains the
  flag-vs-escape-hatch step. "CRUD command flag canon" renamed to the
  hard-required-flags pattern with the contrast (TTY-prompts-fill)
  defined inline rather than via opaque shorthand.
- cli/CHANGELOG.md: search docs case-sensitivity shift promoted to its
  own #### Breaking changes subsection. MCP doc_list filter count
  corrected from 5 to 6. Drops the bogus go.mod yaml.v3 entry (yaml.v3
  was already a dependency on main; v0.5 added zero go.mod lines).
  Replaces internal-Go identifiers (fuzzyTime, NoOptDefVal) with
  user-language and drops the § section-symbol jargon.
2026-05-16 16:56:33 +08:00
nullkey f89d54362d feat(cli): doc/kb resource expansion — upload flags + list filters + view fields
Closes deep-tuning gaps in v0.4-shipped doc / kb / session commands.
Each command had multiple SDK fields the CLI silently hardcoded or
omitted; this commit threads them through.

doc upload:
- --enable-multimodel (tri-state via NoOptDefVal): toggle multimodal
  extraction (PDF/DOCX image-to-text) per upload. Empty explicit
  value (e.g. --enable-multimodel="" from uninterpolated shell var)
  rejected as input.invalid_argument rather than silently coerced.
- --metadata key=value (repeatable): attach arbitrary metadata
- --channel <name> (default api, override for browser / wechat ingests)
- URL mode gains --title / --file-type / --tag-id; URL-only flags
  rejected with input.invalid_argument when used without --from-url

doc list filter flags:
- --keyword (server-side LIKE — case-sensitive per PG)
- --file-type / --source / --tag-id
- --start-time / --end-time (RFC3339)

search docs switches from client-side substring to server-side Keyword
via ListKnowledgeWithFilter — smaller wire payload, but case-sensitivity
shifts (documented in CHANGELOG + help text + Long).

MCP doc_list schema gains the same 6 filter fields (parity with CLI).

session view --full + --limit: loads chat history via LoadMessages SDK
method.

kb view human KV expanded: type / pinned / temporary / processing
state + count / summary model / created timestamp. All omit-empty.

doc view human KV expanded: title (when distinct from filename) /
description / source / channel / summary_status / enable_status /
tag_id / storage (human bytes) / file_hash (12-char prefix).
2026-05-16 16:56:33 +08:00
nullkey 7bccd72ba3 feat(cli): search --all-pages canon catch-up + AGENTS.md SOP / CRUD canon
Brings search docs and search sessions to v0.4 pagination canon
(--limit / --page-size / --all-pages, matching session list / doc list).
Both default --all-pages=true to preserve prior silent walk-all
behavior; explicit knobs added for users who want one-page fetch.

cli/AGENTS.md gains two new sections:
- Command surface design SOP — a 5-step SDK-schema-first pre-design
  checklist for future contributors. Earlier spec drafts produced
  schema-error classes (missing/mismatched fields, missing pagination
  flags) when commands were designed from convention rather than from
  the SDK; the SOP makes the SDK the ground truth.
- CRUD command flag canon — Mode A (hard-required + flag error, no
  interactive prompts), the established pattern for non-auth CRUD.

Also fixes the agent invoke rationale source: the CLI-layer precedent
for invoke being a separate verb (not a chat mode) is documented
inline rather than referencing other vendor CLI behavior.
2026-05-16 16:56:33 +08:00