Commit Graph

45 Commits

Author SHA1 Message Date
wizardchen 7444c2190d chore(rbac): update default behavior for tenant RBAC configuration
Refactor the tenant RBAC configuration to change the default value from false to true, enabling role enforcement by default. This change allows operators to opt into a logging-only rollout window by explicitly setting the configuration to false.

Updates include:
- Modifications to .env.example and docker-compose.yml to reflect the new default.
- Adjustments in rbac.md documentation to clarify the new default behavior and the opt-in process.
- Code changes across various files to utilize the new pointer-based configuration for EnableRBAC, ensuring nil safety and clearer intent.

No functional changes were introduced; the adjustments primarily enhance clarity and maintainability of the RBAC feature.
2026-05-18 21:24:28 +08:00
wizardchen 14f62a6e14 fix(audit_log): prevent deadlock in Stop() when Start() is never called
Updated the AuditLogRetentionRunner to include a 'started' flag, allowing Stop() to return immediately if Start() was never invoked. This change prevents potential deadlocks during graceful shutdown. Additionally, added tests to ensure that Stop() behaves correctly when called before Start() and when the service is nil.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 884ac44283 feat(invitations): implement pending invitations feature with polling and dialog
Added functionality for managing pending invitations, including a polling mechanism to update the invitation count in real-time. Introduced a new dialog component for users to view and respond to their invitations without navigating away from their current context. Updated API endpoints for fetching and managing invitations, and enhanced the user interface with relevant internationalization support.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 8efe1b4187 refactor(rbac): address kb-access guard review findings
Follow-up to the route-level KB-access guard introduced earlier in
this PR. Five reviewer findings, all fixed in middleware + tests
without changing handler call sites:

  - Honour `?agent_id` in the agent-share fallback. The in-handler
    resolution in knowledgebase.go branches on a specific agent's
    KBSelectionMode (all / selected / none); the guard was only ever
    calling TenantCanAccessKBViaSomeSharedAgent, which let some
    requests pass at the guard layer that the handler would then
    reject — and rewrote the request's tenant context to the source
    tenant before the rejection happened. The guard now mirrors the
    handler: a specific agent_id pins the resolution to that agent
    (no fallback to any-agent), an empty agent_id keeps the existing
    any-agent behaviour. The knowledge.go / knowledgebase.go handler
    helpers stay in place as defence in depth until they're folded
    into the guard in a follow-up.

  - Distinguish not-found vs. transient on the knowledge / chunk
    resolvers. A DB hiccup used to surface as a 404, both confusing
    clients and hiding the underlying failure from monitoring. The
    resolvers now return apperror 404 only for the known not-found
    sentinels (apprepo.ErrKnowledgeBaseNotFound, ErrResourceNotFound)
    and let other errors propagate; the guard maps those to a fresh
    NewServiceUnavailableError (503).

  - Stop 500-ing on legacy chunks with empty knowledge_base_id. The
    chunk is effectively unresolvable to a KB, so the client gets the
    same 404 they'd get for a missing chunk instead of a 500 that
    pollutes alerting. The warn log preserves the operator signal.

  - Honour cfg.Tenant.EnableRBAC. The guard now mirrors RequireRole /
    RequireOwnershipOrRole: with enforcement off, 401/403 paths log
    the would-be rejection and pass through. 404 still fires either
    way (a missing resource is not an authorisation event). Keeps the
    rollout window safe — operators can flip enforcement on globally
    without code changes elsewhere.

  - Document the c.Keys vs. c.Request.Context() split. The guard
    intentionally rewrites only the request context (handlers that
    still run their own share resolution, currently knowledge.go /
    knowledgebase.go, would otherwise see kb.TenantID == c.Keys
    tenant and misclassify shared-Viewer access as Admin). The
    package-level comment now spells this out so the migration of the
    remaining handlers in a follow-up doesn't accidentally regress
    `my_permission` rendering.

Tests:

  - Existing four cases keep working; runGuard signature gained a
    guardOpts knob.
  - Eight new cases cover the agent-share resolution paths (any-agent
    viewer, Editor required → must reject, specific agent in modes
    all / selected-match / selected-miss / none, cross-tenant
    mismatch) plus the EnableRBAC=false fail-open + 404-still-fires
    pair.
2026-05-18 17:28:58 +08:00
wizardchen ef01f99e69 refactor(rbac): extend KB-access guard to chunk / knowledge / knowledgebase routes
Follow-up to the FAQ + Tag refactor in this same PR. Three additional
handler areas now run through the route-level g.KBAccess* guard:

  - chunk.go  : full refactor — helper effectiveCtxForKnowledge and
                validateAndGetChunk's access check are gone; the
                handler's chunk-fetch + ownership-mismatch check stay
                as defence-in-depth (a same-tenant attacker can't
                pass mismatched knowledge_id + chunk_id).
  - knowledge.go and knowledgebase.go : routes now run the guard
                BEFORE the handler. The in-handler
                validateKnowledgeBaseAccess /
                resolveKnowledgeAndValidateKBAccess helpers stay for
                now because they branch on the agent_id query param
                (specific-agent check vs. any-agent fallback) — that
                handler-specific branch can collapse into the guard
                in a follow-up. The duplicated DB roundtrip is
                negligible compared to the handler's main work.

New middleware exports:

  - KBIDFromChunkIDParam   : walks chunk_id → KnowledgeBaseID via the
                             chunk's denormalised column.
  - g.KBAccessReadFromChunkIDParam / g.KBAccessWriteFromChunkIDParam
                           : convenience methods on rbacGuards.

Route changes are purely additive (existing role / ownership guards
stay in place; the new KB-access guard runs after them and rewrites
the request's tenant context to the source tenant for shared KBs).
No new dependencies on the kbShareService / agentShareService at the
handler level for chunk.go — those move to the route layer.

Tests pass; behaviour unchanged.
2026-05-18 17:28:58 +08:00
wizardchen 1109cb7bff refactor(rbac): KB-access guard at the route layer (no helper in handlers)
Five handler files used to carry near-identical 30-line helpers
(effectiveCtxForKB / validateAndGetKnowledgeBase) that did the same
own-or-shared resolution before every KB-scoped operation:

  1. KB belongs to caller's tenant     -> grant own access
  2. Org-shared KB                      -> grant min(share, role) cap
  3. Shared agent carries the KB        -> grant Viewer (read-only)

A bug found in any one of them had to be fixed in five places, and
several already drifted apart (different log levels, slightly
different error messages, one path missing the share-cap check).

Move the resolution to a route-level gin.HandlerFunc:
middleware.RequireKBAccess, exposed on rbacGuards as
g.KBAccessRead("id") / g.KBAccessWrite("id"). The guard runs after
the role/ownership checks, then on success:

  - stashes the resolved (KB + effective tenant id + permission)
    on c.Keys under middleware.KBAccessContextKey, and
  - rewrites c.Request.Context() to carry the effective tenant id

so handlers downstream just read tenant the way they always did
(types.MustTenantIDFromContext) without a per-handler helper.

Refactor FAQ and Tag handlers to drop their helper methods; their
service constructors lose the kbShareService / agentShareService
dependencies (now lifted to the route layer). Routes:

  faq.GET("/entries",            g.Viewer(),         g.KBAccessRead("id"),  ...)
  faq.POST("/entries",           g.OwnedKBOrAdmin(), g.KBAccessWrite("id"), ...)
  kbTags.GET("",                 g.Viewer(),         g.KBAccessRead("id"),  ...)
  kbTags.POST("",                g.OwnedKBOrAdmin(), g.KBAccessWrite("id"), ...)

The middleware also exports KBIDFromKnowledgeIDParam so chunk routes
(URL :knowledge_id) can adopt the same pattern in a follow-up PR
without reshaping the helper.

Five new tests cover own-KB, not-found, shared-with-sufficient-perm,
shared-but-below-min, and missing-tenant-context paths. Handler
tests still pass (no behaviour change).
2026-05-18 17:28:58 +08:00
wizardchen 0e1482bc4c fix(rbac): propagate TenantRole through logger.CloneContext
logger.CloneContext rebuilds a fresh context.Background() and only
copies a hand-curated allow-list of context keys (LoggerContextKey,
TenantIDContextKey, UserContextKey, etc.) before passing it to
downstream handlers — the chat / agent QA pipelines all do
`ctx := logger.CloneContext(c.Request.Context())` at the top of
parseQARequest.

TenantRoleContextKey was missing from that allow-list. Any call site
that read the role via TenantRoleFromContext(reqCtx.ctx) saw the
type-zero TenantRole and the helper's least-privilege fallback
returned Viewer. The auth middleware had set role=Owner correctly
on the parent context, but the cloned context lost it.

Symptom: an Owner of their home tenant got 403 on POST /agent-chat
when the agent had runnable_by_viewer=false, because the gate at
session/qa.go:520 saw role=viewer despite middleware logging
role=owner two ms earlier.

Diagnostic log lines added in the previous commits (auth.go:145
"[auth] resolved role=...", auth.go:283 "[auth] resolveTenantRole
step1 hit row_role=...") made this immediately visible — a request
showed role=owner at the middleware layer and role=viewer at the
handler layer, which can only mean ctx propagation lost it. Keeping
the diagnostic logs in place so similar regressions surface fast.

Fix is one line: add TenantRoleContextKey to the CloneContext
allow-list, with a comment pinning it to the bug it fixes so a
future drive-by removal has to weigh that loss.
2026-05-18 17:28:58 +08:00
wizardchen 766a6c482f feat(rbac): per-tenant audit log for RBAC events (PR 6, #1303)
PR 1-5 of the multi-tenant RBAC series enforce the role matrix; what
they don't do is keep a durable record of who did what. Today the
RBAC middleware logs "[rbac] role insufficient ..." to stderr -
perishable, not searchable. Operators cannot answer "who removed
Alice from tenant T-foo on Tuesday" or "did we 403 a probing user
200 times last hour" after the fact.

This PR adds a generic per-tenant audit log that:

- Captures mutating RBAC actions (member add/remove/role-change/leave)
  durably from the tenant_member service after the repo write succeeds.
- Captures enforcement rejections (RequireRole / RequireOwnershipOrRole
  denies under EnableRBAC=true) via a gin-context-injected hook.
- Sliding-window dedup (1 minute) on (tenant, actor, action, path)
  for denied events so a probing client cannot flood audit_logs.
- Default-on - no extra env flag. Schema is generic enough for future
  KB / agent / datasource events to reuse via new action namespaces
  without another migration.

Schema (migration 000044) mirrors wiki_log_entries' shape: a single
audit_logs table with action, target_type/id, request path/method,
outcome, and a JSONB details column. Three indexes: (tenant_id, id
DESC) for the per-tenant cursor feed, actor_user_id for "what did
Alice do", (tenant_id, action) for both the action filter and the
dedup lookup. Sqlite mirror added for Lite mode.

Wiring is nil-safe: the audit field on tenantMemberService and the
gin-context lookup in rbac.go both tolerate a nil service so existing
tests construct without a stub and dormant deployments degrade
gracefully if the provider is misconfigured.

The query API is GET /tenants/:id/audit-log, gated by Admin+ on top
of the existing PathTenantMatch (denial histories should not surface
to ordinary members). Cursor-paginated by descending id (sidesteps
the duplicate-timestamp tie-breaking that BIGSERIAL trivially solves).

Frontend: TenantMembers.vue gets a t-tabs wrapper with Members
(existing) and Audit log (new) tabs; the audit tab is gated to
Admin+ to mirror the server, lazy-loads on first open, paginates via
the next_cursor pattern, and renders coloured chips per
action/outcome so an operator can scan for anomalies. i18n keys
added in all four locales.

Tests: 14 new tests across service (6), middleware (4), and handler
(4). Service tests pin the dedup window behaviour (writes again
after expiry, dedup is per-actor-per-path, degrades on lookup
error). Middleware tests pin both reject sites' audit hook and the
"dormant mode does not fire" + "nil audit does not panic"
invariants. Handler tests pin the cursor envelope and that an empty
page returns next_cursor=0.

Out of scope: per-action ACL on the audit log, retention/TTL job,
KB/agent/datasource event wiring (schema supports it - just no
emitters yet), async writes, frontend filter UI.
2026-05-18 17:28:58 +08:00
wizardchen 9e1c2c2546 refactor(tenant_member): simplify tenant ID resolution in handlers
Removed the resolveTenantIDFromPath method from TenantMemberHandler, consolidating tenant ID parsing directly into the ListMembers, AddMember, UpdateMemberRole, and RemoveMember methods. This change streamlines the code by utilizing the existing parseTenantIDFromPath function, enhancing readability and reducing redundancy. No functional changes were introduced, maintaining the existing behavior of tenant ID resolution.
2026-05-18 17:28:58 +08:00
wizardchen 2082b63951 refactor(rbac): centralise tenant-access guards in middleware
Tenant-access decisions previously lived in four places with subtly
different shapes: canAccessTenant in middleware/auth.go, an unexported
isCrossTenantSuperuser in middleware/rbac.go, authorizeTenantAccess +
two opening if-blocks in handler/tenant.go, and resolveTenantIDFromPath
in handler/tenant_member.go. Each variant had its own logging, its
own treatment of the cluster-wide EnableCrossTenantAccess flag, and
its own subtle difference in what "URL :id matches active tenant"
meant.

Collapse all four into middleware/access.go:

- IsCrossTenantSuperuser(ctx, cfg): exported, flag-aware. Both the
  per-user CanAccessAllTenants attribute AND
  cfg.Tenant.EnableCrossTenantAccess must be true; either alone
  rejects, so flipping the cluster flag off operationally revokes
  every superuser without a token re-issue.
- IsTenantAccessible(ctx, user, target, memberService, cfg): home
  tenant + superuser path + active tenant_members row. Replaces the
  old superuser-only canAccessTenant; the X-Tenant-ID gate now also
  honours ordinary multi-tenant memberships.
- RequireCrossTenantAccess(cfg): route guard for /tenants/all and
  /tenants/search, replacing the 12-line if-blocks at the top of
  ListAllTenants and SearchTenants.
- RequirePathTenantMatch(cfg): route guard for /tenants/:id/...,
  replacing authorizeTenantAccess (4 callers in tenant.go) and
  resolveTenantIDFromPath's superuser carve-out (5 endpoints in
  tenant_member.go). The router declares it once on the /tenants/:id
  group so the rule lives next to the route definitions.

router/rbac.go grows two factory methods (CrossTenant,
PathTenantMatch) so RegisterTenantRoutes stays declarative and reads
the same as the existing g.Viewer() / g.Owner() / g.OwnedKBOrAdmin()
guards. The /tenants/:id endpoints now group under one
PathTenantMatch() middleware instead of each handler re-deriving the
check.

Handler cleanups follow naturally: tenant.go drops
authorizeTenantAccess and the cross-tenant if-blocks; tenant_member.go
drops the cfg field and shrinks resolveTenantIDFromPath to a thin
parser. NewTenantMemberHandler loses its *config.Config argument;
container/container.go's dig wiring picks up the simpler signature
automatically.

middleware/access_test.go covers all four exports with truth-table
cases (flag/attribute/membership combinations, lookup errors, missing
context). tenant_member_test.go's existing handler-level cross-tenant
assertions still pass because memberTestRouter now mounts
RequirePathTenantMatch the same way the production router does.

Net -41 lines across 8 changed files + 2 new files; no behaviour
change other than the X-Tenant-ID gate accepting active memberships
(which was the original PR 4 goal that motivated this refactor).
2026-05-18 17:28:58 +08:00
wizardchen 0f0f7e4339 test(rbac): cover review gaps from PR 2 self-review
- Add TestRequireRole_CrossTenantSuperuserBypass: pin the
  CanAccessAllTenants -> Owner-gate bypass behaviour for RequireRole,
  matching the existing coverage on RequireOwnershipOrRole. Catches
  reorderings of the fast paths that would re-lock org-level
  superusers out of foreign-tenant Owner-only routes.

- Add TestAgentCreatorLookup_CrossTenantIsHiddenAsNotFound: lock down
  the defence-in-depth tenant re-check inside AgentCreatorLookup so a
  future refactor that loosens the service-layer scope (or wires a
  cross-tenant variant by mistake) cannot leak access via the
  ownership shortcut.

- Reword RequireOwnershipOrRole step 7-8 doc comment so "role check
  already failed" is unambiguous (it refers to step 1 above).

- Annotate the synthetic-system-user construction in middleware/auth.go
  with a back-pointer to types.IsSyntheticUserID so anyone changing
  the prefix or numeric suffix knows to update the helper too.
2026-05-18 17:28:58 +08:00
wizardchen 8b71def60b fix(rbac): address PR 2 review feedback
Middleware
- RequireOwnershipOrRole now skips the creator lookup entirely when
  EnableRBAC is off, eliminating a hidden per-request DB roundtrip in
  the rollout window.
- New ErrResourceNotFound sentinel lets lookups signal "not visible to
  this tenant"; the middleware passes the request through so handlers
  can return a real 404 instead of a misleading 403.
- Transient lookup failures now surface as 503 (was 403) so monitoring
  can tell denied-by-policy from cannot-verify-right-now.
- Cross-tenant superusers (CanAccessAllTenants) bypass tenant role gates,
  restoring DELETE /tenants/:id and ResetAPIKey for org operators that
  resolve to Admin in foreign tenants.
- Construction-time warning fires once when cfg is nil/incomplete, so a
  wiring bug that disables enforcement globally is visible at boot.

Route coverage
- Add role guards to FAQ, KnowledgeTag, WikiPage, Skill, WebSearch,
  WebSearchProvider, WeKnoraCloud, Session, Chat, and Message route
  groups so the matrix actually matches the PR description.

Service layer
- KB and agent creators are no longer recorded as the synthetic
  system-<tenantID> user used by the X-API-Key path; resources stay
  tenant-owned. New types.IsSyntheticUserID helper centralises the
  prefix check.
- KBCreatorLookup and AgentCreatorLookup re-check tenant scope so a
  same-ID resource in another tenant cannot piggyback through the
  ownership shortcut.

Docs
- CustomAgent.RunnableByViewer comment now matches enforcement: it only
  blocks Viewer-driven agent-mode runs, not plain RAG/Wiki QA.

Tests
- New cases for the 503/404 paths, lookup-skip behaviour, cross-tenant
  superuser bypass, the lookup helpers (not-found, cross-tenant,
  built-in agent, missing tenant context), and IsSyntheticUserID.
2026-05-18 17:28:58 +08:00
wizardchen 9e3c92ca95 feat(rbac): enforce tenant role matrix on routes (PR 2 of 3, #1303)
Build on PR 1's tenant_members schema by enforcing the role matrix on
every authenticated route and at agent run time. Enforcement is gated
behind cfg.Tenant.EnableRBAC so the change ships dormant — the same
code paths log the would-be rejection during the rollout window and
start blocking once operators flip the flag.

Middleware (internal/middleware/rbac.go):
- RequireRole(min, cfg): role-only gate using TenantRole.HasPermission.
- RequireOwnershipOrRole(min, lookup, cfg): allows the original creator
  through even when their role is below min, otherwise falls back to
  the role check. Lookup errors fail closed; an empty creator_id is
  treated as tenant-owned and requires the role.
- Both honour cfg.Tenant.EnableRBAC for the rollout window.

Creator lookups (internal/handler/rbac_lookups.go):
- KBCreatorLookup resolves :id -> KnowledgeBase.CreatorID.
- AgentCreatorLookup resolves :id -> CustomAgent.CreatedBy; built-in
  agents (IsBuiltin=true) return ("", nil) so they remain Admin+ only.
- Compile-time guards keep the methods aligned with CreatorLookup.

Route guards (internal/router/rbac.go + router.go):
- rbacGuards bundles the matrix into Viewer/Contributor/Admin/Owner
  helpers plus OwnedKBOrAdmin / OwnedAgentOrAdmin. Each Register*
  function takes *rbacGuards so route lines remain readable.
- KB / Agent / Knowledge / Chunk / Tenant / Model / MCPService /
  VectorStore / DataSource / IMChannel routes annotated per the PR 1
  design doc matrix. Tenant-wide infra (models, vector stores, MCP,
  data sources, IM channels) reads as Viewer+ and writes as Admin+;
  per-resource mutations on KB / Agent route through the
  ownership-or-Admin guard.

Resource ownership (services):
- CreateKnowledgeBase / CopyKnowledgeBase populate CreatorID from the
  authenticated user, so the OwnedKBOrAdmin guard has a creator to
  match.
- CreateAgent / CopyAgent populate CreatedBy similarly. Built-in
  agents stay tenant-owned.

Agent run-time gate (internal/handler/session/qa.go):
- AgentQA refuses to launch the agent engine for Viewers when the
  target agent has RunnableByViewer=false. Same EnableRBAC gating as
  the middleware so it's a no-op until enforcement is turned on.

Tests:
- 12 unit tests cover RequireRole / RequireOwnershipOrRole, including
  the fail-open and fail-closed branches and the lookup-error path.
2026-05-18 17:28:58 +08:00
wizardchen af23483433 fix(auth): honour JWT tenant_id claim in middleware
POST /auth/switch-tenant mints an access token whose `tenant_id` claim
points to the target tenant, but the middleware previously discarded
that claim entirely:

    user, err := userService.ValidateToken(ctx, token)  // returned only *User
    targetTenantID := user.TenantID                     // always the home tenant

The session therefore stayed scoped to the user's home tenant despite
the supposedly fresh token, making the new endpoint a no-op once
the request hit any tenant-scoped query.

This change widens the contract:
- UserService.ValidateToken now returns (user, activeTenantID, error)
  where activeTenantID is the JWT's `tenant_id` claim, falling back to
  user.TenantID when the claim is absent (backward compatibility with
  pre-RBAC tokens still in the wild).
- middleware/auth.go uses activeTenantID as the default target before
  applying any X-Tenant-ID header override, and recomputes
  crossTenantSwitch from `activeTenantID != user.TenantID` so the
  resolveTenantRole branches stay correct for switch-tenant sessions.
- The non-middleware /auth/validate handler in handler/auth.go absorbs
  the new return value (it only cares about user identity).

Tests:
- internal/application/service/user_jwt_test.go pins the JWT->tenant-id
  projection (`tenantIDFromClaims`) — claim present beats fallback,
  zero/negative/string claims fall back, int64/uint64 are accepted for
  test-built tokens.

Required for PR 2 multi-tenant integration tests, where a single user
needs to switch between Owner-in-A and Viewer-in-B sessions and observe
distinct authority on each side.
2026-05-18 17:28:58 +08:00
wizardchen f3d1210577 refactor(rbac): address PR 1 review feedback
Tighten correctness, security and observability of the tenant RBAC
foundation introduced in 3c4cdd70 in response to internal review.

Bugfixes
- custom_agents.runnable_by_viewer: drop the GORM `default:true` tag.
  GORM treats a plain bool's zero value as "unspecified" and silently
  rewrites it to the column default on insert, which would prevent
  admins from ever creating an agent with viewer-runtime disabled. The
  column-level DEFAULT TRUE on the database side still backstops rows
  inserted by raw SQL / migrations.
- TenantMemberRepository.HasAnyMembers: replace `Limit(1).Count(&n)`
  with `Take(&probe)`. GORM's Count ignores Limit and scans the whole
  table, which matters because this runs on the auth middleware's
  hot path for users without a cached membership.
- service/user.LoginWithOIDC: include User, Tenant and Memberships in
  the OIDCCallbackResponse so the OIDC flow returns the same shape as
  password login (previously User and friends were left nil).

Security
- middleware/auth.resolveTenantRole: restrict orphan-tenant
  auto-promotion to the caller's home tenant. Cross-tenant superuser
  visits and JWTs minted for foreign tenants no longer trigger silent
  Owner grants. Two regression tests pin the new behaviour.
- service/user.synthFallbackMembership: default the synthesised role
  to Viewer instead of Owner. The login response only feeds UI; the
  backend re-derives the real role from tenant_members on every
  request, so showing a Viewer UI is preferable to a misleading Owner
  UI that would surface admin controls the backend will then 403.
- migrations/versioned/000043_tenant_rbac.down.sql: gate the destructive
  drop behind a `weknora.allow_destructive_migration` GUC so an
  accidental `migrate down` cannot wipe role data in production.
- frontend/src/stores/auth.ts: comment currentTenantRole as
  rendering-only so future PRs cannot mistake the localStorage-backed
  value for an authorisation signal.

API contract
- Memberships in LoginResponse and OIDCCallbackResponse now serialises
  without omitempty, and buildMembershipsForUser / synthFallbackMembership
  always return a non-nil slice — frontend can rely on JSON `[]`.
- handler/auth.GetCurrentUser includes memberships so /auth/me refreshes
  recover currentTenantRole after a page reload.

Performance
- Add TenantService.GetTenantsByIDs (with the matching repo method) and
  use it inside buildMembershipsForUser to batch tenant-name lookups
  instead of issuing one query per membership.

Observability
- Replace the remaining `log.Printf` calls in middleware/auth with
  contextual logger.Warnf / logger.Infof so role resolution events ride
  the standard request-id / Langfuse trace plumbing. The successful
  auto-promote line is tagged `[audit]` for log mining.

Tests
- internal/application/service/tenant_member_test.go: hand-rolled
  fakeTenantMemberRepo plus 11 cases covering AddMember validation,
  EnsureOwner idempotence, "cannot demote/remove the last Owner",
  same-role no-op, ErrInvalidTenantRole / ErrMembershipNotFound, and
  the TenantRole.HasPermission lattice.
- internal/middleware/auth_role_test.go: fakeMemberService plus 8 cases
  pinning every branch of resolveTenantRole, including the two H1
  regressions (cross-tenant superuser must not auto-promote; orphan
  auto-promote requires the home tenant).
2026-05-18 17:28:58 +08:00
wizardchen 7f7a48880f feat(rbac): add tenant_members schema and auth wiring (PR 1 of 3, #1303)
Lay the foundation for tenant-level RBAC without enforcing it yet.
Schema, types, middleware lookup, and login-flow shape changes ship now;
RequireRole/route annotations and member-management endpoints follow in
PR 2 and PR 3.

Highlights:
- New tenant_members table (Postgres + SQLite) with (user_id, tenant_id)
  unique key and (owner|admin|contributor|viewer) role values. Migration
  000043 backfills one row per existing user, marking the earliest-created
  active user in each tenant as Owner and the rest as Contributor.
- KnowledgeBase.creator_id and CustomAgent.runnable_by_viewer added so
  PR 2 can wire ownership-based and Viewer-runtime checks.
- Auth middleware loads the active TenantMember row, sets a new
  TenantRoleContextKey, and falls back via three escape hatches:
  cross-tenant superusers (CanAccessAllTenants) get Admin in the target
  tenant; orphan tenants (zero members) auto-promote the first authenticating
  human to Owner; and when tenant.enable_rbac is false (the v1 default)
  unmatched lookups fail open with Admin to preserve current behaviour.
- Register / OIDC provisioning now insert an Owner membership for the
  registrant via TenantMemberService.EnsureOwner.
- LoginResponse renamed tenant -> active_tenant and gained a memberships
  array. POST /auth/switch-tenant issues a fresh token pair scoped to a
  target membership; GET /auth/config exposes auth.registration_mode for
  the frontend register-tab gating.
- New config keys: auth.registration_mode (self_serve | invite_only,
  defaults to self_serve) and tenant.enable_rbac (defaults to false).
  Both have WEKNORA_* env overrides.
- Frontend store persists memberships and exposes currentTenantRole;
  Login.vue calls /auth/config and hides the Register tab in invite_only
  mode. Go SDK types updated to match the new LoginResponse shape.
2026-05-18 17:28:58 +08:00
wizardchen 86b05d923e fix(middleware): mask camelCase secret fields in request logs
The request logger's sanitizeBody only matched lowercase / snake_case
field names (api_key, apikey, access_token, ...), so values for the
camelCase JSON fields actually used by the API (apiKey, secretKey,
refreshToken, accessToken, ...) were written to logs in clear text.

Replace the per-field patterns with a single case-insensitive regex
that tolerates optional `_`/`-` separators, covering snake_case,
camelCase and PascalCase variants, and extend coverage to id_token,
client_secret, private_key, auth_token, api_secret and passwd. The
field name is preserved; only its value is replaced with "***".

Add unit tests for sanitizeBody covering the previously-leaking
camelCase fields and common variants.

Fixes #1287
2026-05-12 19:27:29 +08:00
wizardchen 82947c726e fix(wiki-ingest,asynqdl): review fixups for PR #1241
- repo: drop r.db.Debug() from FindSimilarPages — it was dumping every
  trigram probe's SQL+args (per-alias, per-item) into production logs.
- wiki_ingest dedup: fix Printf format string ("selected for %d new
  items" had two args), and harden validMerge against un-prefixed
  slugs whose strings.Index returned -1 and silently passed the type
  check.
- wiki_ingest_batch: drop the duplicated loggedBatchSize/MapPar/
  ReducePar assignments.
- asynqdl: record the real attempt count (retried + 1) on the dead
  letter row instead of a hard-coded 0; tighten payloadProbe to the
  set of field names with consistent semantics across payloads
  (drop source_id/target_id/target_kb_id which differ by task type).
- asynqdl tests: update for the trimmed probe and assert FailCount=0
  outside an asynq worker ctx so the semantics stay pinned.
2026-05-10 00:01:06 +08:00
wizardchen 04e8105728 perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter
The wiki ingest post-process pipeline OOM'd and ran for hours on KBs
with ~40k documents. The dominant tail was a per-batch ListAllPages
that pulled every page (multi-MB content blobs) into Go memory, plus a
24h-TTL Redis pending list whose data could be evicted before a long
serialized backlog drained. dedup did O(P × N) Jaccard scoring in Go
on every batch. Lint loaded the full graph. The index page kept the
entire wiki directory in its content column and rewrote the TOAST
chunk on every ingest. None of these survive at 4w docs.

This change reworks the write path end to end and pulls the durable-
queue + dead-letter primitives out of wiki and into shared
infrastructure that every asynq task type now benefits from.

  - migration 000041: task_pending_ops + task_dead_letters tables, plus
    three GIN indexes on wiki_pages (source_refs jsonb_path_ops,
    source_refs text fallback, lower(title) trgm).
  - TaskPendingOpsRepository + TaskDeadLetterRepository in
    internal/application/repository/task_queue.go: cursor-paginated
    list, atomic IncrFailCount via UPDATE…RETURNING, dedup-key scoped
    delete that refuses empty keys so a buggy caller can't wipe a KB's
    queue.
  - internal/middleware/asynqdl: writes a task_dead_letters row when an
    asynq task exhausts its retry budget. Payload-agnostic — a small
    probe struct extracts TenantID + scope hints across every existing
    payload type, so summary:generation / image:multimodal /
    faq_import / kb:clone / etc. all dead-letter without per-handler
    code. Best-effort: an insert failure never masks the underlying
    task error. Installed in router/task.go before the langfuse mw so
    it sees raw errors.

  - Pending queue moved Redis → PG (no TTL, restart-durable). Redis
    keeps just the active-batch lock and the delete tombstone.
  - In-batch retry budget (wikiMaxFailRetries=5) tracked via
    pendingRepo.IncrFailCount; over the cap the row is moved to
    task_dead_letters with task_type=wiki:ingest, related_id=knowledge
    id. Asynq retries (10) are handled by the new middleware.
  - Removed the per-batch ListAllPages. WikiBatchContext now carries
    lazy fetcher closures (SlugTitleMany, SummaryByKnowledgeID) with
    mutex-protected caches; reduce reaches for titles / summaries on
    demand instead of pre-loading the whole KB.
  - getExistingPageSlugsForKnowledge now uses ListSlugsBySourceRef,
    which the new GIN index on source_refs serves as a Bitmap Index
    Scan instead of a sequential text LIKE.
  - Dedup pre-filter uses pg_trgm via FindSimilarPages
    (idx_wiki_pages_title_trgm). For each new entity/concept (and each
    of its aliases) we ask the DB for the top-K trigram-similar
    existing pages and union the results — bounded prompt size, no Go-
    side O(P × M) loop. Small KBs (≤25 entities) bypass the pre-filter.
  - cleanDeadLinks / injectCrossLinks are scoped to the batch's
    affected slugs (a few dozen) instead of every page in the KB. Both
    use the new lite ListBySlugs / ExistsSlugs repo methods so they
    pull only slug + title + outlinks, not full content.
  - Slug fuzzy resolve (slug_fuzzy.go): when the LLM emits
    [[bad-slug|display]] the cleanup path now tries display-text
    reverse lookup → hyphen/case normalized equality → char-bigram
    Jaccard ≥ 0.8 before stripping. Recovers the common pinyin-word-
    break drift case ("shang-hai-tower" vs "shanghai-tower") in place
    instead of replacing the link with plain text.
  - rebuildIndexPage uses ListByTypeRecent(200) for the first-time
    intro and drops the full DocumentSummaries blob from the
    incremental update prompt, so its context stays bounded regardless
    of KB size.
  - Concurrency tunables surfaced in WikiConfig: IngestBatchSize /
    IngestMapParallel / IngestReduceParallel, with sensible defaults
    via OrDefault helpers. scheduleFollowUp drops to ProcessIn(0) so
    follow-ups don't waste asynq retry slots bouncing on the active
    lock.

  - RunLint walks pages via the new ListPagesCursor in 200-page
    windows and computes the live-slug set with a one-column
    ListAllSlugs Pluck instead of a Limit:0 GetGraph that materialized
    every node + edge. Memory is now bounded; 40k pages walks in
    constant ~4MB.

  - 14 GORM tests for both repos against an in-memory SQLite mirror of
    the production DDL (task_queue_test.go).
  - 6 tests for the dead-letter middleware covering retry budget
    detection, payload-agnostic scope inference, error truncation, and
    repo-failure isolation.
  - 7 tests for the slug fuzzy resolve helper covering the three
    resolution stages, display-text priority over normalized equality,
    bigram fallback acceptance, and rejection of unrelated slugs.
  - Existing wiki_ingest / wiki_lint / wiki_page tests updated to the
    new fetcher / cursor APIs.
2026-05-10 00:01:06 +08:00
wizardchen 7fd566bc15 fix(im): rewrite private storage URLs to HTTP in IM channel replies
IM platforms (Feishu, Slack, Telegram, DingTalk, Mattermost, WeCom) cannot
render provider:// URLs (local://, minio://, s3://, etc.) that appear in
LLM answers containing knowledge base images. The web frontend handles
these via the authenticated /files endpoint, but IM clients need publicly
resolvable HTTP URLs.

Changes:
- Add HMAC-SHA256 presigned URL utility (internal/utils/presign.go) for
  generating time-limited, signature-verified file access URLs
- Add GET /api/v1/files/presigned endpoint that serves files without
  session auth, verified by HMAC signature and expiry
- Update localFileService.GetFileURL() to return presigned HTTP URLs
  when APP_EXTERNAL_URL is configured (cloud backends already return
  presigned URLs via their SDKs)
- Add IM content rewriting pipeline: strip <image> XML tags, strip
  citation tags, rewrite storage URLs to HTTP — applied at all IM
  output points (streaming flush, non-streaming reply, fallback)
- Add holdback buffer in streaming flush to prevent URL/tag splitting
  across chunk boundaries

Closes #1058
2026-04-30 11:40:30 +08:00
wizardchen fba6d7d145 refactor: Update wiki prompts and ingestion logic for improved content handling
- Renamed `WikiPageUpdatePrompt` to `WikiPageModifyPrompt` to better reflect its functionality of adding new information and removing outdated content in a single operation.
- Enhanced the instructions for both `WikiPageModifyPrompt` and `WikiPageRetractPrompt` to clarify the handling of additions and retractions, ensuring accurate content updates.
- Consolidated the logic for enqueuing retraction tasks in the `cleanupWikiOnKnowledgeDelete` method to streamline the process of managing affected pages.
- Removed the `rebuildWikiIndexSimple` function to simplify the codebase, as its functionality is no longer needed with the new ingestion approach.

These changes improve the clarity and efficiency of the wiki content management process, enhancing overall system performance.
2026-04-22 21:17:31 +08:00
Mao Meng e3b493e316 fix: use constant-time comparison for API key validation 2026-04-19 21:41:22 +08:00
wizardchen 92bf52721c feat: implement auto-setup for Lite edition, enhancing user experience by automating user and tenant creation on first launch 2026-04-11 18:51:16 +08:00
ochan.kwon 8a375749f1 fix(i18n): use WEKNORA_LANGUAGE env for prompt language fallback instead of hardcoded zh-CN
- Add DefaultLanguage() that reads WEKNORA_LANGUAGE env with zh-CN fallback
- Replace hardcoded zh-CN in LanguageNameFromContext with DefaultLanguage()
- Unify middleware/language.go to use types.DefaultLanguage()
- Add Language field to DocumentProcessPayload (8 enqueue sites + worker restore)
- Add Language field to ImageMultimodalPayload (enqueue + worker restore)
- Propagate language from ImageMultimodalPayload to QuestionGenerationPayload
2026-04-02 12:01:25 +08:00
Windfarer c1816fe6d6 add oidc 2026-03-30 11:13:44 +08:00
wizardchen 72dfb9ce75 feat(prompt): enhance context and intent classification in templates
- Updated context template to include runtime metadata such as current time and week, improving contextual awareness in user queries.
- Enhanced rewrite template with critical instructions for intent classification, ensuring that rewritten questions preserve essential entities and keywords.
- Refined intent classification logic to prioritize user intents more effectively, improving the accuracy of responses based on user queries.
- Added examples to clarify expected input and output formats for intent classification, enhancing usability for developers.
2026-03-26 18:57:12 +08:00
wizardchen 17da967f92 refactor: update session title generation and language context handling
- Modified the session title generation template to focus on extracting only the intent of the user's question.
- Enhanced the session service to include language context when generating titles.
- Updated middleware to inject language into the request context.
- Added a new test suite for validating SSRF safety of URLs.
2026-03-17 22:23:24 +08:00
wizardchen f7750d65c1 fix: update default language setting to Chinese 2026-03-17 22:23:24 +08:00
wizardchen 139a9c40ff feat: enhance localization and configuration support
- Added timezone and language settings to the environment configuration.
- Introduced built-in agent configurations with multilingual support for various agents.
- Updated Docker Compose to utilize new environment variables for timezone and language.
- Created new prompt templates for question generation, summary generation, and keywords extraction.
2026-03-17 22:23:24 +08:00
wizardchen f41b0e77fd fix: update grep_chunks tool to support multiple patterns
- Renamed the input parameter from "pattern" to "patterns" to reflect that multiple text patterns can be provided.
- Updated the schema definition and input structure accordingly to ensure consistency in handling multiple patterns.
- Enhanced error logging to specify "patterns" when no valid input is provided, improving clarity for debugging.

These changes improve the functionality of the grep_chunks tool by allowing users to search for multiple patterns simultaneously.
2026-03-13 11:57:40 +08:00
wizardchen 4ddc468fea feat: add GetUserByTenantID functionality to user repository and service
- Implemented GetUserByTenantID method in userRepository to retrieve the first user associated with a given tenant ID.
- Added corresponding GetUserByTenantID method in userService to facilitate user retrieval by tenant ID.
- Updated UserService and UserRepository interfaces to include the new method, ensuring consistency across the application.
- Enhanced auth middleware to store user information in context based on tenant ID, improving user management.

These changes enhance user retrieval capabilities by allowing access to user data based on tenant associations, improving overall application functionality.
2026-03-05 18:14:14 +08:00
wizardchen 397689d2f3 feat: introduce WeKnora Lite edition with lightweight configuration and deployment
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.

This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
2026-03-02 21:21:49 +08:00
wizardchen 4158773849 feat: add organization management features
- Introduced a new API for organization management, including creation, updating, and deletion of organizations.
- Implemented organization sharing functionality for knowledge bases, allowing users to share knowledge bases with specific organizations and set permissions.
- Added UI components for organization selection and sharing dialogs.
- Enhanced existing components to support organization-related features, including shared knowledge base handling and organization role management.

This update enhances collaboration capabilities within the application by allowing users to manage and share knowledge bases across different organizations.
2026-02-09 11:40:25 +08:00
hobostay a39c9dab9b fix: replace standard log with structured logger in main and recovery middleware
This PR continues the migration to structured logging by replacing the
standard Go log package usage with the project's structured logger.

Changes:
- cmd/server/main.go: Replace all log.Printf, log.Println, log.Fatalf
  with logger.Infof, logger.Info, logger.Fatalf
- internal/middleware/recovery.go: Replace log.Printf with
  logger.ErrorWithFields for better panic logging with structured data
- Remove log.SetFlags and log.SetOutput from main.go as they're no
  longer needed with the structured logger

Benefits:
- Consistent logging format across the entire application
- Better log parsing and analysis with structured fields
- Request ID tracking in recovery middleware for easier debugging
- Proper integration with the project's logging configuration

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-06 17:59:15 +08:00
DJ 8b160bba15 fix: use typed context key instead of string literal- Add UserContextKey constant to types package- Replace bare string 'user' with types.UserContextKey in auth middleware- Update user service to use UserContextKey for context value retrieval- Fixes staticcheck SA1029 warningThis prevents potential context key collisions. 2026-01-21 19:12:24 +08:00
wizardchen cdf576eb18 feat: Implement cross-tenant access functionality
- Added configuration for enabling cross-tenant access in the application.
- Introduced new API endpoints for listing and searching tenants, accessible only to users with the appropriate permissions.
- Updated user and tenant models to include the `can_access_all_tenants` field.
- Enhanced the authentication middleware to handle cross-tenant requests and validate user permissions.
- Created a new TenantSelector component for selecting tenants in the frontend, improving user experience.
- Updated internationalization files to support new tenant-related strings and messages.
2025-12-02 12:17:39 +08:00
wizardchen eb8bfddfd1 refactor: Update function signatures across multiple files to improve readability by adding context parameters and enhancing code structure 2025-11-28 16:17:52 +08:00
wizardchen 016df3a4db refactor: Sanitize log inputs across various handlers to enhance security
- Implemented input sanitization for logging in chunk, evaluation, initialization, knowledge, knowledgebase, mcp_service, model, tenant, and session handlers.
- Updated logging statements to use sanitized values for sensitive information, preventing potential injection attacks.
2025-11-25 23:33:34 +08:00
wizardchen bf35a2861c refactor: Sanitize log inputs to prevent injection attacks 2025-11-25 22:19:41 +08:00
wizardchen 64c44c9e27 refactor: Update tenant ID handling across the application 2025-11-25 16:30:54 +08:00
wizardchen 057cfd9eb0 feat: Enhance FAQ import functionality and UI improvements
- Added support for tracking the status of FAQ import tasks, including progress indicators for ongoing imports.
- Introduced new localization strings for processing tasks and batch operations in multiple languages.
- Updated the FAQ entry manager to display import progress and improved user feedback during the import process.
- Enhanced the KnowledgeBase component to reflect the processing state of knowledge items, improving user experience during document uploads.
- Implemented batch deletion functionality for chunks, streamlining the management of knowledge base content.
2025-11-23 17:32:50 +08:00
wizardchen 81bd2e6c2c feat: Add Login Page 2025-09-16 02:47:39 +08:00
wizardchen 78088057fb fix: frontend depends app health 2025-09-11 14:22:29 +08:00
wizardchen bdabed6bfa feat: Added web page for configuring model information 2025-08-10 17:11:07 +08:00
wizardchen 56eb2bce33 init commit 2025-08-05 15:08:07 +08:00