Commit Graph

155 Commits

Author SHA1 Message Date
wizardchen bb202a203a feat: implement tenant validation for file access and enhance related tests
This commit introduces a new validation mechanism to ensure that file access paths include the correct tenant segment, preventing cross-tenant access. The `ValidateStoragePathTenant` function has been added to enforce this rule, and the `serveFiles` function has been updated to return a forbidden status for invalid paths. Additionally, new tests have been added to verify the behavior of the file service under various tenant scenarios, ensuring robust handling of file access permissions.
2026-05-29 13:47:32 +08:00
wizardchen 44d6175559 feat: add knowledge parse cancellation with finalizing post-process state
Lets users stop an in-flight document parse to free up LLM / worker
resources without losing the chunks and index already written. The
core insight is that the previous parse_status=completed flipped as
soon as primary chunks landed, while the most expensive subtasks
(graph extract = N LLM calls per chunk, plus summary, question
generation) were still running in the background — so "completed"
wasn't actually terminal from a resource standpoint.

State machine

  pending -> processing -> finalizing -> completed
                              |
                              +-> cancelled (any of the three
                                            in-flight states)
                              +-> failed
                              +-> deleting

`finalizing` is the new post-process fan-out window. parse_status
only promotes to `completed` once pending_subtasks_count (a new
column tracking summary + question + per-chunk graph extract)
drains to zero via atomic FinalizeSubtask. Wiki ingest is
intentionally excluded from the counter — it's a KB-scoped
debounced batch and would otherwise pin parse_status in
`finalizing` for the wiki batch window.

Backend

- New ParseStatusFinalizing + pending_subtasks_count column with
  migration 000056.
- knowledgeRepository.SetFinalizing transitions processing -> finalizing
  conditionally so a racing cancel cannot be clobbered.
- knowledgeRepository.FinalizeSubtask atomically decrements the
  counter and self-promotes the row to completed when it hits zero.
- KnowledgePostProcess restructured to compute expected subtask
  count up front, flip to finalizing (or completed when no
  enrichment is enabled), and only then fan out subtasks. Subtask
  handlers (summary, question, graph extract) defer-decrement on
  terminal exit using the existing isFinalAsynqAttempt convention.
- New POST /api/v1/knowledge/{id}/cancel-parse handler accepting
  pending / processing / finalizing. Marks the row cancelled,
  zeroes the counter, best-effort dequeues asynq tasks via a new
  TaskInspector abstraction (asynq-mode walks pending/scheduled/
  retry queues; Lite-mode noop), and scrubs wiki ingest pending op.
- SpanTracker.AbortAttempt flat-sweeps every still-running span
  for the attempt via a new repo.CancelAllOpenSpans helper so the
  trace viewer's striped bars all flip to cancelled, even leaf
  generations whose parent stage already EndSpan'd (multimodal
  fan-out pattern). knowledge_post_process closes its postSpan
  via SkipSpan on the cancel/deleting entry guard so a worker
  that opens a span AFTER the cancel sweep doesn't leak it.
- Housekeeping and resetPendingTasks sweep finalizing rows
  identically to processing so a crash/restart can't strand them.
- DeleteKnowledge/DeleteKnowledgeList proactively dequeue
  downstream tasks via the same TaskInspector path.
- ChunkExtractService gets a cancel entry guard so the most
  expensive enrichment (graph extract) bails immediately when the
  parent knowledge is aborted.

Frontend

- New cancelKnowledgeParse API client + "Stop parsing" entry in
  both list view and card view more menus, gated on
  pending/processing/finalizing.
- Polling predicate refactored to a shared isParseInFlight helper
  that recognises `finalizing` (previously the doc list silently
  stopped polling once parse_status flipped from processing).
- Knowledge processing timeline: isPolling includes finalizing,
  new isHardTerminal short-circuits LIVE for cancelled/failed/
  completed so stranded child spans cannot pin LIVE on.
- DocumentListView.computeStatus distinguishes finalizing
  ("增强中") from completed and shows the previous "生成摘要中"
  copy when summary_status is still pending under finalizing.
  Added cancelled badge as well.
- i18n: statusFinalizing / statusCancelled / cancelParse* keys
  across zh-CN, en-US, ko-KR, ru-RU.

Docs / SDK

- docs/api/knowledge.md: documents the new finalizing state,
  cancel-parse semantics, and which statuses accept cancel.
- client (Go SDK): CancelKnowledgeParse with docstring listing
  the cancellable statuses.
2026-05-28 20:16:02 +08:00
wizardchen 4db6e69b4e fix(knowledge): close root span on terminal state, enrich stage metadata
The root span created by OpenAttempt was never closed: PostProcess only
ended the postprocess stage, so the root row stayed at status=running
forever even after parse_status flipped to completed/failed. The
timeline rendered "进行中" indefinitely on the root, defeating the
whole "is the document done" question the timeline is meant to answer.

- SpanTracker.FinalizeAttempt(kid, attempt, status, output, code, msg):
  closes the root row idempotently. Re-closing a terminal root no-ops
  so success / cascade-fail / dead-letter paths can fire without
  coordination.
- PostProcess.Handle calls FinalizeAttempt(done) after EndSpan(postprocess)
  on the success path. Async downstream work (summary/question/wiki/
  graph) still records its own spans; their completion extends the
  trace's wall-clock end-time but doesn't reopen the root.
- FailSpan auto-closes the root when a MAIN pipeline stage fails
  (docreader / chunking / embedding / multimodal / postprocess).
  Cascade-cancelled siblings stay closed-with-the-cascade as before.
- Dead-letter callback (router/task.go) accepts the SpanTracker via
  DI and calls FinalizeAttempt(failed, TASK_TIMEOUT) when a
  document-related task exhausts retries. The probe payload now
  extracts the Attempt field that Document/Manual/PostProcess
  payloads already thread through.

Stage spans were also being recorded with nil input/output, leaving
the new detail panel with timestamps only. Each Begin/End site now
emits useful work metrics:
- DocReader input: file_name, file_type, is_url, url
              output: text_length, images_found, is_audio, pages
- Chunking input: chunks_planned
            output: chunks_written, total_text_chars
- Embedding input: chunks_to_embed, model_id, dim
             output: vectors_written, storage_bytes
- Multimodal input: image_count, enable_ocr, enable_caption
- PostProcess output: chunks_total, enqueued_summary, _question, _wiki, _graph

i18n: add knowledgeStages.root ("Knowledge processing") so the UI
can render a localized name instead of the raw span identifier.
2026-05-28 15:14:45 +08:00
wizardchen c82b098f44 fix(knowledge): address span-tree review feedback
- Collapse migrations 000052 (flat stages) + 000053 (span tree) into a
  single 000052_knowledge_processing_spans migration; the flat stages
  table never escaped this branch and the create-then-drop sequence had
  no value.
- BeginStage: detect an existing (kid, attempt, stage) row before
  inserting and reuse its span_id with reset state, so re-entry from
  asynq retries or adjacent code paths no longer produces duplicate
  timeline segments.
- FailSpan: when sibling-stage cascade flips a dependent stage to
  cancelled, also CancelDescendants on its subtree so already-running
  subspans (embedding.batch[i] etc.) cannot remain as orphan running
  rows under a cancelled parent.
- Dead-letter callback: replace two sequential UpdateKnowledgeColumn
  writes with a single UpdateKnowledgeColumns map update so we cannot
  end up with parse_status=failed and stale error_message (or vice
  versa) when one of the two writes fails.
- touchKnowledgeHeartbeat: skip subspan/generation transitions; only
  root and stage transitions poke knowledge.updated_at. Spans-table
  MAX(updated_at) already covers subspan progress for housekeeping, so
  this avoids 2*N+ UPDATE bursts on the same hot row when a multimodal
  stage fans out to many images.
- Add regression tests for the BeginStage idempotency contract and the
  cascade-into-subspans behaviour.
2026-05-28 15:14:45 +08:00
wizardchen c9941a6688 refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:

  1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
     are independent of each other, both downstream of Chunking, both
     upstream of PostProcess. The flat (knowledge_id, stage) table
     couldn't represent that, so a Chunking failure left dependents
     stranded as "pending" forever instead of being marked as
     impossible-to-run.

  2. No history across attempts. A reparse erased the previous run's
     status before the new run started, leaving operators with no way
     to investigate "why did this fail twice?".

  3. Stages had only status + duration. Operators want to know how big
     the work was — pages parsed, chunks created, tokens embedded, VLM
     calls made — to distinguish "slow because the file is huge" from
     "slow because the docreader is wedged".

  4. Multimodal fans out N image tasks; Embedding fans out M batches;
     PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
     is interesting on its own (Langfuse already captures this for
     LLM calls). The flat model couldn't express it.

The redesign mirrors Langfuse's trace/span/generation hierarchy:

  * Migration 000053 supersedes 000052: knowledge_processing_spans
    table with (knowledge_id, attempt, span_id) primary key, plus
    parent_span_id, kind ∈ {root, stage, subspan, generation},
    status ∈ {pending,running,done,failed,skipped,cancelled}, and
    JSONB input/output/metadata fields.

  * SpanTracker (replacing StageTracker) exposes OpenAttempt /
    BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
    LookupStage. Cross-process workers (image_multimodal) get the
    parent's attempt + span via payload + LookupStage so subspans
    attach correctly.

  * StageDependencies declares the DAG; FailSpan now cascades —
    descendants of the failed span and dependent stages are flipped
    to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
    blast radius instead of orphan spinners.

  * Reparse now calls OpenAttempt up front so the timeline reflects
    "new attempt, all pending" instead of letting the previous run's
    status linger until the worker picks up the task.

  * Image_multimodal records each image as a generation subspan with
    its own success/failure on the parent attempt's multimodal stage.
    The finalize-on-last-attempt counter logic is preserved unchanged.

  * GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
    a tree shape with synthesized pending placeholders so the
    frontend always renders five timeline segments. ?attempt=N
    enables history navigation.
2026-05-28 15:14:45 +08:00
wizardchen 04f56f9cda feat(knowledge): track per-stage parsing progress with /stages API
Adds a five-segment progress model for the document parsing pipeline so
the UI (PR③) can render a timeline showing where each document is
(DocReader → Chunking → Embedding → Multimodal → PostProcess) and
which stage failed with what error code.

- New table `knowledge_processing_stages` (migration 000052) with one
  row per (knowledge_id, stage). UPSERT on Begin/Done/Fail bumps an
  attempt counter so re-parses don't lose history.

- StageTracker service exposes Begin/Done/Fail/Skip; all calls are
  best-effort and never break the pipeline if persistence fails.

- Stable error codes (DOCREADER_TIMEOUT / EMBEDDING_RATE_LIMIT /
  VECTORSTORE_WRITE_FAILED / ...) the UI can map to localized
  remediation hints.

- Tracker call sites added at the four meaningful failure points:
  convert (DocReader), CreateChunks (Chunking), BatchIndex (Embedding),
  enqueueImageMultimodalTasks (Multimodal start),
  KnowledgePostProcess.Handle (Multimodal close + PostProcess).

- New endpoint `GET /api/v1/knowledge/:id/stages` returns the five
  canonical stages — missing rows are synthesized as "pending" so
  the timeline always renders five segments. Includes current_stage
  and last_error block.
2026-05-28 15:14:45 +08:00
wizardchen 3ae3ea97c5 fix(knowledge): prevent documents from getting stuck in "processing"
Several failure modes left Knowledge.parse_status pinned at "processing"
forever, with no signal to users beyond a permanent spinner. This commit
addresses the root causes and adds a safety net.

- Asynq worker pool: explicit Concurrency (default 16, env-tunable via
  WEKNORA_ASYNQ_CONCURRENCY) so batch uploads don't queue behind a
  CPU-count-sized worker pool. Redis op timeouts raised to 500ms/1000ms
  (WEKNORA_REDIS_OP_TIMEOUT_MS) to absorb bursty multimodal counter ops.

- DocReader RPC: cap each call with WEKNORA_DOCREADER_CALL_TIMEOUT
  (default 30m). Without this, a hung docreader pinned a worker for the
  full DocumentProcessTimeout window.

- ImageMultimodal: finalize-on-last-attempt semantics. A permanently
  failing single image no longer strands the parent — the asynq retry
  is allowed to run, but on the final attempt we count the image
  regardless of outcome. Redis DECR errors fall back to enqueuing the
  post-process task instead of returning silently.

- Dead-letter callback: when DocumentProcess / KnowledgePostProcess /
  ManualProcess exhausts retries, immediately mark the corresponding
  Knowledge as failed with the last error. This surfaces the failure
  in the UI without waiting for the housekeeping sweep.

- HousekeepingService: 5-minute cron that flips knowledge rows stuck
  in "processing" past DocumentProcessTimeout + 10m to failed, plus
  summary rows stuck > 1h. Catches anything the other safety nets
  miss (worker SIGKILL mid-handler, etc.). Disable with
  WEKNORA_HOUSEKEEPING_ENABLED=false.

- Distributed startup recovery: previously the post-restart sweep was
  skipped whenever REDIS_ADDR was set, even though Asynq does not
  reschedule the task that was actively running on the dead instance.
  Now the sweep runs in distributed mode too, but only against rows
  older than 30 minutes to avoid racing peer instances.
2026-05-28 15:14:45 +08:00
wizardchen ae4ec0cf06 fix(im): make presigned URL flow diagnosable end-to-end
When IM image rendering breaks, operators previously had no log line to
correlate against the IM platform's fetch attempt. Add observability
hooks on both ends and unblock HEAD probes so common IM previews work
at all.

- log rewriteStorageURLs success/failure/no-op with the full signed URL
  (operators can copy it from logs and verify public reachability)
- log presigned handler 4xx with client_ip + UA + tenant_id + file_path
  so failures correlate against IM platform fetch logs; use the request
  context so trace IDs are preserved
- accept HEAD on /api/v1/files/presigned: IM platforms (Feishu, Slack
  etc.) probe with HEAD before GET when rendering image previews, and a
  401 there is enough to break the inline image even when the GET would
  have succeeded
- add Admin-only GET /api/v1/files/presigned-preview that returns the
  exact URL an IM channel would embed for the calling tenant, for
  self-service verification without sending a real IM message
- clarify APP_EXTERNAL_URL and MINIO_ENDPOINT public-reachability rules
  in .env.example; misconfigured endpoints are the most common cause of
  "image broken in IM" reports
2026-05-28 08:03:57 +08:00
lyingbug 3611a8607d feat(rbac): add multi-use share-link invitations for invite_only mode (#1504)
* feat(rbac): add multi-use share-link invitations for invite_only mode

When the auth.registration_mode toggle flips to invite_only, the public
/auth/register endpoint returns 403 — but until now there was no
channel for unregistered users to actually join a tenant. The existing
invitation table required a registered invitee, so new emails could
not enter the system at all.

This change introduces a share-link model:

- Owner generates a multi-use registration link with a per-tenant role
- Recipient opens the link, registers with their own email, and is
  added to the tenant on the same request
- Token is stored in plaintext (not hashed) so the management UI can
  re-display the URL on demand without a "copy now or revoke" trap.
  Threat model is bounded by 7-day TTL, revocability, and the fact
  that all the link grants is membership in one tenant.
- accepted_count tracks how many users joined through the link so the
  Owner can tell whether a link is fresh or has already been spread

Per-user invitations (registered email -> in-app inbox accept) are
unchanged. Share-link rows use empty invitee_user_id as discriminator;
the partial unique index on (tenant_id, invitee_user_id) was relaxed
to skip empty values so multiple share links can coexist per tenant.

Frontend reuses Login.vue for /register?token=xxx instead of a
parallel page, and adds icon-only revoke / remove actions plus a
distinct "active" status tag for share-link rows.

Migration 000054 adds the token column, the accepted_count column,
and the relaxed pending unique index in a single step.

* feat(auth): update invitation token handling and add rate limiting

- Refactor the invitation token retrieval to use POST instead of GET, enhancing security by preventing the plaintext token from appearing in logs and browser history.
- Update the API endpoint from `/auth/invitations/:token` to `/auth/invitations/lookup` to reflect the new method.
- Introduce a rate limiter for unauthenticated share-link endpoints to mitigate brute-force attacks and abuse, ensuring a maximum of 30 requests per minute per IP.
- Adjust the registration flow to maintain user experience while securing the invitation process.
2026-05-27 19:42:01 +08:00
wizardchen acdc0a526f feat(system): expose platform audit log + polish audit drawers
The system_settings update, system admin promote/revoke, and
apply-default-storage-quota routes have been writing audit rows since
the prior commits, but with TenantID=0 (system-scope). The per-tenant
GET /tenants/:id/audit-log endpoint filters by tenant_id and never
returns them, so until now those rows existed only in the DB with no UI
surface. This commit closes the loop:

Backend:
- GET /api/v1/system/admin/audit-log: new SystemAdmin-gated endpoint
  reusing AuditLogService.List with tenant_id=0. Same cursor-paged
  shape and query params (after_id / limit / action / outcome / actor)
  as the per-tenant feed, so the frontend reuses the same client logic.
- Wired through RegisterSystemAdminRoutes (mounted on the existing
  adminRoutes group so it inherits the SystemAdmin() guard). The
  handler dependency is optional: nil auditLogHandler skips the route,
  mirroring RegisterTenantRoutes' /audit-log handling.

Frontend — new platform audit drawer in SystemSettings.vue:
- "审计日志" entry button in the section header opens a side drawer
  (880px) listing system-scope events. Lazy-loaded on first open;
  refresh is explicit via a button inside the drawer.
- Table columns: stacked date/time (so 50 events in the same minute
  remain distinguishable), stacked actor/role, action tag, structured
  target (subject key + diff line), outcome. The dead request column
  (system actions don't go through middleware path capture) is dropped
  in favour of richer target rendering.
- Per-action target formatters:
    * system.setting_changed: subject = registry key, diff = `old → new`
      (JSON-encoded, 80-char truncation). Reset shows `old → (空)`.
    * tenant_storage_quota bulk apply: subject = "批量同步", diff =
      "applied to N tenants (X GB)".
    * system.admin_promoted / revoked: subject = "name (email)", diff
      annotates idempotent / noop branches so an audit reader can tell
      a real grant from a probe.
- Click-to-expand row reveals the full audit context: actor UUID,
  target_user_id / target_type / target_id, and raw details JSON in
  monospaced scroll-capped block. No psql round-trip needed for
  forensic spot checks.
- Sticky thead pinned to the scroll container so column labels survive
  long scrolls. Cells vertical-aligned middle to keep single-line tag
  cells visually balanced against multi-line target cells. No zebra
  stripes — the stacked content already provides row separation, and
  stripes on top read as noise.

Frontend — same polish back-ported to TenantMembers.vue audit drawer:
- Same drawer width, stacked time / actor cells, structured target +
  diff layout, expandable raw-details row, sticky thead, vertical-
  align middle, no stripes. Refresh button reformulated as a text
  button with label (was an outlined square icon-only).
- request_path column kept (rbac.access_denied carries meaningful
  paths) but empty values render as a placeholder dash so they don't
  read as broken.
- Diff line now covers rbac.invitation_sent / invitation_revoked role
  in addition to the existing role_changed / access_denied details.

API:
- frontend/src/api/system/index.ts: listSystemAuditLog() reuses the
  AuditLog / ListAuditLogParams types from @/api/tenant/audit-log
  (re-exported) so consumers don't need to cross-import.

i18n (zh-CN / en-US / ko-KR / ru-RU):
- system.globalSettings.audit.*: full drawer copy + per-action labels
  (system.setting_changed / admin_promoted / admin_revoked) + target
  diff templates + expanded-row labels.
- tenantMember.audit.expanded.*: expanded-row labels added so the
  shared drawer treatment renders cleanly under tenant scope.
2026-05-26 21:13:56 +08:00
wizardchen 9cce0c8e5e feat(system): consolidate system admin and settings into one Settings panel
Replace the standalone /platform/system/* routes with a single
"系统设置" section inside the canonical Settings modal. The previous
SystemLayout.vue / SystemAdmins.vue surfaces are removed and their
functionality (system admin roster, global settings) is hosted directly
in SystemSettings.vue under the standard `.section-header` /
`.settings-group` skeleton. Legacy URLs redirect to the modal section
so external bookmarks don't 404.

Backend:
- SystemSettingService.Reset + repo.Delete: drop the DB override for a
  key so the 3-tier resolver falls back to ENV / built-in default.
  Idempotent (resetting a never-persisted key returns nil); emits an
  audit row only on real deletions, invalidates the local cache, and
  publishes to peers via the existing pubsub channel.
- TenantService.BulkSetStorageQuota: overwrite every tenant's
  storage_quota in one statement. Powers POST /system/admin/tenants/
  apply-default-storage-quota; bypasses the per-tenant whitelist on
  PUT /tenants/:id which intentionally forbids storage_quota edits.
- AuditAction{SystemAdminPromoted,SystemAdminRevoked} constants and
  emitAdminAudit() in SystemHandler — promote / revoke now leave an
  audit trail with TenantID=0 and {target_email,target_username,
  idempotent|changed} in details.
- SystemSetting.LastModifiedByName: derived per-request display label
  (username, email fallback) so the UI shows "wizardchen" instead of
  a UUID prefix without storing a denormalised column.

Frontend:
- SystemSettings.vue rewritten against the Settings modal skeleton with
  auto-persisting controls (switch/select @change, input/inputnumber
  @blur, tag-input + per-delta popconfirm for SSRF whitelist and admin
  roster). auth.registration_mode change goes through an inline
  popconfirm; cancel rolls back. Reset / bulk-apply also inline
  popconfirms — no dialog modal for these per-row affordances.
- Priority hint panel surfaces the DB > ENV > default resolver order
  so operators can reason about "I set the env but it doesn't show up".
- Router: /platform/system, /platform/system/settings, /platform/system/
  admins are now compatibility redirects to /platform/settings?section=
  system-global.
- Settings modal sized 900x700 → 1080x780 and content-wrapper 600 → 760
  so the wider tables (members, system settings) breathe; <1100px
  viewport still flexes to the screen.
- i18n: system.globalSettings.* keys for title / description / loading /
  empty / badges / priority hint / per-key labels / reset / bulkApply /
  admins (label, placeholder, save messages, popconfirm copy) across
  zh-CN, en-US, ko-KR, ru-RU.

Misc:
- internal/utils/filesize.go doc: clarify MAX_FILE_SIZE_MB is a
  deploy-time-only knob (nginx + docreader + frontend each cache the
  env at startup); a SystemAdmin UI override would mislead operators
  because nginx would still 413. Until all four layers can hot-reload
  the limit in lockstep, this stays env-only.
- internal/utils/security.go: SSRF whitelist parser/runtime now drives
  off SystemSettingService for live updates; ENV remains the fallback
  for never-overridden deployments.
2026-05-26 21:13:56 +08:00
wizardchen 47a183aa65 feat(system-admin): implement bootstrap for system admin promotion and enhance system settings management
- Added WEKNORA_BOOTSTRAP_SYSTEM_ADMIN_EMAIL environment variable to promote a specified user to system admin on startup.
- Introduced a new bootstrap process in `bootstrap.go` to handle the promotion logic.
- Updated `.env.example` to document the new environment variable and its behavior.
- Created new views for managing system administrators and system settings, including listing, promoting, and revoking admin privileges.
- Enhanced the frontend to reflect the new system admin features, including UI elements for admin management and settings configuration.
- Updated API interfaces to support system admin functionalities, ensuring proper data handling and user management.
2026-05-26 21:13:56 +08:00
wizardchen 7528175a74 feat(knowledge-base): implement per-user pinning for knowledge bases
Refactored the pinning mechanism for knowledge bases to be user-specific, allowing any user with read access to pin knowledge bases independently. This change includes the introduction of a new `user_kb_pins` table to manage pin states, replacing the previous tenant-wide pinning system. Updated relevant API endpoints and service logic to support this new functionality, ensuring that the UI reflects the user's personal pin status.

Additionally, adjusted the knowledge base listing to sort by user-specific pin status, enhancing the user experience by prioritizing pinned items.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 68ac7fb973 feat(user-preferences): implement user preferences management with backend support
Added functionality for managing user preferences, including the ability to update preferences via a new API endpoint. The preferences are now stored server-side, allowing for synchronization across devices. Updated relevant components to handle user preferences, including the UserMenu and settings views. Enhanced internationalization support for error messages related to preference updates.
2026-05-18 17:28:58 +08:00
wizardchen 8b681e4a2e refactor(GlobalCommandPalette): streamline template structure for improved readability
Consolidated the template structure in GlobalCommandPalette.vue to enhance code clarity and maintainability. Adjusted the layout of various elements for better alignment without altering existing functionality. This refactor aims to improve the overall readability of the component for future development.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 884ac44283 feat(invitations): implement pending invitations feature with polling and dialog
Added functionality for managing pending invitations, including a polling mechanism to update the invitation count in real-time. Introduced a new dialog component for users to view and respond to their invitations without navigating away from their current context. Updated API endpoints for fetching and managing invitations, and enhanced the user interface with relevant internationalization support.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 8332a0dab6 feat(favorites): implement user favorites and recents functionality
Introduced a new API client for managing user favorites, allowing users to list, add, and remove starred resources. Enhanced the sidebar to display favorites and recents, improving user navigation. Updated internationalization files to include relevant labels and tooltips for the new features across multiple languages.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 29dc99ae79 feat(tenant): implement self-service tenant creation functionality
Added a new API endpoint for creating tenants, allowing any logged-in user to create their own workspace. The backend automatically assigns the creator as the Owner of the new tenant. A new CreateTenantDialog component was introduced in the frontend, enabling users to input the workspace name and an optional description. Internationalization files were updated to include relevant messages for this feature across multiple languages.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen dc8d033b4d feat(organization): transition membership model to tenant-level and enhance invite functionality
Updated the organization membership model to operate at the tenant level, aligning with Plan 3. This change introduces new interfaces and modifies existing ones to accommodate tenant-centric operations, including the `InviteMemberRequest` and `TenantInviteCandidate`. The search functionality for inviting members has been updated to reflect this shift, allowing for tenant-based searches instead of user-based. Additionally, internationalization files have been updated to ensure consistent messaging across languages regarding tenant membership and roles.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 7e20b9a57e fix(rbac): gate remaining organization mutating routes with Admin+
Plan 3 (#1303) fixed several Org mutating routes (POST /organizations,
/join, /join-request, /join-by-id, /:id/leave, /:id/request-upgrade)
to Admin+ in the caller's tenant, and e1deb810 followed up by gating
DELETE /:id/members/:tenant_id symmetric with /:id/leave.

The remaining Org-admin surfaces still relied solely on a
service-layer "is the caller's tenant admin in this org?" check, with
no route-layer floor. Add g.Admin() to:

  - PUT    /organizations/:id                          (update org)
  - DELETE /organizations/:id                          (delete org)
  - POST   /organizations/:id/invite-code              (rotate code)
  - GET    /organizations/:id/search-users             (member picker)
  - POST   /organizations/:id/invite                   (direct invite)
  - PUT    /organizations/:id/members/:tenant_id       (change role)
  - GET    /organizations/:id/join-requests            (review queue)
  - PUT    /organizations/:id/join-requests/:request_id/review

Each of these takes an action that is visible to the whole org or to
another tenant; per the RBAC matrix in router/rbac.go they belong on
Admin+. The service-layer check stays as the authority on org-level
membership; the route guard prevents a tenant Viewer/Contributor from
ever reaching the service in the first place.

Behaviour is unchanged when cfg.Tenant.EnableRBAC is false: RequireRole
continues to log the would-be rejection and let the request through.
2026-05-18 17:28:58 +08:00
wizardchen 8efe1b4187 refactor(rbac): address kb-access guard review findings
Follow-up to the route-level KB-access guard introduced earlier in
this PR. Five reviewer findings, all fixed in middleware + tests
without changing handler call sites:

  - Honour `?agent_id` in the agent-share fallback. The in-handler
    resolution in knowledgebase.go branches on a specific agent's
    KBSelectionMode (all / selected / none); the guard was only ever
    calling TenantCanAccessKBViaSomeSharedAgent, which let some
    requests pass at the guard layer that the handler would then
    reject — and rewrote the request's tenant context to the source
    tenant before the rejection happened. The guard now mirrors the
    handler: a specific agent_id pins the resolution to that agent
    (no fallback to any-agent), an empty agent_id keeps the existing
    any-agent behaviour. The knowledge.go / knowledgebase.go handler
    helpers stay in place as defence in depth until they're folded
    into the guard in a follow-up.

  - Distinguish not-found vs. transient on the knowledge / chunk
    resolvers. A DB hiccup used to surface as a 404, both confusing
    clients and hiding the underlying failure from monitoring. The
    resolvers now return apperror 404 only for the known not-found
    sentinels (apprepo.ErrKnowledgeBaseNotFound, ErrResourceNotFound)
    and let other errors propagate; the guard maps those to a fresh
    NewServiceUnavailableError (503).

  - Stop 500-ing on legacy chunks with empty knowledge_base_id. The
    chunk is effectively unresolvable to a KB, so the client gets the
    same 404 they'd get for a missing chunk instead of a 500 that
    pollutes alerting. The warn log preserves the operator signal.

  - Honour cfg.Tenant.EnableRBAC. The guard now mirrors RequireRole /
    RequireOwnershipOrRole: with enforcement off, 401/403 paths log
    the would-be rejection and pass through. 404 still fires either
    way (a missing resource is not an authorisation event). Keeps the
    rollout window safe — operators can flip enforcement on globally
    without code changes elsewhere.

  - Document the c.Keys vs. c.Request.Context() split. The guard
    intentionally rewrites only the request context (handlers that
    still run their own share resolution, currently knowledge.go /
    knowledgebase.go, would otherwise see kb.TenantID == c.Keys
    tenant and misclassify shared-Viewer access as Admin). The
    package-level comment now spells this out so the migration of the
    remaining handlers in a follow-up doesn't accidentally regress
    `my_permission` rendering.

Tests:

  - Existing four cases keep working; runGuard signature gained a
    guardOpts knob.
  - Eight new cases cover the agent-share resolution paths (any-agent
    viewer, Editor required → must reject, specific agent in modes
    all / selected-match / selected-miss / none, cross-tenant
    mismatch) plus the EnableRBAC=false fail-open + 404-still-fires
    pair.
2026-05-18 17:28:58 +08:00
wizardchen ef01f99e69 refactor(rbac): extend KB-access guard to chunk / knowledge / knowledgebase routes
Follow-up to the FAQ + Tag refactor in this same PR. Three additional
handler areas now run through the route-level g.KBAccess* guard:

  - chunk.go  : full refactor — helper effectiveCtxForKnowledge and
                validateAndGetChunk's access check are gone; the
                handler's chunk-fetch + ownership-mismatch check stay
                as defence-in-depth (a same-tenant attacker can't
                pass mismatched knowledge_id + chunk_id).
  - knowledge.go and knowledgebase.go : routes now run the guard
                BEFORE the handler. The in-handler
                validateKnowledgeBaseAccess /
                resolveKnowledgeAndValidateKBAccess helpers stay for
                now because they branch on the agent_id query param
                (specific-agent check vs. any-agent fallback) — that
                handler-specific branch can collapse into the guard
                in a follow-up. The duplicated DB roundtrip is
                negligible compared to the handler's main work.

New middleware exports:

  - KBIDFromChunkIDParam   : walks chunk_id → KnowledgeBaseID via the
                             chunk's denormalised column.
  - g.KBAccessReadFromChunkIDParam / g.KBAccessWriteFromChunkIDParam
                           : convenience methods on rbacGuards.

Route changes are purely additive (existing role / ownership guards
stay in place; the new KB-access guard runs after them and rewrites
the request's tenant context to the source tenant for shared KBs).
No new dependencies on the kbShareService / agentShareService at the
handler level for chunk.go — those move to the route layer.

Tests pass; behaviour unchanged.
2026-05-18 17:28:58 +08:00
wizardchen 1109cb7bff refactor(rbac): KB-access guard at the route layer (no helper in handlers)
Five handler files used to carry near-identical 30-line helpers
(effectiveCtxForKB / validateAndGetKnowledgeBase) that did the same
own-or-shared resolution before every KB-scoped operation:

  1. KB belongs to caller's tenant     -> grant own access
  2. Org-shared KB                      -> grant min(share, role) cap
  3. Shared agent carries the KB        -> grant Viewer (read-only)

A bug found in any one of them had to be fixed in five places, and
several already drifted apart (different log levels, slightly
different error messages, one path missing the share-cap check).

Move the resolution to a route-level gin.HandlerFunc:
middleware.RequireKBAccess, exposed on rbacGuards as
g.KBAccessRead("id") / g.KBAccessWrite("id"). The guard runs after
the role/ownership checks, then on success:

  - stashes the resolved (KB + effective tenant id + permission)
    on c.Keys under middleware.KBAccessContextKey, and
  - rewrites c.Request.Context() to carry the effective tenant id

so handlers downstream just read tenant the way they always did
(types.MustTenantIDFromContext) without a per-handler helper.

Refactor FAQ and Tag handlers to drop their helper methods; their
service constructors lose the kbShareService / agentShareService
dependencies (now lifted to the route layer). Routes:

  faq.GET("/entries",            g.Viewer(),         g.KBAccessRead("id"),  ...)
  faq.POST("/entries",           g.OwnedKBOrAdmin(), g.KBAccessWrite("id"), ...)
  kbTags.GET("",                 g.Viewer(),         g.KBAccessRead("id"),  ...)
  kbTags.POST("",                g.OwnedKBOrAdmin(), g.KBAccessWrite("id"), ...)

The middleware also exports KBIDFromKnowledgeIDParam so chunk routes
(URL :knowledge_id) can adopt the same pattern in a follow-up PR
without reshaping the helper.

Five new tests cover own-KB, not-found, shared-with-sufficient-perm,
shared-but-below-min, and missing-tenant-context paths. Handler
tests still pass (no behaviour change).
2026-05-18 17:28:58 +08:00
wizardchen eda6b1011a fix(rbac): close org-tenant gate gaps from Plan 3 review
Three follow-ups from #1350 review:

1. DELETE /organizations/:id/members/:tenant_id was un-gated, so a
   tenant Viewer could remove their own tenant from an org via the
   members endpoint and bypass the Admin+ requirement that POST
   /:id/leave already enforces. Gate it with g.Admin() — symmetric
   with /leave.

2. isOwnerTenant() returned (false, err) when the owner user lookup
   failed, and the two call sites only checked the bool. A deleted /
   unreadable owner user would silently fall through and let admins
   remove or re-role the would-be owner tenant. Make the helper
   fail-closed: any lookup error now returns (true, err), and both
   call sites refuse the destructive op on error.

3. POST /:id/request-upgrade was un-gated. Approval of the request
   changes the whole tenant's org role, so it is in the same trust
   tier as join/leave/create. Add g.Admin().
2026-05-18 17:28:58 +08:00
wizardchen 928f5abe3f feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"

This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.

Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:

  effective = min(share.Permission, tenant_org_role, tenant_role_cap)

where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.

Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.

Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.

Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-18 17:28:58 +08:00
wizardchen c6a2f6c8cc refactor(permissions): enhance role-based access controls across knowledge base and agent views
- Updated visibility conditions for various UI elements based on user roles, ensuring that only users with appropriate permissions (Admin, Contributor, or KB creator) can access certain actions.
- Introduced computed properties to manage permissions for sharing knowledge bases, managing data sources, and executing graph extraction.
- Adjusted the AgentList and KnowledgeBase components to reflect these changes, improving user experience by preventing unauthorized actions from being displayed.
- Ensured that the backend routes are aligned with the updated frontend role checks, enhancing security and clarity in user interactions.
2026-05-18 17:28:58 +08:00
wizardchen 619e14a2eb fix(rbac): wire chunk-id ownership chain for generated-question delete
DELETE /chunks/by-id/:id/questions used to be gated at flat
Contributor+ as a deliberate carve-out: the URL only carries the
chunk's own id, so the chunk -> knowledge -> KB ownership chain
wasn't reachable from request params alone. The carve-out had two
problems:

  - Inconsistent with every other chunk mutation. PUT
    /chunks/:knowledge_id/:id is OwnedKBOrAdmin (KB creator OR
    Admin+); a Contributor with no KB ownership can't update a
    chunk but COULD delete a generated question on the same chunk.
  - User-visible: a Contributor in tenant A who never created any
    KB still saw the 'delete question' button work, while every
    other chunk control 403'd.

Wire the missing hop:

  KBCreatorLookupFromChunkIDParam (handler) reads :id, fetches the
  chunk via the existing GetChunkByIDOnly (unscoped repo call),
  re-checks chunk.TenantID against the caller's tenant context to
  block cross-tenant chunk-id probing, then defers to the existing
  resolveKBCreatorByKnowledgeID helper for the rest of the chain.

  OwnedChunkKBOrAdminFromChunkID (router) wraps it in
  RequireOwnershipOrRole(Admin, ...) — same matrix as every other
  chunk/knowledge/wiki mutation guard.

  chunks.DELETE switches from g.Contributor() to the new guard.

4 unit tests cover happy path, chunk-not-found, cross-tenant
chunk-id probing, and KB-not-found upstream — same edge-case shape
as the existing KBCreatorLookup_* tests so future refactors get
caught loudly.
2026-05-18 17:28:58 +08:00
wizardchen f832bce16d docs(rbac): document Contributor vs OwnedXxxOrAdmin selection rule
Adds a checklist comment at the top of internal/router/rbac.go that
explains how to pick between the four role-only guards
(Viewer / Contributor / Admin / Owner) and the ownership-or-role
guards (OwnedKBOrAdmin and friends) when wiring a new route.

The two questions encoded here:

  Q1. Does the resource have a creator?
        YES  → mutating routes use OwnedXxxOrAdmin
        NO   → tenant-wide infra → Admin()
        ENTRY POINT (creating a brand-new owned resource) → Contributor()

  Q2. Is the side effect "private to me" or "visible to others"?
        PRIVATE → Contributor() is fine (e.g. POST /agents/:id/copy)
        PUBLIC  → OwnedXxxOrAdmin or Admin (sharing, transferring,
                  tenant-wide toggles)

The recurring failure mode caught in this PR — FAQ entries, tag CRUD,
KB share, agent share, /shared-agents/disabled all gated to plain
Contributor while the matrix demanded OwnedXxxOrAdmin — is the
"sub-resource drift" anti-pattern: someone wires a new sub-resource
with a freshly-invented gate instead of inheriting from the parent
KB/Agent's matrix.

Also documents the user-facing invariant the guards together produce
("Contributor in their own KB acts like Owner; Contributor in someone
else's KB acts like Viewer") so reviewers have a UX-level sanity check
beyond the role labels.
2026-05-18 17:28:58 +08:00
wizardchen fb947205b7 fix(rbac): align FAQ and tag mutating routes with KB ownership matrix
FAQ entries (CRUD on /knowledge-bases/:id/faq/entries) and KB tags
(/knowledge-bases/:id/tags) were gated to plain Contributor+, while
the KB main routes plus chunks/wiki sub-resources require
OwnedKBOrAdmin (creator OR Admin+). The mismatch let any Contributor
mutate FAQ entries and tags inside someone else's KB even though
they could not edit the KB's other content.

Both are KB child resources by design: FAQ entries are the body of
FAQ-type KBs, and tags categorise KB content. Touching them affects
the KB owner's organisation of their KB, so they belong on the same
matrix as chunks and wiki pages — Contributor with KB ownership can
manage their own, others need Admin+.

Sweep checked router gates and handler/service tenant scoping for
the rest of the surface (Models / MCP / WebSearchProvider /
VectorStore / DataSource / WeKnoraCloud / IM channels). All Admin+
routes correctly load tenant_id from context, and per-id mutating
service calls go through (id, tenantID) primary keys, so a Viewer
cannot reach those even with a guessed cross-tenant id.
2026-05-18 17:28:58 +08:00
wizardchen 96de7e3976 fix(rbac): gate sharing routes; tenant-disable shared agent → Admin+
Three more mutating routes were ungated:

- POST /shared-agents/disabled — sets the "disabled by me" preference
  for a shared agent. Despite the "by me" naming this writes a
  tenant-level row (tenant_disabled_shared_agents) that hides the
  agent from every member of the tenant's chat dropdown. So a Viewer
  could silently nuke an agent for the entire tenant. Now Admin+
  only.
- POST /knowledge-bases/:id/shares — sharing a KB to an organization
  exposes its content to every member of that org. The service layer
  only checked "is this KB inside the caller's tenant", so any Viewer
  could share any tenant KB. Gate matches the KB-update matrix:
  OwnedKBOrAdmin (creator OR Admin+).
- POST /agents/:id/shares — same problem, same fix:
  OwnedAgentOrAdmin. PUT /shares/:share_id and DELETE /shares/:share_id
  for both KB and agent share routes get the same matrix — changing
  permission or revoking a share is at least as sensitive as
  creating one.

GET /knowledge-bases/:id/shares and GET /agents/:id/shares stay
Viewer+ since they're read-only.

The /organizations/* routes are intentionally left unchanged: they
implement organization-level RBAC (org owner / admin / editor /
viewer) inside the service layer, which is a different matrix from
tenant RBAC and not in scope for this PR.
2026-05-18 17:28:58 +08:00
wizardchen 0a44077498 fix(rbac): plug ungated mutating routes; log RBAC state at startup
Sweep over the route table found four classes of mutating endpoints
that had no role guard:

- POST /tenants — creating a brand-new tenant. Now g.CrossTenant():
  only org-level superusers may create tenants, matching /tenants/all
  and /tenants/search.
- POST /initialization/initialize/:kbId, PUT /initialization/config/:kbId
  — these change a KB's embedding/parser/storage configuration, which
  is at least as sensitive as PUT /knowledge-bases/:id. New
  KBCreatorLookupFromKbIDParam (sister of KBCreatorLookup but reading
  :kbId) lets us reuse the OwnedKBOrAdmin matrix; routes now use the
  new g.OwnedKBOrAdminFromKbIDParam() guard.
- POST /initialization/{ollama,remote,embedding,rerank,asr,multimodal,
  extract}/* — system-level model probes / downloads / extraction
  pipelines. These trigger LLM calls and outbound network fanout
  with tenant credentials; gated to Admin+. The corresponding GETs
  (status / list / progress) drop to Viewer+ since they're read-only
  observability.
- POST /system/{parser-engines,docreader,storage-engine}/* — same
  pattern: GETs that report state are Viewer+, the *-check /
  /reconnect actions are Admin+.

POST /evaluation/ runs the eval harness which fans out LLM calls
across the tenant — gated to Admin+.

Operational improvement: log the resolved tenant RBAC state at
LoadConfig completion. air's hot-reload only rebuilds the binary on
Go-source changes; it does NOT re-source .env, so flipping
WEKNORA_TENANT_ENABLE_RBAC=true while the dev loop is already
running has no effect until the parent dev-app process restarts.
This was a real diagnostic loop in PR review — printing the value at
startup makes the "I edited .env but the gates still aren't firing"
trap obvious from the first console line.

The Printf includes both the resolved cfg.Tenant.EnableRBAC value
and the raw env var, so a mismatch between "what the .env file
says" and "what this process actually inherited" is one glance.
2026-05-18 17:28:58 +08:00
wizardchen 766a6c482f feat(rbac): per-tenant audit log for RBAC events (PR 6, #1303)
PR 1-5 of the multi-tenant RBAC series enforce the role matrix; what
they don't do is keep a durable record of who did what. Today the
RBAC middleware logs "[rbac] role insufficient ..." to stderr -
perishable, not searchable. Operators cannot answer "who removed
Alice from tenant T-foo on Tuesday" or "did we 403 a probing user
200 times last hour" after the fact.

This PR adds a generic per-tenant audit log that:

- Captures mutating RBAC actions (member add/remove/role-change/leave)
  durably from the tenant_member service after the repo write succeeds.
- Captures enforcement rejections (RequireRole / RequireOwnershipOrRole
  denies under EnableRBAC=true) via a gin-context-injected hook.
- Sliding-window dedup (1 minute) on (tenant, actor, action, path)
  for denied events so a probing client cannot flood audit_logs.
- Default-on - no extra env flag. Schema is generic enough for future
  KB / agent / datasource events to reuse via new action namespaces
  without another migration.

Schema (migration 000044) mirrors wiki_log_entries' shape: a single
audit_logs table with action, target_type/id, request path/method,
outcome, and a JSONB details column. Three indexes: (tenant_id, id
DESC) for the per-tenant cursor feed, actor_user_id for "what did
Alice do", (tenant_id, action) for both the action filter and the
dedup lookup. Sqlite mirror added for Lite mode.

Wiring is nil-safe: the audit field on tenantMemberService and the
gin-context lookup in rbac.go both tolerate a nil service so existing
tests construct without a stub and dormant deployments degrade
gracefully if the provider is misconfigured.

The query API is GET /tenants/:id/audit-log, gated by Admin+ on top
of the existing PathTenantMatch (denial histories should not surface
to ordinary members). Cursor-paginated by descending id (sidesteps
the duplicate-timestamp tie-breaking that BIGSERIAL trivially solves).

Frontend: TenantMembers.vue gets a t-tabs wrapper with Members
(existing) and Audit log (new) tabs; the audit tab is gated to
Admin+ to mirror the server, lazy-loads on first open, paginates via
the next_cursor pattern, and renders coloured chips per
action/outcome so an operator can scan for anomalies. i18n keys
added in all four locales.

Tests: 14 new tests across service (6), middleware (4), and handler
(4). Service tests pin the dedup window behaviour (writes again
after expiry, dedup is per-actor-per-path, degrades on lookup
error). Middleware tests pin both reject sites' audit hook and the
"dormant mode does not fire" + "nil audit does not panic"
invariants. Handler tests pin the cursor envelope and that an empty
page returns next_cursor=0.

Out of scope: per-action ACL on the audit log, retention/TTL job,
KB/agent/datasource event wiring (schema supports it - just no
emitters yet), async writes, frontend filter UI.
2026-05-18 17:28:58 +08:00
wizardchen d42a41b784 feat(rbac): per-KB ownership for knowledge / chunks / wiki pages (#1303)
PR 1–4 of the multi-tenant RBAC series gated KB-level and Agent-level
mutations behind a "creator OR Admin+" check via OwnedKBOrAdmin /
OwnedAgentOrAdmin. The layer underneath — knowledge entries, chunks,
wiki pages — was still gated only by Contributor, which is wider than
intended: any tenant Contributor could edit/delete documents in a
private KB they didn't own. PR 2 / wiki author left explicit follow-up
breadcrumbs at the three call sites in router.go acknowledging this gap.

This PR closes those breadcrumbs by reusing the existing scaffolding:

- KnowledgeService.GetOwningKBCreatorID walks knowledge_id -> kb_id ->
  KB.CreatorID, tenant-scoped through GetKnowledgeByID. The chain lives
  in the service layer so the handler-side lookups stay pure adapters.
- Three new CreatorLookup closures (KBCreatorLookupFromKnowledgeID,
  KBCreatorLookupFromKnowledgeIDParam, KBCreatorLookupFromKBPath) are
  added in handler/rbac_lookups.go alongside the existing
  KBCreatorLookup / AgentCreatorLookup. Each translates the repository
  sentinels into middleware.ErrResourceNotFound and re-checks tenant
  scope as defence-in-depth, mirroring KBCreatorLookup exactly.
- router/rbac.go gains three guard factories — OwnedKnowledgeKBOrAdmin,
  OwnedChunkKBOrAdmin, OwnedWikiKBOrAdmin — wired through the same
  rbacGuards bundle.
- Mutating routes in RegisterKnowledgeRoutes / RegisterChunkRoutes /
  RegisterWikiPageRoutes switch from g.Contributor() to the new
  ownership guards. Cross-KB batch operations (POST /knowledge/move,
  POST /knowledge/batch-delete, PUT /knowledge/tags) stay
  Contributor-gated since there's no single owning KB to check
  against. KB-scoped uploads (POST /knowledge-bases/:id/knowledge/...)
  reuse the existing OwnedKBOrAdmin because the URL :id is the KB id
  directly. The chunks.DELETE("/by-id/:id/questions") route addresses
  chunks by their own id with no knowledge id in the URL, so it stays
  Contributor-gated as a documented carve-out.

No schema changes: documents inside a shared KB are KB-owned content,
not personal uploads — that's how the product is positioned and matches
the original "per-KB-ownership granularity" wording from the PR 2
breadcrumbs. Legacy/built-in KBs with empty CreatorID surface as
tenant-owned (Admin+ only), same rule as built-in agents.

Tests:
- 9 new test cases in handler/rbac_lookups_test.go covering the three
  new lookups: NotFoundMapsToSentinel (each repository sentinel),
  CrossTenantIsHiddenAsNotFound, OwnerMatchReturnsCreatorID,
  MissingTenantContext (fail-closed), and ChunkRouteShape (param-name
  regression guard).
- 5 new test cases in service/knowledge_owner_chain_test.go covering
  the chain helper: knowledge-not-found, kb-not-found, nil-kb-without-
  err, happy path, and legacy KB with empty CreatorID.

Build, vet, and the targeted ./internal/middleware/...
./internal/handler/... ./internal/router/...
./internal/application/service test runs are clean. Pre-existing
TestCreateStore_* / TestOssEnsureBucket_CreateFails failures on main
are unrelated and reproduce on a clean checkout.
2026-05-18 17:28:58 +08:00
wizardchen 2082b63951 refactor(rbac): centralise tenant-access guards in middleware
Tenant-access decisions previously lived in four places with subtly
different shapes: canAccessTenant in middleware/auth.go, an unexported
isCrossTenantSuperuser in middleware/rbac.go, authorizeTenantAccess +
two opening if-blocks in handler/tenant.go, and resolveTenantIDFromPath
in handler/tenant_member.go. Each variant had its own logging, its
own treatment of the cluster-wide EnableCrossTenantAccess flag, and
its own subtle difference in what "URL :id matches active tenant"
meant.

Collapse all four into middleware/access.go:

- IsCrossTenantSuperuser(ctx, cfg): exported, flag-aware. Both the
  per-user CanAccessAllTenants attribute AND
  cfg.Tenant.EnableCrossTenantAccess must be true; either alone
  rejects, so flipping the cluster flag off operationally revokes
  every superuser without a token re-issue.
- IsTenantAccessible(ctx, user, target, memberService, cfg): home
  tenant + superuser path + active tenant_members row. Replaces the
  old superuser-only canAccessTenant; the X-Tenant-ID gate now also
  honours ordinary multi-tenant memberships.
- RequireCrossTenantAccess(cfg): route guard for /tenants/all and
  /tenants/search, replacing the 12-line if-blocks at the top of
  ListAllTenants and SearchTenants.
- RequirePathTenantMatch(cfg): route guard for /tenants/:id/...,
  replacing authorizeTenantAccess (4 callers in tenant.go) and
  resolveTenantIDFromPath's superuser carve-out (5 endpoints in
  tenant_member.go). The router declares it once on the /tenants/:id
  group so the rule lives next to the route definitions.

router/rbac.go grows two factory methods (CrossTenant,
PathTenantMatch) so RegisterTenantRoutes stays declarative and reads
the same as the existing g.Viewer() / g.Owner() / g.OwnedKBOrAdmin()
guards. The /tenants/:id endpoints now group under one
PathTenantMatch() middleware instead of each handler re-deriving the
check.

Handler cleanups follow naturally: tenant.go drops
authorizeTenantAccess and the cross-tenant if-blocks; tenant_member.go
drops the cfg field and shrinks resolveTenantIDFromPath to a thin
parser. NewTenantMemberHandler loses its *config.Config argument;
container/container.go's dig wiring picks up the simpler signature
automatically.

middleware/access_test.go covers all four exports with truth-table
cases (flag/attribute/membership combinations, lookup errors, missing
context). tenant_member_test.go's existing handler-level cross-tenant
assertions still pass because memberTestRouter now mounts
RequirePathTenantMatch the same way the production router does.

Net -41 lines across 8 changed files + 2 new files; no behaviour
change other than the X-Tenant-ID gate accepting active memberships
(which was the original PR 4 goal that motivated this refactor).
2026-05-18 17:28:58 +08:00
wizardchen ae7cd02cd6 fix(rbac): polish tenant member UI; add /leave endpoint
Self-review of PR 3's TenantMembers.vue surfaced a handful of real bugs
and a few rough edges. This commit fixes them and adds the self-leave
endpoint that PR 3's design called out as a follow-up.

Frontend bug fixes
- Read err.status (not err.response.status). The axios interceptor in
  utils/request.ts already flattens errors into { status, message,
  ... }, so the original branch-by-status code never matched and every
  4xx fell through to the generic toast. Fixes 404/409 surfacing as
  "userNotFound" / "alreadyMember" / "lastOwner" instead of "Something
  went wrong".
- t-dialog @confirm replaces :on-confirm so the dialog stays open on
  validation failure and on API error. Users can fix and retry without
  retyping the email.
- Drop the t-select revert hack; with :model-value the rendered value
  is already source-of-truth-on-row.role, so no setTimeout dance is
  needed.
- Don't toast "no active tenant" on cold mount; watch the activeTenantId
  ref and load once the auth store hydrates.

Frontend UX additions
- Empty state via t-empty (also covers the search-no-results case).
- Compact role-permissions matrix above the table, mirroring
  OrganizationSettingsModal so Owners can see what each role grants
  before promoting/inviting.
- Search box filtering by username/email (client-side, sufficient for
  expected per-tenant member counts).
- "Leave Tenant" button — visible to any active member except the
  last-Owner case (server still enforces).

Backend
- New POST /tenants/:id/leave route. Calls TenantMemberService.RemoveMember
  with the caller's own user id so non-Owners can quit on their own.
  Gated Viewer+ in the matrix; the existing last-Owner invariant in the
  service still rejects last-Owner self-leave with 409.
- 3 new handler tests cover the happy path, the last-Owner 409 path,
  and the missing-caller-id 401 path (defensive — auth middleware
  always attaches a user, but the test pins the contract).

i18n
- Add tenantMember.permissions / search / leave / empty strings in all
  four locales (en/zh/ko/ru).
2026-05-18 17:28:58 +08:00
wizardchen 3f7150cb5c feat(rbac): tenant member management + invite_only gate (PR 3 of 3, #1303)
Wire the human-facing surface for the tenant-RBAC system that PR 1
introduced and PR 2 enforced: REST endpoints + UI for managing tenant
members, plus the long-pending invite_only registration gate.

Backend
- internal/handler/auth.go: Register now rejects with 403 when
  auth.registration_mode=invite_only. PR 1 already exposes the mode via
  /auth/config so the frontend hides the sign-up link; this is the
  server-side enforcement for direct API calls.
- internal/handler/tenant_member.go: new TenantMemberHandler with
  ListMembers / AddMember / UpdateMemberRole / RemoveMember. Maps
  service-layer sentinels onto HTTP semantics (ErrLastOwner -> 409,
  ErrMembershipAlreadyExists -> 409, ErrUserNotFound on AddMember -> 404,
  ErrInvalidTenantRole -> 400). AddMember requires the invitee to
  already have an account; SMTP-driven email invites are deferred.
- internal/types/tenant_member.go: new TenantMemberResponse projection
  carrying the fields the management UI renders (email, username,
  avatar, role, joined_at).
- internal/router/router.go: four routes under /tenants/:id/members
  gated per the matrix (Viewer+ for list, Owner+ for every mutation).
- internal/container/container.go: dig provider for the new handler.

Tests
- handler/tenant_member_test.go: 12 table-driven cases covering happy
  path, dangling-user tolerance, ErrLastOwner -> 409, ErrMembershipNotFound
  -> 404, ErrMembershipAlreadyExists -> 409, unknown email -> 404,
  invalid role rejected before user lookup, bad tenant id -> 400.
- handler/auth_register_invite_only_test.go: invite_only -> 403,
  self_serve -> 201, nil Auth config falls through.

Frontend
- src/api/tenant/members.ts: typed client (TenantRole/TenantMember/...)
  for the four endpoints.
- src/views/settings/TenantMembers.vue: members table with add dialog,
  inline role select, confirm-remove modal. All mutation widgets gated
  behind currentTenantRole === 'owner'; server still enforces.
- src/views/settings/Settings.vue: register the Members tab; show only
  when currentTenantRole is loaded (avoids tab flash on cold nav).
- i18n in en-US/zh-CN/ko-KR/ru-RU: new tenantMember namespace covering
  all UI strings and structured error messages (lastOwner / userNotFound
  / alreadyMember / invalidRole etc).

This completes the #1303 RBAC rollout. With cfg.Tenant.EnableRBAC=true
operators now have a full management surface; with EnableRBAC=false
the routes still exist but mutations log-and-pass per PR 2 contract.
2026-05-18 17:28:58 +08:00
wizardchen 8b71def60b fix(rbac): address PR 2 review feedback
Middleware
- RequireOwnershipOrRole now skips the creator lookup entirely when
  EnableRBAC is off, eliminating a hidden per-request DB roundtrip in
  the rollout window.
- New ErrResourceNotFound sentinel lets lookups signal "not visible to
  this tenant"; the middleware passes the request through so handlers
  can return a real 404 instead of a misleading 403.
- Transient lookup failures now surface as 503 (was 403) so monitoring
  can tell denied-by-policy from cannot-verify-right-now.
- Cross-tenant superusers (CanAccessAllTenants) bypass tenant role gates,
  restoring DELETE /tenants/:id and ResetAPIKey for org operators that
  resolve to Admin in foreign tenants.
- Construction-time warning fires once when cfg is nil/incomplete, so a
  wiring bug that disables enforcement globally is visible at boot.

Route coverage
- Add role guards to FAQ, KnowledgeTag, WikiPage, Skill, WebSearch,
  WebSearchProvider, WeKnoraCloud, Session, Chat, and Message route
  groups so the matrix actually matches the PR description.

Service layer
- KB and agent creators are no longer recorded as the synthetic
  system-<tenantID> user used by the X-API-Key path; resources stay
  tenant-owned. New types.IsSyntheticUserID helper centralises the
  prefix check.
- KBCreatorLookup and AgentCreatorLookup re-check tenant scope so a
  same-ID resource in another tenant cannot piggyback through the
  ownership shortcut.

Docs
- CustomAgent.RunnableByViewer comment now matches enforcement: it only
  blocks Viewer-driven agent-mode runs, not plain RAG/Wiki QA.

Tests
- New cases for the 503/404 paths, lookup-skip behaviour, cross-tenant
  superuser bypass, the lookup helpers (not-found, cross-tenant,
  built-in agent, missing tenant context), and IsSyntheticUserID.
2026-05-18 17:28:58 +08:00
wizardchen 9e3c92ca95 feat(rbac): enforce tenant role matrix on routes (PR 2 of 3, #1303)
Build on PR 1's tenant_members schema by enforcing the role matrix on
every authenticated route and at agent run time. Enforcement is gated
behind cfg.Tenant.EnableRBAC so the change ships dormant — the same
code paths log the would-be rejection during the rollout window and
start blocking once operators flip the flag.

Middleware (internal/middleware/rbac.go):
- RequireRole(min, cfg): role-only gate using TenantRole.HasPermission.
- RequireOwnershipOrRole(min, lookup, cfg): allows the original creator
  through even when their role is below min, otherwise falls back to
  the role check. Lookup errors fail closed; an empty creator_id is
  treated as tenant-owned and requires the role.
- Both honour cfg.Tenant.EnableRBAC for the rollout window.

Creator lookups (internal/handler/rbac_lookups.go):
- KBCreatorLookup resolves :id -> KnowledgeBase.CreatorID.
- AgentCreatorLookup resolves :id -> CustomAgent.CreatedBy; built-in
  agents (IsBuiltin=true) return ("", nil) so they remain Admin+ only.
- Compile-time guards keep the methods aligned with CreatorLookup.

Route guards (internal/router/rbac.go + router.go):
- rbacGuards bundles the matrix into Viewer/Contributor/Admin/Owner
  helpers plus OwnedKBOrAdmin / OwnedAgentOrAdmin. Each Register*
  function takes *rbacGuards so route lines remain readable.
- KB / Agent / Knowledge / Chunk / Tenant / Model / MCPService /
  VectorStore / DataSource / IMChannel routes annotated per the PR 1
  design doc matrix. Tenant-wide infra (models, vector stores, MCP,
  data sources, IM channels) reads as Viewer+ and writes as Admin+;
  per-resource mutations on KB / Agent route through the
  ownership-or-Admin guard.

Resource ownership (services):
- CreateKnowledgeBase / CopyKnowledgeBase populate CreatorID from the
  authenticated user, so the OwnedKBOrAdmin guard has a creator to
  match.
- CreateAgent / CopyAgent populate CreatedBy similarly. Built-in
  agents stay tenant-owned.

Agent run-time gate (internal/handler/session/qa.go):
- AgentQA refuses to launch the agent engine for Viewers when the
  target agent has RunnableByViewer=false. Same EnableRBAC gating as
  the middleware so it's a no-op until enforcement is turned on.

Tests:
- 12 unit tests cover RequireRole / RequireOwnershipOrRole, including
  the fail-open and fail-closed branches and the lookup-error path.
2026-05-18 17:28:58 +08:00
wizardchen 7f7a48880f feat(rbac): add tenant_members schema and auth wiring (PR 1 of 3, #1303)
Lay the foundation for tenant-level RBAC without enforcing it yet.
Schema, types, middleware lookup, and login-flow shape changes ship now;
RequireRole/route annotations and member-management endpoints follow in
PR 2 and PR 3.

Highlights:
- New tenant_members table (Postgres + SQLite) with (user_id, tenant_id)
  unique key and (owner|admin|contributor|viewer) role values. Migration
  000043 backfills one row per existing user, marking the earliest-created
  active user in each tenant as Owner and the rest as Contributor.
- KnowledgeBase.creator_id and CustomAgent.runnable_by_viewer added so
  PR 2 can wire ownership-based and Viewer-runtime checks.
- Auth middleware loads the active TenantMember row, sets a new
  TenantRoleContextKey, and falls back via three escape hatches:
  cross-tenant superusers (CanAccessAllTenants) get Admin in the target
  tenant; orphan tenants (zero members) auto-promote the first authenticating
  human to Owner; and when tenant.enable_rbac is false (the v1 default)
  unmatched lookups fail open with Admin to preserve current behaviour.
- Register / OIDC provisioning now insert an Owner membership for the
  registrant via TenantMemberService.EnsureOwner.
- LoginResponse renamed tenant -> active_tenant and gained a memberships
  array. POST /auth/switch-tenant issues a fresh token pair scoped to a
  target membership; GET /auth/config exposes auth.registration_mode for
  the frontend register-tab gating.
- New config keys: auth.registration_mode (self_serve | invite_only,
  defaults to self_serve) and tenant.enable_rbac (defaults to false).
  Both have WEKNORA_* env overrides.
- Frontend store persists memberships and exposes currentTenantRole;
  Login.vue calls /auth/config and hides the Register tab in invite_only
  mode. Go SDK types updated to match the new LoginResponse shape.
2026-05-18 17:28:58 +08:00
wizardchen 7643dd457e refactor(credentials): introduce /credentials subresource pattern
Replace the legacy "redacted placeholder + Clear* boolean" pattern with
dedicated per-resource credential subresources across MCP services,
Models, Web search providers, and Data sources.

Why
---
The previous design had three problems:

1. Main PUT body carried secret fields. The frontend echoed back a
   redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty
   defense in the service layer tried to detect "user did not change this"
   vs "user wants to clear this". A regression in that defense (or a new
   frontend forgetting it) silently overwrites the stored secret with the
   placeholder.

2. The "remove this credential" UX was a red checkbox under a pre-filled
   password input. Three intents (preserve / replace / clear) collapsed
   onto one field, and credential changes were bundled with unrelated
   config edits in the same submit. Users wiped working keys by mistake.

3. Secret presence was inferred from "did the response come back with a
   '***' placeholder", which couples the contract to a magic string.

Design
------
Each of the four resources now exposes:

  PUT    /{resource}/{id}/credentials          # write one or more fields
  DELETE /{resource}/{id}/credentials/{field}  # clear a single field

"Is this configured?" metadata travels on the main resource response as
a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET
endpoint. The frontend reads the configured boolean from the main GET
and never sees secret values at all.

Main PUT endpoints now ignore any secret fields in the request body and
log a deprecation warning if they appear, so legacy clients fail loudly
rather than overwriting silently.

Frontend
--------
- New reusable <CredentialResource> component renders a three-state card
  (unconfigured / configured / editing) and drives the dedicated
  endpoints. Used by MCP, Model, Web search; DataSource has a bespoke
  card with the same behaviour because its credentials are a single
  atomic per-connector map.
- Cancel from edit mode now restores state synchronously from the meta
  prop. The previous async refresh() was a no-op while state was still
  'editing', leaving the input frozen open.
- Remove is single-click + toast. The danger-themed button is the
  deterrent; a modal confirm adds friction without adding safety (the
  plaintext is irrecoverable client-side either way — recovery means
  re-typing).

DTOs (internal/handler/dto/) are deliberately separate from the GORM
models so "no secret in response" is a compile-time invariant: a future
contributor cannot leak a secret without explicitly adding the field to
the DTO, which is review-able in a single diff.

Storage is unchanged — credentials still live in the existing jsonb /
parameters columns. No schema migration.

Cleanup
-------
- types.MCPService / types.Model / types.WebSearchProviderEntity /
  types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret
  / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData().
- utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty.
  RedactedSecretPlaceholder constant is retained because VectorStore
  still uses the old pattern and is out of scope here.
- Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat
  redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 15:27:52 +08:00
wizardchen 5510ea8f5a feat(agent): human-in-the-loop approval for MCP tool calls (#1173)
Add an opt-in human approval gate so Agent runs pause before executing
MCP tools that operators flag as dangerous, surface an approval card in
the chat UI, and only resume after the user approves (optionally with
edited args) or rejects.

Backend
- New mcp_tool_approvals table + repo/service to mark per-tool approval
  required (PG migration 000042 + sqlite init).
- approval.Gate coordinates RequestAndWait / Resolve with sync.Once
  delivery, configurable timeout, and Redis Pub/Sub fan-out so multi-
  replica deployments work without sticky sessions.
- MCPTool.Execute integrates the gate; uses a round-level ApprovalCtx
  (without the per-tool 60s timeout) for the wait, and re-derives a
  fresh 60s exec ctx after approval so CallTool keeps a full window.
- New SSE response types (tool_approval_required / _resolved) and
  EventBus events plumb approval state to AgentStreamDisplay.
- REST: list/set per-tool approval flag, resolve pending approval.
- Configurable via agent.tool_approval_timeout_seconds (yaml) or
  WEKNORA_AGENT_TOOL_APPROVAL_TIMEOUT env (accepts seconds or Go
  duration).

Frontend
- MCP settings: per-tool "require approval" switch on the test panel.
- Chat: ToolApprovalCard renders the pause point with editable JSON
  args, validation feedback, mm:ss countdown that turns warning/danger
  near deadline, and a resolved state that retains context.
- i18n strings added for zh-CN / en-US / ko-KR / ru-RU.

Docs
- docs/zh/mcp-approval.md covering behavior, config, API, deployment
  considerations (Redis cross-instance, restart limitations).
2026-05-10 22:57:12 +08:00
wizardchen 04e8105728 perf(wiki-ingest): scale to 4w-doc KBs + generic task queue / dead-letter
The wiki ingest post-process pipeline OOM'd and ran for hours on KBs
with ~40k documents. The dominant tail was a per-batch ListAllPages
that pulled every page (multi-MB content blobs) into Go memory, plus a
24h-TTL Redis pending list whose data could be evicted before a long
serialized backlog drained. dedup did O(P × N) Jaccard scoring in Go
on every batch. Lint loaded the full graph. The index page kept the
entire wiki directory in its content column and rewrote the TOAST
chunk on every ingest. None of these survive at 4w docs.

This change reworks the write path end to end and pulls the durable-
queue + dead-letter primitives out of wiki and into shared
infrastructure that every asynq task type now benefits from.

  - migration 000041: task_pending_ops + task_dead_letters tables, plus
    three GIN indexes on wiki_pages (source_refs jsonb_path_ops,
    source_refs text fallback, lower(title) trgm).
  - TaskPendingOpsRepository + TaskDeadLetterRepository in
    internal/application/repository/task_queue.go: cursor-paginated
    list, atomic IncrFailCount via UPDATE…RETURNING, dedup-key scoped
    delete that refuses empty keys so a buggy caller can't wipe a KB's
    queue.
  - internal/middleware/asynqdl: writes a task_dead_letters row when an
    asynq task exhausts its retry budget. Payload-agnostic — a small
    probe struct extracts TenantID + scope hints across every existing
    payload type, so summary:generation / image:multimodal /
    faq_import / kb:clone / etc. all dead-letter without per-handler
    code. Best-effort: an insert failure never masks the underlying
    task error. Installed in router/task.go before the langfuse mw so
    it sees raw errors.

  - Pending queue moved Redis → PG (no TTL, restart-durable). Redis
    keeps just the active-batch lock and the delete tombstone.
  - In-batch retry budget (wikiMaxFailRetries=5) tracked via
    pendingRepo.IncrFailCount; over the cap the row is moved to
    task_dead_letters with task_type=wiki:ingest, related_id=knowledge
    id. Asynq retries (10) are handled by the new middleware.
  - Removed the per-batch ListAllPages. WikiBatchContext now carries
    lazy fetcher closures (SlugTitleMany, SummaryByKnowledgeID) with
    mutex-protected caches; reduce reaches for titles / summaries on
    demand instead of pre-loading the whole KB.
  - getExistingPageSlugsForKnowledge now uses ListSlugsBySourceRef,
    which the new GIN index on source_refs serves as a Bitmap Index
    Scan instead of a sequential text LIKE.
  - Dedup pre-filter uses pg_trgm via FindSimilarPages
    (idx_wiki_pages_title_trgm). For each new entity/concept (and each
    of its aliases) we ask the DB for the top-K trigram-similar
    existing pages and union the results — bounded prompt size, no Go-
    side O(P × M) loop. Small KBs (≤25 entities) bypass the pre-filter.
  - cleanDeadLinks / injectCrossLinks are scoped to the batch's
    affected slugs (a few dozen) instead of every page in the KB. Both
    use the new lite ListBySlugs / ExistsSlugs repo methods so they
    pull only slug + title + outlinks, not full content.
  - Slug fuzzy resolve (slug_fuzzy.go): when the LLM emits
    [[bad-slug|display]] the cleanup path now tries display-text
    reverse lookup → hyphen/case normalized equality → char-bigram
    Jaccard ≥ 0.8 before stripping. Recovers the common pinyin-word-
    break drift case ("shang-hai-tower" vs "shanghai-tower") in place
    instead of replacing the link with plain text.
  - rebuildIndexPage uses ListByTypeRecent(200) for the first-time
    intro and drops the full DocumentSummaries blob from the
    incremental update prompt, so its context stays bounded regardless
    of KB size.
  - Concurrency tunables surfaced in WikiConfig: IngestBatchSize /
    IngestMapParallel / IngestReduceParallel, with sensible defaults
    via OrDefault helpers. scheduleFollowUp drops to ProcessIn(0) so
    follow-ups don't waste asynq retry slots bouncing on the active
    lock.

  - RunLint walks pages via the new ListPagesCursor in 200-page
    windows and computes the live-slug set with a one-column
    ListAllSlugs Pluck instead of a Limit:0 GetGraph that materialized
    every node + edge. Memory is now bounded; 40k pages walks in
    constant ~4MB.

  - 14 GORM tests for both repos against an in-memory SQLite mirror of
    the production DDL (task_queue_test.go).
  - 6 tests for the dead-letter middleware covering retry budget
    detection, payload-agnostic scope inference, error truncation, and
    repo-failure isolation.
  - 7 tests for the slug fuzzy resolve helper covering the three
    resolution stages, display-text priority over normalized equality,
    bigram fallback acceptance, and rejection of unrelated slugs.
  - Existing wiki_ingest / wiki_lint / wiki_page tests updated to the
    new fetcher / cursor APIs.
2026-05-10 00:01:06 +08:00
wizardchen fb9ae11aa8 fix: 为前端静态资源设置正确的 Cache-Control 响应头
修复升级前端版本后,用户必须退出重新登录才能看到新版页面的问题。

根因:nginx 和 Lite 版 Go 静态服务均未设置 Cache-Control,浏览器按
启发式策略长期缓存 index.html,导致刷新仍加载旧版入口 HTML,旧版
引用的 hash 文件名也就不会变更。

改动:
- frontend/nginx.conf: index.html 及 SPA fallback 设 no-cache,
  must-revalidate;/assets/* (Vite 产物带 hash 文件名)设一年
  immutable 长缓存。location 内重复声明原有安全头,避免 nginx
  add_header 不继承导致安全头丢失。
- internal/router/router.go: Lite 版 serveFrontendStatic 按同样
  策略设置 Cache-Control。
2026-05-07 12:08:02 +08:00
Claude a867b1b29d feat(handler): add chunker preview endpoint for debugging
Adds POST /api/v1/chunker/preview — a read-only endpoint that runs the
adaptive chunker on supplied text and returns the chunks plus
diagnostic info (which tier won, which were rejected and why, the
DocProfile that the selector saw). Used by the upcoming KB editor
debug panel so users can experiment with chunking parameters without
re-uploading documents.

Backend pieces:
- chunker.SplitWithDiagnostics(text, cfg) returns chunks plus a
  Diagnostics struct (SelectedTier, TierChain, Rejected). Split() now
  delegates here.
- chunker.DocProfile gets JSON tags so it serializes cleanly.
- handler.PreviewChunking caps input at 256k chars and chunks at 500
  per response, computes a size-distribution summary, and never
  touches DB or embedding APIs.

Auth: lives under the v1 group so the standard JWT middleware applies.

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00
sqkstwj 0e19c978ae fix(files): fall back to global file service when tenant storage config is unavailable 2026-05-06 17:04:38 +08:00
wizardchen fe0d24ae87 fix(sessions): review fixes for keyword search / pinning / IM titles
- Apply the session-list additions to the SQLite init migration (Lite build)
  and make QueryPaged dialect-aware: LOWER(..) LIKE on SQLite, ILIKE on
  Postgres, drop NULLS LAST on SQLite. Escape LIKE wildcards in the keyword
  via the existing escapeLikeKeyword helper.
- SetPinned now returns rowsAffected so the handler can respond 404 on
  unknown / unauthorized session IDs instead of a misleading 200.
- GET /sessions always returns the enriched shape (pin state + IM origin
  fields) so the frontend never needs a second roundtrip; the dual-path
  legacy branch in the handler is gone.
- Register pin routes with the wildcard name that matches each verb's
  existing radix tree (POST :session_id, DELETE :id) and accept either
  param name in the handler; avoids gin's "wildcard conflicts" panic.
- Drop the redundant [platform] prefix from IM session titles now that
  the list renders a platform icon alongside the title; add unit tests
  for shortID / buildUserSessionTitle / buildThreadSessionTitle.
- Frontend: remove the submenu search input and its i18n/keyword wiring
  (search lives elsewhere in the app); pin icon uses the TDesign `pin`
  glyph and inherits color so the active session turns green; optimistic
  pin moves the item to the top of the list so it shows up at the top
  of the Pinned group; IM text badge replaced by the platform SVG icons
  from assets/img/im, desaturated by default and full-color on hover
  or when the session is active.
2026-04-30 16:23:00 +08:00
wizardchen dbd804d6e3 feat(sessions): add keyword search, user-scoped pinning, and IM source visibility
Sessions today are a flat list ordered by updated_at. Two gaps showed up in
practice:
- Users cannot find specific chats as the list grows beyond a screen.
- IM-created sessions (WeCom/Feishu/Slack/...) are indistinguishable:
  every title was "IM-<platform>" or "IM-<platform>-<username>" and the
  list API hid the underlying im_channel_sessions mapping, so admins had
  no way to tell which Feishu group a session came from.

Backend
- Migration 000039 adds sessions.user_id (owner), is_pinned, pinned_at
  plus a composite index for the list query. Existing rows keep user_id
  NULL and stay visible at the tenant level for backward compatibility.
- CreateSession now writes the caller's user_id from auth context.
- GET /sessions accepts keyword / source / agent_id. When any filter is
  set, the response switches to enriched items that LEFT JOIN
  im_channel_sessions and expose im_platform / im_chat_id / im_thread_id
  / im_user_id / im_agent_id / im_channel_id. No filters => legacy shape,
  existing clients unaffected.
- Ordering: is_pinned DESC, pinned_at DESC NULLS LAST, updated_at DESC.
- POST/DELETE /sessions/:id/pin for user-scoped pin/unpin.
- IM session titles: "[platform] <user|chat|thread>" with short ID
  suffixes so group/DM/thread sessions are visually distinct without
  needing a round-trip to fetch a display name from the IM adapter.

Frontend
- Search input debounced at 300ms drives the keyword filter.
- Pinned chats render in a dedicated group above the time-based groups,
  with a pin icon and a pin/unpin entry in the per-chat dropdown.
- IM chats get a short [platform] badge in the list.
- Pin toggle is optimistic and guards against double-clicks.
- zh/en/ko/ru i18n keys added for the new strings.
2026-04-30 16:23:00 +08:00
wizardchen e029d31f86 feat(im): tenant-wide IM channels overview under the user menu
Until now the IM channels a tenant has connected were only visible from
inside each agent's editor (AgentEditorModal → IMChannelPanel). Finding
out which bots are live across all agents meant clicking through every
one of them. Add a tenant-scoped overview so the set of connected IMs is
always one hop away from the avatar.

Backend

- im.Service.ListChannelsByTenant(tenantID) returns all non-deleted
  channels in the tenant, LEFT JOIN'd with custom_agents.name so
  built-in agents (which don't have rows in custom_agents) still show
  up with an empty agent_name; the frontend substitutes a localized
  "built-in agent" label.
- Credentials are intentionally stripped from this response — the list
  view is read-only and the editor route (GET /agents/:id/im-channels)
  remains the only source for secrets.
- New ChannelWithAgent DTO + IMHandler.ListAllIMChannels wired at
  GET /api/v1/im-channels under the existing RegisterIMChannelRoutes
  group, so auth middleware is inherited.

Frontend

- IMChannelsOverviewPanel.vue: floating submenu pane, stacked rows
  (agent on top, channel below) with identical 20px avatars and 12px
  labels so neither identity dominates. Each row is the whole click
  target (jumps to the agent editor); the switch uses @click.stop to
  toggle in place without triggering navigation.
- UserMenu.vue: new hover-driven submenu entry ("已接入的 IM" + link
  icon). The pane is teleported to <body> because the sidebar container
  has overflow:hidden that would otherwise clip right-flying content.
  Position is computed from the menu item's rect and clamps/flips
  against the viewport edges.
- "Live" indicator: a pulsing green dot on the menu item when at least
  one channel is enabled. UserMenu prefetches the list once on mount;
  the panel re-emits channels-changed after every load/toggle so the
  dot stays in sync. Respects prefers-reduced-motion.
- AgentList.vue: supplement the existing onMounted-only
  checkAndOpenEditModal with a watch(route.query.edit) so
  router.push({ path: '/platform/agents', query: { edit, section } })
  from an already-mounted AgentList (the common case when navigating
  from the overview) opens the editor immediately instead of only
  after a hard refresh.

Assets

- Seven platform SVGs under assets/img/im/, pulled from iconify
  (simple-icons / logos / tdesign / icon-park / remix icon) and baked
  with brand colors so monochrome sources (wecom/wechat/dingtalk/
  feishu) don't render black.

i18n

- New imOverview namespace in zh-CN / en-US / ko-KR / ru-RU covering
  the menu label, panel title/subtitle, column headers, builtin-agent
  fallback, and the live-indicator tooltip.

Tested

- go build ./...
- go test ./internal/im/...
- vue-tsc --noEmit
- npm run build
2026-04-30 15:29:14 +08:00
wizardchen 7fd566bc15 fix(im): rewrite private storage URLs to HTTP in IM channel replies
IM platforms (Feishu, Slack, Telegram, DingTalk, Mattermost, WeCom) cannot
render provider:// URLs (local://, minio://, s3://, etc.) that appear in
LLM answers containing knowledge base images. The web frontend handles
these via the authenticated /files endpoint, but IM clients need publicly
resolvable HTTP URLs.

Changes:
- Add HMAC-SHA256 presigned URL utility (internal/utils/presign.go) for
  generating time-limited, signature-verified file access URLs
- Add GET /api/v1/files/presigned endpoint that serves files without
  session auth, verified by HMAC signature and expiry
- Update localFileService.GetFileURL() to return presigned HTTP URLs
  when APP_EXTERNAL_URL is configured (cloud backends already return
  presigned URLs via their SDKs)
- Add IM content rewriting pipeline: strip <image> XML tags, strip
  citation tags, rewrite storage URLs to HTTP — applied at all IM
  output points (streaming flush, non-streaming reply, fallback)
- Add holdback buffer in streaming flush to prevent URL/tag splitting
  across chunk boundaries

Closes #1058
2026-04-30 11:40:30 +08:00
wizardchen 7fcd92994b feat(tenant): expose API Key reset from the API Info page
Add a danger-themed reset button next to the existing copy/eye buttons
on the API Info settings page. Confirmation dialog warns that the old
key is revoked immediately; on success the new plaintext key replaces
the displayed value and is auto-revealed so the user can copy it.

Backend: new POST /api/v1/tenants/:id/api-key handler that wraps the
existing TenantService.UpdateAPIKey; access is gated by the same
authorizeTenantAccess check as other tenant endpoints. The handler
returns the freshly generated plaintext key, while the encrypted form
is persisted to the database.

Frontend: new resetTenantApiKey API client, reset button + confirm
dialog wiring in ApiInfo.vue, plus matching i18n entries for zh-CN,
en-US, ko-KR, and ru-RU.
2026-04-29 20:07:38 +08:00