Commit Graph

974 Commits

Author SHA1 Message Date
MidnightSun 6210f44fb6 fix(docparser): preserve MinerU markdown and persist relative images
MinerU already returns markdown with embedded HTML blocks, but the current\nreader runs the whole document back through html-to-markdown. That\nsecond conversion escapes valid headings and image syntax, so chunk\nprofiling sees plain text instead of markdown structure and relative\nimage references stop matching the storage pipeline.\n\nKeep MinerU output in its original markdown form and only apply narrow\ncompatibility normalization for the specific over-escaped patterns we\nactually need to recover. The converter now matches image refs by the\npaths that are really present in markdown or embedded HTML instead of\nassuming a single images/<name> form.\n\nExtend ImageResolver so relative HTML <img src=...> references share the\nsame storage rewrite path as markdown images, deduplicate repeated saves,\nand keep the frontend sanitizer compatible with MinerU's details/summary\nblocks. Add focused docparser tests that cover escaped markdown repair,\nvariant image path matching, and relative HTML image persistence.
2026-05-21 11:39:57 +08:00
wolfkill 5bdaf58d45 fix(session): scope wiki fixer to shared KB tenant 2026-05-21 11:19:19 +08:00
wolfkill bb95488ac8 fix(knowledge): complete indexed documents immediately 2026-05-20 22:40:48 +08:00
ochan.kwon fd3d9f547c feat(search): fan-out KB retrieval across bound vector stores
Multi-KB hybrid search now groups KBs by their bound VectorStore (partition
key (storeID, owner_tenant_id)), retrieves in parallel via errgroup with a
SetLimit(4) cap and a per-group timeout (MULTI_STORE_RETRIEVE_TIMEOUT_SEC,
default 30s), and merges results. When the collected results span more than
one engine type, an EngineAwareNormalizer rescales vector scores to [0, 1];
keyword (BM25) scores pass through to the existing RRF fusion. Single-group
calls take the fast path with zero fan-out overhead, preserving today's
behavior for deployments where every KB has vector_store_id = NULL.

Embedding-model consistency is now enforced explicitly via
ResolveEmbeddingModelKeys. Multi-KB searches across KBs whose resolved
model identities differ return BadRequest instead of silently producing
incomparable scores. Cross-tenant Organization-shared KBs are preserved by
partitioning on KB.TenantID so the factory's ownership lookup runs against
the source tenant. Foreign-tenant KB UUIDs injected via the request body
are rejected via kbShareService.HasTenantKBPermission (Plan 3 of #1303,
3-D capped) before any retrieval; rejected scopes surface as 404 to avoid
leaking foreign KB existence.

Service-layer typed AppErrors (ErrVectorStoreBindingInvalid 2200 /
ErrVectorStoreUnavailable 2201) are mapped from PR2 sentinel hierarchy and
preserved end-to-end: the iterative FAQ path returns them rather than
swallowing, and the HybridSearch handler routes typed AppErrors to the
client unchanged instead of downgrading to 500.

Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 4 (Multi-store fan-out search).
Depends on #994, #1310, #1372.
2026-05-20 22:25:39 +08:00
wizardchen 07afec1499 fix(chat): stabilize history pagination and keep message order 2026-05-20 21:03:39 +08:00
wizardchen f3c7281f47 refactor(agent): simplify grep_chunks tool to a single regex query
The grep_chunks tool previously accepted an array of regex queries (1-5)
and an optional knowledge_base_ids filter and limit. In practice the LLM
either fired multiple near-duplicate calls or split synonyms across
entries instead of using POSIX alternation, and KB scoping plus result
limit are server-side concerns the model should not control.

Reshape the contract to match `grep -E -i` semantics:

- Schema accepts a single required `query` string. Combine concepts with
  `|` alternation in one regex instead of multiple calls.
- Drop `knowledge_base_ids` and `limit` from the schema; the tool now
  always searches the full agent scope and uses a fixed internal cap.
- Legacy `pattern`, `queries`, `patterns`, `max_results` keys are still
  accepted and joined into a single alternation regex so older callers
  and in-flight model outputs keep working.
- Update the agent system prompt template to document the new single
  `query` field.
- Frontend tool title now reads `query`/`queries`/`pattern`/`patterns`
  in that order so the search text is shown again under the new schema.
- Add a dedicated `grepSearch` / `grepSearchFailed` tool status (zh-CN,
  en-US, ko-KR, ru-RU) and rename the zh-CN tool label to "搜索关键词"
  so the UI no longer prefixes the call with a generic "调用 ..." label.
2026-05-20 19:34:27 +08:00
knight 513e589494 feat: add vLLM server URL configuration for MinerU
Add support for configuring vLLM server URL when using vlm-http-client or hybrid-http-client backend in MinerU.

Changes:
- internal/types/tenant.go: Add MinerUVLMServerURL field to ParserEngineConfig
- internal/infrastructure/docparser/mineru_converter.go: Pass server_url to MinerU API when backend is vlm-http-client or hybrid-http-client
- frontend/src/api/system/index.ts: Add TypeScript type definition
- frontend/src/views/settings/ParserEngineSettings.vue: Add vLLM server URL input field
- frontend/src/i18n/locales/*.ts: Add translations (zh-CN, en-US, ko-KR)
2026-05-20 16:33:12 +08:00
Miles Lai e62c0563aa doris: add configurable compatibility modes and guard mode switches
Problem:
The hard-coded Doris vector function implementation (cosine_distance_approximate with
UNIQUE KEY ANN tables) fails on SelectDB 4.0.2-rc01 and other Doris builds lacking that
specific function support. Users had no way to adapt without code changes.

Root cause:
There is assumption all Doris deployments support the same vector function API, but different
builds (Doris OSS, SelectDB, Doris Cloud) ship with different function variants and table
key constraints. No capability detection or user configuration existed.

Solution:
Implement DORIS_COMPAT_MODE environment variable with three modes:

  * auto (default/recommended): probe Doris server on first use to detect available vector
    functions; prefer inner_product_duplicate (modern Doris 4.0+), fall back to legacy
    (older builds lacking inner_product_approximate)

  * legacy: hard-set to cosine_distance_approximate + UNIQUE KEY (for older Doris/SelectDB
    builds without inner_product_approximate support)

  * inner_product_duplicate: hard-set to inner_product_approximate + DUPLICATE KEY
    (for modern Doris 4.0+ and current SelectDB with normalized embeddings)

Implementation details:
- add compat.go with one-time mode resolution (sync.Once) and capability probing
- inspect existing weknora_embeddings_* table DDL via SHOW CREATE TABLE to detect and
  enforce schema compatibility; prevents silent mismatches
- fail fast with clear error message when configured mode does not match existing tables,
  with explicit remediation steps (recreate tables or change env var)
- branch all query paths (inner_product_approximate vs cosine_distance_approximate),
  DDL generation (DUPLICATE KEY vs UNIQUE KEY), write paths (embed normalization),
  and chunk updates (Stream Load vs read-modify-write) by resolved compat mode
- add comprehensive repository tests for mode selection, auto-detection, and mismatch
  scenarios; all tests pass
- expose DORIS_COMPAT_MODE in docker-compose.yml with auto as default
- document in .env.example with clear mode decision guidance
- log all mode decisions (requested, detected, probed, final) at INFO/WARN level

Key guarantee:
⚠️ DORIS_COMPAT_MODE is NOT interchangeable after embedding tables are created.
App will reject mode switches that conflict with existing table layout, preventing
silent data mismatches and query failures.
2026-05-19 17:15:59 +08:00
wolfkill eb52caf033 fix(chunker): keep top-level heading chunks separate 2026-05-19 17:08:56 +08:00
wizardchen 7bb38f81b6 feat(session): persist last request state for UI restoration
Sessions now record the input-bar state used for the most recent QA
request (agent, model, KB scope, web search). The chat UI hydrates
those settings on session reopen so users see the same configuration
they used last time, instead of the global default.

The state is stored in the existing sessions.agent_config JSONB column
to avoid a new migration. Frontend snapshots the user's global defaults
on session enter and restores them on session leave, so opening an old
session does not pollute new-chat defaults.
2026-05-18 22:28:00 +08:00
yuheng.huang 28e595734f fix(moonshot): pin temperature=1 for models that reject other values moonshot-v1-* and kimi-k2.5/k2.6 reject any temperature ≠ 1 with HTTP 400. Detect these models in BuildChatCompletionRequest and force Temperature=1 while leaving kimi-k2/k2-turbo/k2-thinking unaffected. 2026-05-18 21:28:40 +08:00
yuheng.huang 00d39e006d fix(docparser): unescape MinerU markdown image syntax html-to-markdown over-escapes to on MinerU'smixed-Markdown+HTML output, breaking downstream image extraction. Restore the canonical image syntax after conversion so images are persisted. 2026-05-18 21:27:53 +08:00
wizardchen 7444c2190d chore(rbac): update default behavior for tenant RBAC configuration
Refactor the tenant RBAC configuration to change the default value from false to true, enabling role enforcement by default. This change allows operators to opt into a logging-only rollout window by explicitly setting the configuration to false.

Updates include:
- Modifications to .env.example and docker-compose.yml to reflect the new default.
- Adjustments in rbac.md documentation to clarify the new default behavior and the opt-in process.
- Code changes across various files to utilize the new pointer-based configuration for EnableRBAC, ensuring nil safety and clearer intent.

No functional changes were introduced; the adjustments primarily enhance clarity and maintainability of the RBAC feature.
2026-05-18 21:24:28 +08:00
wizardchen 49f3ad185a fix(auth): make DISABLE_REGISTRATION drive registration_mode too
DISABLE_REGISTRATION=true used to block /auth/register at the handler
layer only, leaving /auth/config still reporting self_serve. The
frontend therefore kept showing the Register entry even when the env
var was set — clicking it just hit the 403. Two gates, out of sync.

Wire DISABLE_REGISTRATION=true through applyAuthAndTenantDefaults so
it coerces auth.registration_mode to invite_only (env wins over YAML,
matching docs/rbac.md). The handler-side os.Getenv check is now
redundant with IsInviteOnly() and is removed, leaving a single
enforcement path.

Add config tests pinning down the env/YAML matrix, including the
explicit-self_serve override case that would otherwise be the easy
regression to ship.
2026-05-18 21:15:01 +08:00
wizardchen 2ab54f1542 chore(auth): drop redundant WEKNORA_AUTH_REGISTRATION_MODE env override
DISABLE_REGISTRATION=true (handler layer) and
WEKNORA_AUTH_REGISTRATION_MODE=invite_only (config layer) were two
env-level ways of saying the same thing: block /auth/register.
Keeping both invites confusion about which wins and risks operators
setting one while expecting the other.

Remove the WEKNORA_AUTH_REGISTRATION_MODE env override so the env
layer exposes a single knob (DISABLE_REGISTRATION). The
auth.registration_mode config field stays — operators who want the
richer behaviour (frontend hides the registration entry via
/auth/config in addition to the server-side 403) flip it in
config.yaml.

No behaviour change for deployments that did not set the env var.
Update docs/rbac.md to document the deliberate split.
2026-05-18 21:15:01 +08:00
wizardchen 0cdb922d84 feat(tenant): remember user's last-active workspace across logins
Until now a fresh login (new device, expired refresh token, cleared
browser) always dropped the user into their home tenant, even if they
spend most of their time in a peer workspace. The session-local
X-Tenant-ID override in localStorage gave a "same browser sticky"
effect, but never crossed devices or new sessions.

This adds a small server-side preference, `users.preferences.
last_active_tenant_id`, persisted in the existing jsonb column (no
new migration), and threads it through:

* Backend
  * `UserPreferences` gains `LastActiveTenantID *uint64` with sentinel
    semantics (`*0` from the PATCH endpoint = clear preference).
  * `resolveLoginTenantID` validates the stored id (tenant still
    exists + active membership, or CanAccessAllTenants) and falls
    back to home on any failure, best-effort clearing the stale
    preference so subsequent logins don't pay for it again.
  * `Login` and `LoginWithOIDC` resolve once and use the result for
    both the JWT `tenant_id` claim and the returned `active_tenant`,
    keeping the two in sync. `RefreshToken` rides through
    `GenerateTokens` so refresh rotations also land the user back in
    their preferred tenant instead of bouncing to home.
  * `UpdateUserPreferences` learns to merge the new key.
  * `PUT /auth/me/preferences` accepts the new field.

* Frontend
  * `Login.vue` now writes the user's HOME tenant id into
    `user.tenant_id` (matching the field's documented semantics) and
    expresses any active-vs-home divergence via `setSelectedTenant`,
    so `useHomeTenant` and the "current"/"home" badges stay correct
    after the backend honours a remembered preference. `App.vue`'s
    OIDC sync does the same reconciliation.
  * `TenantSelector`, `UserMenu` and the post-tenant-create handlers
    fire `persistLastActiveTenantPreference` after every successful
    user-initiated switch (switching to home sends `0` to clear).
    The call is raced against the existing reload-grace window so
    most writes finish before the page tears down; lost writes are
    recoverable on the next switch.

No new UI. Users will simply notice that, after re-logging in on
another device, they land back in the workspace they were last
using rather than always in their home tenant.

Note: `make docs` is unrelated-broken on `main` (audit_log.go
references `errors.AppError` which swag can't resolve), so the
Swagger artifacts under docs/ are intentionally not regenerated
in this PR. The handler code is the source of truth.
2026-05-18 20:37:57 +08:00
Li Xianggang c19d3543c8 feat(url): 支持docreader不上传替换白名单url的图片 2026-05-18 19:39:30 +08:00
Li Xianggang b3dbf86bf5 feat(obs): 支持华为云obs存储 2026-05-18 19:38:23 +08:00
wizardchen bbcedbee63 feat(tests): enhance vector store tests with fake Elasticsearch server and add user pinning methods to knowledge base repository
Introduced a new helper function to create a fake Elasticsearch server for testing, improving the reliability of connection probes in vector store tests. Additionally, added methods for setting and listing user-specific knowledge base pins in the knowledge base repository, aligning with recent user pinning features.

Refs: #1303
2026-05-18 17:57:46 +08:00
wizardchen fc2d858222 fix(session): update AgentQA to conditionally disable title based on request
Modified the AgentQA function to conditionally pass the title disable flag to the executeQA method, enhancing flexibility in session handling. This change improves the behavior of the QA execution based on user requests.
2026-05-18 17:57:46 +08:00
wizardchen 1506c604e1 refactor(knowledgebase): remove TogglePinKnowledgeBase method from KnowledgeBaseRepository interface 2026-05-18 17:28:58 +08:00
wizardchen 4ca5469d1e refactor(tenant-switch): unify tenant switch navigation to always redirect to KB list
Updated the tenant switch logic to consistently redirect users to the platform's knowledge base list after a tenant switch, simplifying the navigation experience. Removed the previous handling of tenant-scoped routes, ensuring a more predictable behavior during tenant transitions. This change enhances user experience by avoiding potential empty states on reload.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 7528175a74 feat(knowledge-base): implement per-user pinning for knowledge bases
Refactored the pinning mechanism for knowledge bases to be user-specific, allowing any user with read access to pin knowledge bases independently. This change includes the introduction of a new `user_kb_pins` table to manage pin states, replacing the previous tenant-wide pinning system. Updated relevant API endpoints and service logic to support this new functionality, ensuring that the UI reflects the user's personal pin status.

Additionally, adjusted the knowledge base listing to sort by user-specific pin status, enhancing the user experience by prioritizing pinned items.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 5a91487322 feat(memory-management): implement server-side user preference for memory feature
Updated the memory management logic to utilize a server-side per-user preference for enabling memory. The `enable_memory` field is now conditionally set based on user preferences, allowing for better control in both normal and embedded contexts. Adjusted relevant API handlers and request structures to support this change, ensuring backward compatibility with existing clients.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 68ac7fb973 feat(user-preferences): implement user preferences management with backend support
Added functionality for managing user preferences, including the ability to update preferences via a new API endpoint. The preferences are now stored server-side, allowing for synchronization across devices. Updated relevant components to handle user preferences, including the UserMenu and settings views. Enhanced internationalization support for error messages related to preference updates.
2026-05-18 17:28:58 +08:00
wizardchen 8d4c58c5fa feat(creator_name): add creator_name field to agents and knowledge bases for enhanced visibility
Introduced a new optional field, creator_name, in both CustomAgent and KnowledgeBase types to allow the front end to display the creator's name. This enhancement enables better differentiation between resources created by the current user and those created by other members of the same tenant. Updated relevant API handlers to populate this field during list operations, ensuring accurate representation in the UI. Additionally, modified the ResourceOriginBadge component to utilize the creator_name for improved context in resource listings.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 8b681e4a2e refactor(GlobalCommandPalette): streamline template structure for improved readability
Consolidated the template structure in GlobalCommandPalette.vue to enhance code clarity and maintainability. Adjusted the layout of various elements for better alignment without altering existing functionality. This refactor aims to improve the overall readability of the component for future development.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 14f62a6e14 fix(audit_log): prevent deadlock in Stop() when Start() is never called
Updated the AuditLogRetentionRunner to include a 'started' flag, allowing Stop() to return immediately if Start() was never invoked. This change prevents potential deadlocks during graceful shutdown. Additionally, added tests to ensure that Stop() behaves correctly when called before Start() and when the service is nil.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 884ac44283 feat(invitations): implement pending invitations feature with polling and dialog
Added functionality for managing pending invitations, including a polling mechanism to update the invitation count in real-time. Introduced a new dialog component for users to view and respond to their invitations without navigating away from their current context. Updated API endpoints for fetching and managing invitations, and enhanced the user interface with relevant internationalization support.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 3d30a9d842 feat(organization): implement tenant-based access control for organization visibility
Enhanced the GetOrganization and ListMembers methods to enforce tenant-based access control. Users can now only access organization details if their tenant is a member or if the organization is marked as searchable. This change prevents unauthorized enumeration of organizations and ensures sensitive member information is only accessible to authorized tenants.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 4b4225a79f feat(custom_agent): enhance custom agent service with KB sharing support
Updated the customAgentService to include a new KBShareService interface for handling cross-tenant knowledge base sharing. Implemented logic to group knowledge base IDs by effective tenant, ensuring correct querying of shared KBs. Adjusted the GetSuggestedQuestions method to accommodate these changes, improving the handling of FAQ and document chunks for shared knowledge bases.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 8332a0dab6 feat(favorites): implement user favorites and recents functionality
Introduced a new API client for managing user favorites, allowing users to list, add, and remove starred resources. Enhanced the sidebar to display favorites and recents, improving user navigation. Updated internationalization files to include relevant labels and tooltips for the new features across multiple languages.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 7d030a6f6a feat(tenant): implement tenant creation limit and error handling
Added a new configuration option to limit the number of tenants a non-superuser can create via self-service. Introduced a new error type for handling cases where users exceed this limit, returning a 429 status code. Updated the tenant creation and update handlers to enforce this limit and provide appropriate feedback to users. Additionally, refactored the tenant update request structure to ensure only mutable fields are allowed.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen 29dc99ae79 feat(tenant): implement self-service tenant creation functionality
Added a new API endpoint for creating tenants, allowing any logged-in user to create their own workspace. The backend automatically assigns the creator as the Owner of the new tenant. A new CreateTenantDialog component was introduced in the frontend, enabling users to input the workspace name and an optional description. Internationalization files were updated to include relevant messages for this feature across multiple languages.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen dc8d033b4d feat(organization): transition membership model to tenant-level and enhance invite functionality
Updated the organization membership model to operate at the tenant level, aligning with Plan 3. This change introduces new interfaces and modifies existing ones to accommodate tenant-centric operations, including the `InviteMemberRequest` and `TenantInviteCandidate`. The search functionality for inviting members has been updated to reflect this shift, allowing for tenant-based searches instead of user-based. Additionally, internationalization files have been updated to ensure consistent messaging across languages regarding tenant membership and roles.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen ff1f03ce0d feat(rbac): daily retention sweep for audit_logs (default 90 days)
The audit_logs table grows monotonically until something purges it.
PR 6 explicitly listed retention as out of scope for v1; this picks
that up.

Adds an `audit.retention_days` config (default 90, 0 disables) and a
small background goroutine `AuditLogRetentionRunner` that fires once
~10 minutes after boot and then every 24h, calling
`AuditLogService.Purge` which DELETEs rows older than the cutoff in a
single statement. The dependency surface is intentionally minimal —
no robfig/cron, no asynq, no migration — because retention has no
wall-clock alignment requirement.

Wiring is the same shape as the existing data source scheduler:
container `Provide(NewAuditLogRetentionRunner)`, an `Invoke` that
calls `Start`, and a `ResourceCleaner.RegisterWithName` for graceful
shutdown.

Defaults preserve operator intent:
  - `audit:` section omitted from YAML  -> retention_days=90.
  - explicit `audit.retention_days: 0` in YAML  -> purge disabled.
  - `WEKNORA_AUDIT_RETENTION_DAYS=N` env override (incl. N=0).
  - `retention_days < 0` is rejected by ValidateConfig.

Tests cover the service Purge contract (no-op when disabled, cutoff
math, error propagation) and the runner lifecycle (Start no-op when
disabled, idempotent Start/Stop, ticker cadence, runOnce swallows
errors). 10 new tests, all green.

docs/rbac.md updated with the new YAML / env / behaviour.

Refs: #1303
2026-05-18 17:28:58 +08:00
wizardchen cf6c28fe7e fix(rbac-ui): identify org owner row by tenant_id, not user_id
Plan 3 (#1303) lifted Org membership from per-user to per-tenant, and
40fdb978 pinned the org's owner tenant in DB as organizations.owner_tenant_id.
The OrganizationSettingsModal members list, however, still decided
"which row is the owner" by comparing member.user_id against
orgInfo.owner_id. After Plan 3 the members list is keyed on tenant_id,
so the user-id check is structurally wrong: the representative user of
the owner tenant happens to be the owner today, but if that user is
moved to another tenant — or a different user of the owner tenant
becomes the representative — the modal would either lose the "(owner)"
tag, or re-tag a non-owner row, and would offer the role-select /
remove-button affordances on what is in fact the owner tenant row.

This change:

  - Adds OwnerTenantID to OrganizationResponse and populates it in
    toOrgResponse from the persisted column. IsOwner now compares the
    caller's current tenant against owner_tenant_id (with a user-id
    fallback for legacy rows where owner_tenant_id == 0).

  - Adds owner_tenant_id to the frontend Organization type and a small
    isOwnerMember(member) helper in OrganizationSettingsModal, which
    tenant-matches by default and falls back to user-id only on
    pre-000046 rows. The role <select>, role tag suffix, and the
    delete button on member rows now all use the helper.

The "is-me" CSS class and the me-tag in the member name still match
on user_id — that one really is a per-user marker ("this row's user is
me, the signed-in user").
2026-05-18 17:28:58 +08:00
wizardchen 7e20b9a57e fix(rbac): gate remaining organization mutating routes with Admin+
Plan 3 (#1303) fixed several Org mutating routes (POST /organizations,
/join, /join-request, /join-by-id, /:id/leave, /:id/request-upgrade)
to Admin+ in the caller's tenant, and e1deb810 followed up by gating
DELETE /:id/members/:tenant_id symmetric with /:id/leave.

The remaining Org-admin surfaces still relied solely on a
service-layer "is the caller's tenant admin in this org?" check, with
no route-layer floor. Add g.Admin() to:

  - PUT    /organizations/:id                          (update org)
  - DELETE /organizations/:id                          (delete org)
  - POST   /organizations/:id/invite-code              (rotate code)
  - GET    /organizations/:id/search-users             (member picker)
  - POST   /organizations/:id/invite                   (direct invite)
  - PUT    /organizations/:id/members/:tenant_id       (change role)
  - GET    /organizations/:id/join-requests            (review queue)
  - PUT    /organizations/:id/join-requests/:request_id/review

Each of these takes an action that is visible to the whole org or to
another tenant; per the RBAC matrix in router/rbac.go they belong on
Admin+. The service-layer check stays as the authority on org-level
membership; the route guard prevents a tenant Viewer/Contributor from
ever reaching the service in the first place.

Behaviour is unchanged when cfg.Tenant.EnableRBAC is false: RequireRole
continues to log the would-be rejection and let the request through.
2026-05-18 17:28:58 +08:00
wizardchen 8efe1b4187 refactor(rbac): address kb-access guard review findings
Follow-up to the route-level KB-access guard introduced earlier in
this PR. Five reviewer findings, all fixed in middleware + tests
without changing handler call sites:

  - Honour `?agent_id` in the agent-share fallback. The in-handler
    resolution in knowledgebase.go branches on a specific agent's
    KBSelectionMode (all / selected / none); the guard was only ever
    calling TenantCanAccessKBViaSomeSharedAgent, which let some
    requests pass at the guard layer that the handler would then
    reject — and rewrote the request's tenant context to the source
    tenant before the rejection happened. The guard now mirrors the
    handler: a specific agent_id pins the resolution to that agent
    (no fallback to any-agent), an empty agent_id keeps the existing
    any-agent behaviour. The knowledge.go / knowledgebase.go handler
    helpers stay in place as defence in depth until they're folded
    into the guard in a follow-up.

  - Distinguish not-found vs. transient on the knowledge / chunk
    resolvers. A DB hiccup used to surface as a 404, both confusing
    clients and hiding the underlying failure from monitoring. The
    resolvers now return apperror 404 only for the known not-found
    sentinels (apprepo.ErrKnowledgeBaseNotFound, ErrResourceNotFound)
    and let other errors propagate; the guard maps those to a fresh
    NewServiceUnavailableError (503).

  - Stop 500-ing on legacy chunks with empty knowledge_base_id. The
    chunk is effectively unresolvable to a KB, so the client gets the
    same 404 they'd get for a missing chunk instead of a 500 that
    pollutes alerting. The warn log preserves the operator signal.

  - Honour cfg.Tenant.EnableRBAC. The guard now mirrors RequireRole /
    RequireOwnershipOrRole: with enforcement off, 401/403 paths log
    the would-be rejection and pass through. 404 still fires either
    way (a missing resource is not an authorisation event). Keeps the
    rollout window safe — operators can flip enforcement on globally
    without code changes elsewhere.

  - Document the c.Keys vs. c.Request.Context() split. The guard
    intentionally rewrites only the request context (handlers that
    still run their own share resolution, currently knowledge.go /
    knowledgebase.go, would otherwise see kb.TenantID == c.Keys
    tenant and misclassify shared-Viewer access as Admin). The
    package-level comment now spells this out so the migration of the
    remaining handlers in a follow-up doesn't accidentally regress
    `my_permission` rendering.

Tests:

  - Existing four cases keep working; runGuard signature gained a
    guardOpts knob.
  - Eight new cases cover the agent-share resolution paths (any-agent
    viewer, Editor required → must reject, specific agent in modes
    all / selected-match / selected-miss / none, cross-tenant
    mismatch) plus the EnableRBAC=false fail-open + 404-still-fires
    pair.
2026-05-18 17:28:58 +08:00
wizardchen ef01f99e69 refactor(rbac): extend KB-access guard to chunk / knowledge / knowledgebase routes
Follow-up to the FAQ + Tag refactor in this same PR. Three additional
handler areas now run through the route-level g.KBAccess* guard:

  - chunk.go  : full refactor — helper effectiveCtxForKnowledge and
                validateAndGetChunk's access check are gone; the
                handler's chunk-fetch + ownership-mismatch check stay
                as defence-in-depth (a same-tenant attacker can't
                pass mismatched knowledge_id + chunk_id).
  - knowledge.go and knowledgebase.go : routes now run the guard
                BEFORE the handler. The in-handler
                validateKnowledgeBaseAccess /
                resolveKnowledgeAndValidateKBAccess helpers stay for
                now because they branch on the agent_id query param
                (specific-agent check vs. any-agent fallback) — that
                handler-specific branch can collapse into the guard
                in a follow-up. The duplicated DB roundtrip is
                negligible compared to the handler's main work.

New middleware exports:

  - KBIDFromChunkIDParam   : walks chunk_id → KnowledgeBaseID via the
                             chunk's denormalised column.
  - g.KBAccessReadFromChunkIDParam / g.KBAccessWriteFromChunkIDParam
                           : convenience methods on rbacGuards.

Route changes are purely additive (existing role / ownership guards
stay in place; the new KB-access guard runs after them and rewrites
the request's tenant context to the source tenant for shared KBs).
No new dependencies on the kbShareService / agentShareService at the
handler level for chunk.go — those move to the route layer.

Tests pass; behaviour unchanged.
2026-05-18 17:28:58 +08:00
wizardchen 1109cb7bff refactor(rbac): KB-access guard at the route layer (no helper in handlers)
Five handler files used to carry near-identical 30-line helpers
(effectiveCtxForKB / validateAndGetKnowledgeBase) that did the same
own-or-shared resolution before every KB-scoped operation:

  1. KB belongs to caller's tenant     -> grant own access
  2. Org-shared KB                      -> grant min(share, role) cap
  3. Shared agent carries the KB        -> grant Viewer (read-only)

A bug found in any one of them had to be fixed in five places, and
several already drifted apart (different log levels, slightly
different error messages, one path missing the share-cap check).

Move the resolution to a route-level gin.HandlerFunc:
middleware.RequireKBAccess, exposed on rbacGuards as
g.KBAccessRead("id") / g.KBAccessWrite("id"). The guard runs after
the role/ownership checks, then on success:

  - stashes the resolved (KB + effective tenant id + permission)
    on c.Keys under middleware.KBAccessContextKey, and
  - rewrites c.Request.Context() to carry the effective tenant id

so handlers downstream just read tenant the way they always did
(types.MustTenantIDFromContext) without a per-handler helper.

Refactor FAQ and Tag handlers to drop their helper methods; their
service constructors lose the kbShareService / agentShareService
dependencies (now lifted to the route layer). Routes:

  faq.GET("/entries",            g.Viewer(),         g.KBAccessRead("id"),  ...)
  faq.POST("/entries",           g.OwnedKBOrAdmin(), g.KBAccessWrite("id"), ...)
  kbTags.GET("",                 g.Viewer(),         g.KBAccessRead("id"),  ...)
  kbTags.POST("",                g.OwnedKBOrAdmin(), g.KBAccessWrite("id"), ...)

The middleware also exports KBIDFromKnowledgeIDParam so chunk routes
(URL :knowledge_id) can adopt the same pattern in a follow-up PR
without reshaping the helper.

Five new tests cover own-KB, not-found, shared-with-sufficient-perm,
shared-but-below-min, and missing-tenant-context paths. Handler
tests still pass (no behaviour change).
2026-05-18 17:28:58 +08:00
wizardchen a16fe83034 fix(rbac): pin organization owner tenant in DB instead of deriving at runtime
Plan 3's isOwnerTenant resolved "is this tenant the owner tenant?"
by looking up the owner user's CURRENT tenant via userRepo. That
breaks silently the moment the owner user is moved to another
tenant: the formerly-protected owning tenant becomes removable
(can be kicked from OTM, role-downgraded), and the new tenant the
owner moved to is treated as untouchable even though it isn't even
in OTM.

Promote owner-tenant to a first-class column on `organizations`:

- New migration 000046 adds owner_tenant_id BIGINT NOT NULL with
  index. Backfill: (1) owner user's current tenant when known,
  (2) earliest Admin tenant in OTM as fallback for orphans, then
  (3) RAISE EXCEPTION on anything still NULL — refuse to silently
  ship a bogus value. SQLite init schema mirrors the column.

- CreateOrganization now sets OwnerTenantID at create time so the
  pinned value is correct for all new orgs.

- isOwnerTenant becomes a pure equality check on the persisted
  column — no userRepo lookup, no error tuple. Fail-closed on a
  zero column value (legacy data not yet backfilled or unit tests
  bypassing the migration) so the worst case is "membership table
  frozen for that org" rather than "owner tenant is removable",
  matching the prior fail-closed-on-error semantics.

- RemoveTenantMember / UpdateTenantMemberRole simplify to a single
  bool check.
2026-05-18 17:28:58 +08:00
wizardchen f6684f226b fix(rbac): allow source-tenant Admin+ to manage their cross-tenant shares
Plan 3 of #1303 lifts share ownership to the tenant level, but
RemoveShare / UpdateSharePermission still gated on the original sharer
user. If that user left the tenant or moved on, the source tenant lost
the ability to revoke its own share — only the target org's admin
could clean up, which is governance theatre rather than ownership.

Extend the authz envelope on KB-share and agent-share mutations:

  (1) original sharer (unchanged);
  (2) source-tenant Admin+ — new, encodes "ownership lives in the
      tenant" by reading callerTenantRole from ctx;
  (3) target-org admin (unchanged).

Factored out as kbShareService.callerCanManageShare so Update and
Remove cannot drift apart.
2026-05-18 17:28:58 +08:00
wizardchen eda6b1011a fix(rbac): close org-tenant gate gaps from Plan 3 review
Three follow-ups from #1350 review:

1. DELETE /organizations/:id/members/:tenant_id was un-gated, so a
   tenant Viewer could remove their own tenant from an org via the
   members endpoint and bypass the Admin+ requirement that POST
   /:id/leave already enforces. Gate it with g.Admin() — symmetric
   with /leave.

2. isOwnerTenant() returned (false, err) when the owner user lookup
   failed, and the two call sites only checked the bool. A deleted /
   unreadable owner user would silently fall through and let admins
   remove or re-role the would-be owner tenant. Make the helper
   fail-closed: any lookup error now returns (true, err), and both
   call sites refuse the destructive op on error.

3. POST /:id/request-upgrade was un-gated. Approval of the request
   changes the whole tenant's org role, so it is in the same trust
   tier as join/leave/create. Add g.Admin().
2026-05-18 17:28:58 +08:00
wizardchen 928f5abe3f feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"

This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.

Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:

  effective = min(share.Permission, tenant_org_role, tenant_role_cap)

where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.

Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.

Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.

Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-18 17:28:58 +08:00
wizardchen c6a2f6c8cc refactor(permissions): enhance role-based access controls across knowledge base and agent views
- Updated visibility conditions for various UI elements based on user roles, ensuring that only users with appropriate permissions (Admin, Contributor, or KB creator) can access certain actions.
- Introduced computed properties to manage permissions for sharing knowledge bases, managing data sources, and executing graph extraction.
- Adjusted the AgentList and KnowledgeBase components to reflect these changes, improving user experience by preventing unauthorized actions from being displayed.
- Ensured that the backend routes are aligned with the updated frontend role checks, enhancing security and clarity in user interactions.
2026-05-18 17:28:58 +08:00
wizardchen 8c0510f2e1 fix(api): reject /agent-chat early when agent_id is missing
AgentQA used to trust request.AgentEnabled even when
resolveAgent() returned nil (no agent_id, or unresolvable id) — the
request reached the async goroutine, AgentQA service errored with
'custom agent configuration is required', and the user saw a broken
SSE stream with no actionable error.

Most likely root cause is a stale frontend settings store where
isAgentEnabled stayed true after selectedAgentId got blanked
(typically a cross-tenant switch where the previously selected
agent isn't visible in the new tenant). The frontend fix lives in
selectAgent() syncing both flags, but the handler shouldn't trust
that contract.

Add an early 400 with a clear message before executeQA dispatches.
Logs the actual request.AgentID (sanitized) so future stuck-state
reports are diagnosable from a single log line.
2026-05-18 17:28:58 +08:00
wizardchen 619e14a2eb fix(rbac): wire chunk-id ownership chain for generated-question delete
DELETE /chunks/by-id/:id/questions used to be gated at flat
Contributor+ as a deliberate carve-out: the URL only carries the
chunk's own id, so the chunk -> knowledge -> KB ownership chain
wasn't reachable from request params alone. The carve-out had two
problems:

  - Inconsistent with every other chunk mutation. PUT
    /chunks/:knowledge_id/:id is OwnedKBOrAdmin (KB creator OR
    Admin+); a Contributor with no KB ownership can't update a
    chunk but COULD delete a generated question on the same chunk.
  - User-visible: a Contributor in tenant A who never created any
    KB still saw the 'delete question' button work, while every
    other chunk control 403'd.

Wire the missing hop:

  KBCreatorLookupFromChunkIDParam (handler) reads :id, fetches the
  chunk via the existing GetChunkByIDOnly (unscoped repo call),
  re-checks chunk.TenantID against the caller's tenant context to
  block cross-tenant chunk-id probing, then defers to the existing
  resolveKBCreatorByKnowledgeID helper for the rest of the chain.

  OwnedChunkKBOrAdminFromChunkID (router) wraps it in
  RequireOwnershipOrRole(Admin, ...) — same matrix as every other
  chunk/knowledge/wiki mutation guard.

  chunks.DELETE switches from g.Contributor() to the new guard.

4 unit tests cover happy path, chunk-not-found, cross-tenant
chunk-id probing, and KB-not-found upstream — same edge-case shape
as the existing KBCreatorLookup_* tests so future refactors get
caught loudly.
2026-05-18 17:28:58 +08:00
wizardchen 1e8b5c8d40 refactor(api): drop deprecated tenant ConversationConfig field and KV write path
The Tenant.ConversationConfig field was marked deprecated when
CustomAgent (builtin-quick-answer) replaced the per-tenant defaults
path, but the JSONB column, GET/PUT KV handlers, write helpers and
read fall-throughs all stayed wired. This had two concrete failure
modes:

  - Input-field.vue silently PUT-ed per-user model picks back into
    /tenants/kv/conversation-config, which requires Admin+ — Viewers
    and Contributors got a 403 every time they switched chat models.
  - The same write also overwrote the tenant-shared default for every
    other user, turning a per-user UX into a per-tenant footgun.

Remove the field, the struct, the validate helper, the PUT case and
the two dead Summary helpers in session/helpers.go. KnowledgeSearch
loses its tenant-conversation fallback (one chunk-tool reader); the
global config + hardcoded defaults still cover it. session_agent_qa
already required agents to declare their own rerank_model_id, so no
behaviour change there.

Frontend follows:
  - Input-field.vue: per-user 'last selected chat model' moves to
    localStorage, no API call.
  - AgentEditorModal: defaults for builtin-quick-answer creation now
    come from prompt-templates (system/context/rewrite/fallback) and
    retrieval-config (top_k / threshold), the two endpoints that
    actually own those settings now.
  - AgentSettings.vue: orphan settings page with no router/import
    referencing it; deleted.
  - api/system: getConversationConfig / updateConversationConfig and
    the ConversationConfig interface go away.

Net: 137 insertions, 2761 deletions.
2026-05-18 17:28:58 +08:00
wizardchen 0e1482bc4c fix(rbac): propagate TenantRole through logger.CloneContext
logger.CloneContext rebuilds a fresh context.Background() and only
copies a hand-curated allow-list of context keys (LoggerContextKey,
TenantIDContextKey, UserContextKey, etc.) before passing it to
downstream handlers — the chat / agent QA pipelines all do
`ctx := logger.CloneContext(c.Request.Context())` at the top of
parseQARequest.

TenantRoleContextKey was missing from that allow-list. Any call site
that read the role via TenantRoleFromContext(reqCtx.ctx) saw the
type-zero TenantRole and the helper's least-privilege fallback
returned Viewer. The auth middleware had set role=Owner correctly
on the parent context, but the cloned context lost it.

Symptom: an Owner of their home tenant got 403 on POST /agent-chat
when the agent had runnable_by_viewer=false, because the gate at
session/qa.go:520 saw role=viewer despite middleware logging
role=owner two ms earlier.

Diagnostic log lines added in the previous commits (auth.go:145
"[auth] resolved role=...", auth.go:283 "[auth] resolveTenantRole
step1 hit row_role=...") made this immediately visible — a request
showed role=owner at the middleware layer and role=viewer at the
handler layer, which can only mean ctx propagation lost it. Keeping
the diagnostic logs in place so similar regressions surface fast.

Fix is one line: add TenantRoleContextKey to the CloneContext
allow-list, with a comment pinning it to the bug it fixes so a
future drive-by removal has to weigh that loss.
2026-05-18 17:28:58 +08:00