Commit Graph

10 Commits

Author SHA1 Message Date
wizardchen 8ca9c117bb feat(knowledge): enhance FAQ entry filtering and tagging
- Updated the listFAQEntries function to support multiple tag IDs for filtering.
- Modified the frontend components to accommodate the new tag_ids parameter.
- Enhanced the KbTagManageDrawer and FAQEntryManager components to reflect changes in tag handling.
- Improved API documentation to clarify the new tag filtering capabilities.

This commit enhances the flexibility of FAQ entry retrieval, allowing users to filter by multiple tags for more precise results.
2026-07-23 10:50:32 +08:00
wizardchen b6a3b347da feat(knowledge): enhance FAQ export functionality
- Updated the exportFAQEntries function to support exporting FAQ entries in both CSV and JSON formats.
- Modified the frontend to include options for selecting the export format.
- Enhanced localization files to reflect the new export options in English, Russian, and Chinese.
- Implemented new tests to validate the export functionality for both formats.

This commit improves the flexibility of FAQ exports, allowing users to choose their preferred format for better integration with other tools.
2026-07-23 10:50:32 +08:00
wizardchen 5af8eae81d feat(knowledge): enhance activity logging for FAQ imports
- Added new actions for FAQ import lifecycle: `faq.import_started`, `faq.import_completed`, and `faq.import_failed`.
- Updated activity details to include success, failure, and skipped counts for FAQ imports.
- Enhanced UI to display additional activity information, including filtering options and clearer summaries.
- Implemented tests to validate the new FAQ import outcomes and activity details.

This commit improves the tracking and visibility of FAQ import activities within the knowledge base, ensuring better audit trails and user feedback.
2026-07-23 10:50:32 +08:00
wizardchen a3a0308b62 feat(knowledge): add scoped activity audit trail 2026-07-23 10:50:32 +08:00
wizardchen 035093dc35 feat(configuration): update asynq concurrency settings and enhance FAQ handling
- Increased the default worker pool size for the asynq server from 16 to 32 to improve performance during concurrent uploads.
- Updated the .env.example and docker-compose.yml files to reflect the new default concurrency setting.
- Enhanced the handling of FAQ entries in the tool results, ensuring distinct identification through standard questions, improving user experience and clarity in the output.
- Refactored related components to utilize the new FAQ attributes, ensuring consistent display and interaction across the application.
2026-06-04 15:48:04 +08:00
wizardchen 928f5abe3f feat(rbac): lift org membership from per-user to per-tenant (Plan 3, #1303)
Before this change, the `organization_members` table keyed on user_id —
joining an org as a Viewer in tenant T meant only that one user got
visibility on shared KBs/agents, while the rest of T (including its
Owner) saw nothing. The mismatch surfaced as confusing UX bugs:
"my colleague shared this with us, why can't I see it?"

This change replaces `organization_members` with
`organization_tenant_members`. Membership and Org-RBAC role now key on
(org_id, tenant_id); `representative_user_id` stays as a display-only
label of the user who first brought the tenant in.

Permission resolution gains a 3-D cap so tenant-RBAC stays honest under
cross-tenant sharing:

  effective = min(share.Permission, tenant_org_role, tenant_role_cap)

where tenant_role_cap pins tenant Viewers to OrgRoleViewer regardless
of the org-level grant. A Viewer in their own tenant cannot edit a
shared KB even if the org granted Editor — preserves the
"Viewer = read-only everywhere" promise.

Migration 000045 backfills `organization_tenant_members` from the old
table by collapsing each (org, tenant) group to its max role, then
renames the old table to `organization_members_pre_plan3` for safe
rollback. Sqlite mirror (Lite mode) is fresh-bootstrap: the init
schema replaces the old table directly.

Routes: member-management URLs change from `:user_id` to `:tenant_id`.
Org create / join / leave now require Admin+ in the caller's tenant —
joining an Org affects everyone in T, so the decision can't sit with
a Viewer.

Frontend: OrganizationSettingsModal + store update to address members
by tenant_id.
2026-05-18 17:28:58 +08:00
ochan.kwon 7478e3cddb refactor(retriever): introduce factory for KB-scoped retrieve engine resolution
Extracts the 25 repeated NewCompositeRetrieveEngine call sites across
seven services into two factory functions with tenant-ownership
verification: CreateRetrieveEngineForKB (synchronous) and
CreateRetrieveEngineFromPayload (async task handlers). Promotes
GetByStoreID from concrete-only to interfaces.RetrieveEngineRegistry.
Extends KBDeletePayload / IndexDeletePayload with an omitempty
VectorStoreID snapshot.

Design highlights
- Factory verifies tenant ownership of the resolved store
  (defense-in-depth) so a gap in PR3 validation or a tampered Asynq
  payload cannot cross tenants. Cross-tenant attempts return
  ErrVectorStoreForbidden with a structured log entry.
- Sentinel errors (ErrTenantInfoMissing, ErrVectorStoreNotFound,
  ErrVectorStoreForbidden) let async handlers classify non-retryable
  failures as asynq.SkipRetry.
- nil and empty-string pointers for vectorStoreID normalize to
  "unbound" so callers never send an empty UUID to GetByStoreID.
- resolveBoundEngine constructs engineInfos directly (with
  slices.Clone of Support()) instead of going through
  NewCompositeRetrieveEngine. The latter only reads from the
  byEngineType map and cannot reach DB stores in byStoreID -- multiple
  stores can share the same engine type. Unbound fallbacks still reuse
  NewCompositeRetrieveEngine so the tenant effective-engines path is
  byte-for-byte unchanged.
- DeleteKnowledgeBase now reads the KB before soft-delete so the
  enqueued KBDeletePayload carries a VectorStoreID snapshot; once the
  row is soft-deleted GORM's default scope hides it. Cost: +1 SELECT
  per KB-delete request (a rare admin operation).
- enqueueIndexDeleteTask gains a vectorStoreID *string parameter that
  the single caller (tag.go) populates from the owning KB. The worker
  (ProcessIndexDelete) validates ownership via the factory.
- cleanupKnowledgeResources loads the KB so bound-store cleanup routes
  correctly. If the load fails it warns and falls back to tenant
  effective engines; orphan vectors become observable in logs instead
  of disappearing silently.
- DeleteKnowledges (batch) stays on the tenant effective-engines path
  -- batches can span stores and multi-store fan-out is PR4 scope.
  Noted inline.

Merge safety

All knowledge bases have VectorStoreID = NULL at merge time (introduced
nullable in PR1, Tencent/WeKnora#994). Every factory call falls back to
NewCompositeRetrieveEngine(registry, tenantInfo.GetEffectiveEngines()),
which is the existing code path for these KBs. Async payloads from
before this change decode to VectorStoreID = nil (omitempty) and take
the same fallback.

Testing

- go build ./...
- go vet ./...
- go test -race -count=1 ./internal/application/service/retriever/...
  covers unbound (nil/empty), bound, cross-tenant, unregistered store,
  ownership infra error, legacy payload shapes (missing field/null/
  empty string/nil), tampered payload, and parallel invocation.
- grep -r NewCompositeRetrieveEngine internal/application/service
  returns zero results outside factory.go/composite.go.

Refs: Part of Tencent/WeKnora#993. Depends on Tencent/WeKnora#994.
2026-05-14 15:53:58 +08:00
wizardchen d4c30126a9 refactor(knowledge): split 9.8k-line service file by responsibility
internal/application/service/knowledge.go had grown to 9883 lines /
149 functions spanning CRUD, document processing, summary/question
generation, clone/move, FAQ (CRUD+import+index+export), wiki cleanup,
multimodal image, and file utilities. Navigating it, or diffing
changes in it, was increasingly painful.

Purely mechanical split into 8 files in the same package. No
signatures, behaviors, or types were modified. knowledgeService
struct, NewKnowledgeService, and package-level errors remain in
knowledge.go unchanged. Per-file imports were trimmed by goimports.

  knowledge.go              497  struct + constructor + errs + CRUD reads + Search
  knowledge_create.go      1144  CreateFrom{File,URL,Passage,Manual} + helpers
  knowledge_delete.go       686  Delete + wiki cleanup + ProcessKnowledgeListDelete
  knowledge_process.go     2407  ProcessDocument/Summary/Question + Reparse + UpdateImageInfo
  knowledge_clone_move.go  1049  Clone* + Move* + progress persistence
  knowledge_faq.go         1932  FAQ CRUD + Search + Export
  knowledge_faq_import.go  1939  FAQ import/validate/index + runningFAQImportInfo
  knowledge_util.go         352  file-type/url/hash + VLM+Storage config + resolveFileService

Verification: go build, go vet, gofmt on the 8 files, and
golangci-lint all clean (22 issues post-split matches 22 pre-split
in knowledge.go — no new issues introduced). The pre-existing
unused warnings on getVLMConfig / buildStorageConfig were left as-is
rather than removing dead code beyond the scope of this task.
2026-05-06 17:40:10 +08:00
lyingbug 0d8540c93c Revert "refactor(knowledge): split 9.8k-line service file by responsibility"
This reverts commit f67d0b53ce.
2026-05-06 17:09:24 +08:00
wizardchen f67d0b53ce refactor(knowledge): split 9.8k-line service file by responsibility
internal/application/service/knowledge.go had grown to 9883 lines /
149 functions spanning CRUD, document processing, summary/question
generation, clone/move, FAQ (CRUD+import+index+export), wiki cleanup,
multimodal image, and file utilities. Navigating it, or diffing
changes in it, was increasingly painful.

Purely mechanical split into 8 files in the same package. No
signatures, behaviors, or types were modified. knowledgeService
struct, NewKnowledgeService, and package-level errors remain in
knowledge.go unchanged. Per-file imports were trimmed by goimports.

  knowledge.go              497  struct + constructor + errs + CRUD reads + Search
  knowledge_create.go      1144  CreateFrom{File,URL,Passage,Manual} + helpers
  knowledge_delete.go       686  Delete + wiki cleanup + ProcessKnowledgeListDelete
  knowledge_process.go     2407  ProcessDocument/Summary/Question + Reparse + UpdateImageInfo
  knowledge_clone_move.go  1049  Clone* + Move* + progress persistence
  knowledge_faq.go         1932  FAQ CRUD + Search + Export
  knowledge_faq_import.go  1939  FAQ import/validate/index + runningFAQImportInfo
  knowledge_util.go         352  file-type/url/hash + VLM+Storage config + resolveFileService

Verification: go build, go vet, gofmt on the 8 files, and
golangci-lint all clean (22 issues post-split matches 22 pre-split
in knowledge.go — no new issues introduced). The pre-existing
unused warnings on getVLMConfig / buildStorageConfig were left as-is
rather than removing dead code beyond the scope of this task.
2026-05-06 17:06:05 +08:00