Commit Graph

2340 Commits

Author SHA1 Message Date
wizardchen 9733bd28bf feat(storage): add storage backend management API and UI enhancements
Implement a comprehensive API for managing storage backends, including endpoints for listing, creating, updating, and deleting storage instances. Enhance the UI to support multi-instance configurations, allowing users to manage multiple storage types effectively. Update localization files for new storage backend features and improve error handling for connectivity tests. This update also includes detailed documentation for the storage backend API.
2026-07-15 14:54:39 +08:00
wizardchen a8dea5f9a3 feat(storage): improve storage connectivity error handling and multi-instance support
Enhance error messaging for storage connectivity issues by sanitizing raw errors to prevent leaking sensitive information. Introduce a new utility function for consistent error handling across storage backend operations. Additionally, implement logic to support multi-instance storage backends, allowing for more flexible workspace configurations and improved reporting of active storage providers.
2026-07-15 14:54:39 +08:00
wizardchen 6349fff002 feat(security): enhance image source protection for storage backends
Update the image source protection logic to include support for storage backend images. Introduce new regex patterns for identifying storage URLs and refactor the image tag building process to ensure proper handling of both provider and storage backend images. Add corresponding tests to validate the new functionality.
2026-07-15 14:54:39 +08:00
wizardchen f8b3e5f148 feat(storage): add multi-instance storage backends per workspace
Introduce tenant-scoped storage backend registry so workspaces can register
multiple storage instances and bind each knowledge base to a specific backend,
with migration from legacy tenant-level storage config and new settings UI.
2026-07-15 14:54:39 +08:00
wizardchen 139ec1e6b8 fix(storage): align default provider fallback with STORAGE_ALLOW_LIST
Extract storage allowlist into internal/storageallowlist so KB creation
uses FirstAllowed when tenant default_provider is empty or disallowed.
Share pickUsableStorageProvider via editorResources for consistent UI
selection, and add regression tests for OBS status and allow-list defaults.
2026-07-15 11:05:00 +08:00
wizardchen 97490ef7ed feat(api-keys): enhance manage_kbs capability for KB lifecycle management
Update the manage_kbs capability to cover the full lifecycle of knowledge bases, including create, copy, duplicate, update, and delete operations. Adjust related API routes and tests to ensure proper access control and policy requirements for knowledge base management.
2026-07-15 10:56:06 +08:00
ShadowSmallBaby b3bb60f03c fix: 修复/api/v1/tenants/kv/storage-engine-config接口不返回华为云obs配置导致创建知识库弹窗存储引擎华为云OBS未正确显示不可用标识
fix: 修复租户default_provider为空时降级至local,而local未被允许时导致创建知识库报错: Storage provider is not allowed by STORAGE_ALLOW_LIST
2026-07-15 10:54:14 +08:00
wizardchen e3b7ad98c2 feat(api-keys): let manage_kbs cover full KB lifecycle routes
Align create/copy/duplicate with update/delete under manage_kbs or
full-access, and update capability docs and UI hints accordingly.
2026-07-15 10:41:19 +08:00
Binks Riverton be88a56245 fix(knowledge): reload doc list after tag rename in manage drawer 2026-07-15 10:12:31 +08:00
wizardchen bdd459b071 fix(frontend): open chat reference links in new tabs
Prevent in-chat navigation from replacing the active conversation by
opening knowledge-base and citation links in a new tab with
noopener/noreferrer, and apply the same policy to sanitized markdown links.
2026-07-14 18:13:53 +08:00
wizardchen ace84bc757 fix(knowledge): recover tasks safely after server restart
Stop failing distributed-mode knowledge rows on startup; let housekeeping
decide using span activity and Asynq queue state. In Lite mode, resume
wiki work from durable task_pending_ops and surface SERVER_RESTART errors
in the processing timeline with i18n support.
2026-07-14 17:55:26 +08:00
wizardchen 28ebcf2525 feat(chat): polish follow-up suggestion loading and answer toolbar timing
Move follow-up loading into the answer toolbar so the card only appears when suggestions are ready, and delay toolbar actions until the typewriter finishes revealing the answer.
2026-07-14 17:49:03 +08:00
wizardchen 01f1011511 feat(system): add cursor pagination for runtime queue tasks
Replace offset-based page numbers with opaque Redis-backed cursors so
scheduled/retry tasks sort by next run time and live lists stay stable
when queue state changes between requests.
2026-07-14 16:53:07 +08:00
wizardchen f583d77fa0 feat(knowledge): enhance error handling in knowledge spans
Refactor the error handling logic in the GetKnowledgeSpans function to utilize a new helper function, knowledgeSpansLastError, which consolidates the logic for determining the last error from span failures and knowledge row failures. This improves clarity and maintainability of the code. Additionally, update test cases to cover new scenarios for last error handling.
2026-07-14 16:49:56 +08:00
wizardchen fc03e4862d fix(knowledge): align processing timeline status with knowledge row
Latest attempt tabs and header now trust parse_status over a closed root span, surface recovery failures via last_error, and keep attempt hover styles from washing out the active tab.
2026-07-14 16:49:56 +08:00
wizardchen 41975abcc5 refactor(terminology): rename user-facing tenant labels to workspace
Align product copy, API docs, Swagger descriptions, and i18n strings
with workspace terminology while keeping internal tenant identifiers
and headers unchanged for API compatibility.
2026-07-14 16:14:43 +08:00
wizardchen e1572b71b9 feat(models): let system admins edit builtin model config
Allow platform system administrators to update builtin model parameters
and credentials from the management UI while keeping secrets masked in API
responses. UI edits clear YAML ownership so startup reconciliation no
longer overwrites runtime overrides.
2026-07-14 15:56:02 +08:00
wizardchen f409bd756b feat(tests): enhance image multimodal service tests and runtime task handling
Added new tests for the ImageMultimodalService to verify the handling of orphan tasks and the finalization of pending counters. Improved runtime task inspection with better error handling and task cancellation logic, ensuring robust management of orphan tasks in the system.
2026-07-14 15:38:06 +08:00
wizardchen 4ae48d6acf feat(system): expand runtime queue task inspection and safe actions
Replace failed-only task views with state-aware runtime task listing,
operator-safe cancel/run-now/delete actions, orphan detection, and
multimodal queue cleanup for deleted knowledge rows.
2026-07-14 15:38:06 +08:00
wizardchen 1eae475a1f feat(i18n): enhance localization for task management and queue descriptions
Added new translations and improved existing ones for English, Korean, Russian, and Chinese. Introduced short weight labels, refined failed task notices, and updated queue names and descriptions for better clarity and consistency across languages.
2026-07-14 13:11:06 +08:00
wizardchen 6171962e56 feat(system): add failed-task inspection and manual retry for runtime queues
Give system admins visibility into asynq tasks that exhausted retries, with safe retry/delete actions and operator-facing UI in Runtime Queues.
2026-07-14 13:11:06 +08:00
darcy f7ec759cc6 fix(knowledge): propagate chunking strategy to parent-child splitter configs
buildParentChildConfigs only copied ChunkSize/ChunkOverlap/Separators into
the parent and child SplitterConfigs, dropping base.Strategy. Since an empty
Strategy resolves to the legacy tier (resolveChainWithProfile treats "" as
legacy for backwards compatibility), parent-child chunking always ran the
legacy recursive splitter regardless of the configured strategy: no heading
alignment and no ContextHeader breadcrumbs in child embeddings, silently.

The flat path (buildSplitterConfigFromChunking) already propagates Strategy;
parent-child predates the strategy system (b3334990, 2026-03) and was not
updated when the 3-tier strategy landed (ae6fde23, 2026-05). SplitParentChild's
docstring ("Child splitting honours childCfg.Strategy") and mergeBreadcrumbs
both assume the strategy reaches the parent-child path.

Copy base.Strategy into both configs and add a regression test.
2026-07-14 11:56:29 +08:00
Binks Riverton 3b587c401d fix(frontend): add cancelled, finalizing, draft statuses to parse-status dropdown
The '全部状态' (All Statuses) filter dropdown on the knowledge base documents page was missing the 'cancelled', 'finalizing', and 'draft' status options, even though documents can reach these states (e.g. cancelled via the '停止解析' button). Added all three status options and their i18n translations for zh-CN, en-US, ru-RU, and ko-KR.
2026-07-14 11:38:30 +08:00
wizardchen abeb4a0126 style(system): polish admin password reset trigger and dialog layout
Improve the password reset entry button and dialog styling for clearer
visual hierarchy, consistent spacing, and better mobile responsiveness.
2026-07-14 11:30:15 +08:00
wizardchen 087fcc9fe7 feat(agent): add question suggestions with after-answer follow-ups
Enable agents to configure starter prompts and async follow-up questions after completed answers, with durable caching, analytics events, and chat/embed UI integration.
2026-07-14 11:17:25 +08:00
wizardchen a6c85dad28 feat(middleware): enhance logger to sanitize new and old password fields
Updated the logger to include regex patterns for sanitizing "new_password" and "old_password" fields in JSON logs. Added a corresponding test case to verify the sanitization functionality.
2026-07-13 20:11:00 +08:00
wizardchen 8ec65941a5 feat(system): add admin password reset with session revocation
Let system administrators reset another user's local password from the settings UI, revoke all active sessions, and record the action in the audit log.
2026-07-13 20:11:00 +08:00
wizardchen 8554c422dc feat(frontend): reorganize system settings into tabbed sections
Group global settings by product area (access, tenant, runtime, security),
add overview summary badges and inline save feedback, surface high-risk
keys, and collapse config-priority docs into a disclosure panel.
2026-07-13 19:42:18 +08:00
wizardchen 87ba3fc2de refactor(auth): unify OIDC provisioning under shared default_tenant_mode policy
Drop the standalone OIDC_AUTH_DEFAULT_TENANT_MODE / oidc_auth.default_tenant_mode
knob so OIDC auto-provisioning and local password registration share the single
auth.default_tenant_mode policy (resolved by the callback handler). Repair
tenantless sessions by falling back to the first active membership on the login
happy path and stale-preference paths, allow tenantless PUT /auth/me/preferences,
clear the tenant snapshot on /auth/me responses without a tenant, and default the
frontend canCreateTenant capability to fail-closed until /auth/me confirms it.
2026-07-13 19:14:49 +08:00
wizardchen ecf2c1fb9b feat(auth): support tenantless provisioning and gated self-service workspace creation
Allow deployments to register users without auto-creating a personal tenant, expose creation policy via /auth/me, and guide tenantless accounts through a workspace onboarding flow.
2026-07-13 18:13:51 +08:00
wizardchen 50b8c0a72e fix(security): harden agent share listing and align full-access API key routes
Enforce tenant ownership in ListSharesByAgent so full-access API keys cannot enumerate cross-tenant shares, and allow full-access keys to create KBs and manage share routes consistently with other tenant-wide operations.
2026-07-13 17:09:58 +08:00
wizardchen 1dfa23d728 docs(model): add Requesty to API and wiki provider lists
Sync hand-written provider tables with the merged Requesty provider support.
2026-07-13 17:08:36 +08:00
Thibault Jaigu 35b92f73aa feat(provider): add Requesty as an OpenAI-compatible model provider
Requesty is an OpenAI-compatible LLM gateway that uses the same provider/model
naming convention as OpenRouter, so this mirrors the existing OpenRouter provider.

- internal/models/provider/requesty.go: RequestyProvider mirroring openrouter.go
  (base URL https://router.requesty.ai/v1, REQUESTY_API_KEY, RequiresAuth)
- provider.go: ProviderRequesty const + AllProviders() entry + DetectProvider case
- internal/types/model.go + client/model.go: ModelSourceRequesty
- provider_test.go: TestRequestyProviderValidation + DetectProvider case
- frontend model editor + i18n (en/zh/ko/ru) + docs/BUILTIN_MODELS.md provider list
- docs/swagger.{json,yaml} + docs.go: requesty enum value (hand-edited to match; these
  are normally swag-generated)

Verified: go build/vet/test pass; live call to https://router.requesty.ai/v1 succeeds.
Signed-off-by: Thibault Jaigu <thibault.jaigu@gmail.com>
2026-07-13 17:03:23 +08:00
wizardchen eacc278b9a fix(wiki): strip internal chunk aliases from page content
Wiki ingest prompts previously encouraged inline [cNNN] citations that
leak internal chunk metadata into reader-facing Markdown. Strip legacy
aliases on read/write paths and update the modify prompt to keep prose
clean while source links remain in ChunkRefs.
2026-07-13 15:59:09 +08:00
wizardchen 07ba01ec93 feat(worker): replace aggregate asynq concurrency with per-pool governance
Split the monolithic upstream worker budget into explicit core, post-process,
enrichment, maintenance, and shared elastic pools, expose live cluster capacity
on the runtime dashboard, and retire the legacy asynq.concurrency setting.
2026-07-13 15:51:06 +08:00
wizardchen 5eefa70e6f Enhance knowledge base configuration by normalizing and validating custom instruction fields. Implemented length checks for prompt instructions and added tests for validation logic. Updated GraphSettings to retain custom instructions when extraction is disabled. 2026-07-13 12:16:19 +08:00
wizardchen c1cddb4d4d Add instruction fields to knowledge base configuration 2026-07-13 12:16:19 +08:00
wizardchen def92bb74f fix(router): restrict KB-scoped file proxy to exports/ paths
Borrowers with shared-KB read access could previously request any
owner-tenant storage path via /knowledge-bases/:id/files. Limit the
proxy to exports/ embedded images; raw knowledge uploads continue to
use /knowledge/:id/download.
2026-07-13 12:13:58 +08:00
wizardchen 800cea0826 fix(files): serve shared KB images via KB-scoped file proxy
Org-shared and agent-visible knowledge bases store images under the owner
tenant's storage path, which the tenant-scoped /files route rejects for
borrowing tenants. Add GET /api/v1/knowledge-bases/:id/files gated by
RequireKBAccess and route frontend image hydration through it when kbId
is known.
2026-07-13 12:13:58 +08:00
wizardchen abcecc870a fix(router): allow scoped API keys to poll FAQ import progress
Register the FAQ import progress route through apiKeyRoute with retrieve
or ingest capability so scoped keys that start imports can poll task
status without hitting APIKeyGate fail-closed 403.
2026-07-13 11:07:07 +08:00
wizardchen c8793b001d fix(spans): make fitSpanName rune-aware and normalize lookups
Follow-up to the knowledge span name column expansion: truncate span
names by rune count to match PostgreSQL VARCHAR(255), apply the same
normalization in LookupSpanByName, and add regression tests for mid-length
wiki slugs and CJK truncation boundaries.
2026-07-13 10:38:50 +08:00
wizardchen 26f8afa3db fix(spans): expand knowledge span name column and truncate long wiki names
Wiki ingest builds postprocess.wiki.page[<slug>] subspan names that can exceed
varchar(64); widen the column to 255 and hash-suffix truncation so long slugs
still persist distinctly.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-13 10:35:44 +08:00
wizardchen 083e9738f6 fix(docreader): detect legacy doc payloads 2026-07-11 12:21:56 +08:00
wizardchen ac8c5c5540 chore(deps): resolve open Dependabot security alerts across Go, npm, and pip.
Bump vulnerable runtime and transitive dependencies in docreader, frontend, and the Go module graph to address current GitHub security advisories.
2026-07-11 12:15:39 +08:00
wizardchen b9cccc7bee feat(system): expose model concurrency stats on runtime queues dashboard
Surface per-model active/waiting/limit from the background task limiter so admins can distinguish task scheduling from model-service throttling, with full i18n coverage.
2026-07-11 07:17:28 +08:00
wizardchen 80556fbaa7 refactor(RuntimeQueues): enhance UI layout and styling for better readability
Updated the RuntimeQueues component to improve the layout and styling of the task queue display. Changes include a new grid layout for pool items, updated class names for better semantic structure, and adjustments to font sizes and spacing for improved readability. Additionally, added a note section with enhanced styling for clarity.
2026-07-10 23:54:24 +08:00
wizardchen d496d0fc73 feat(task-queue): split upstream workers into isolated core/enrichment/maintenance pools
Route summary/sync/maintenance tasks to dedicated queues and servers so parsing, enrichment, and batch work no longer share one concurrency budget, while extending the runtime dashboard and settings validation to reflect the new topology.
2026-07-10 23:51:29 +08:00
wizardchen fe47ac1f96 feat(system): add runtime task queue observability dashboard for system admins
Expose live asynq queue depths and worker-pool concurrency via a new SystemAdmin API and settings UI, so operators can monitor parsing/wiki backlog without Redis CLI access.
2026-07-10 23:51:29 +08:00
养乐多 30f7630ac4 fix(im/feishu): reply via reply-message API so replies land in the original thread
Previously the Feishu adapter sent replies with the "send message" API
(POST /im/v1/messages) without a root_id. In topic-enabled groups this
spawned a new topic for every reply instead of attaching to the thread
the user @-mentioned the bot from, fragmenting the conversation.

Switch SendReply (text) and sendCardByCardID (streaming card) to the
"reply message" API (POST /im/v1/messages/:message_id/reply), using
incoming.MessageID as the path parameter. Feishu automatically replies
in thread form when the replied-to message is itself a thread message,
so no reply_in_thread flag or session-mode plumbing is needed.

Add sendWithFallback to retry via the plain send-message API on
fallback-eligible error codes (230019 topic gone, 230054 unsupported
message type, 230071 group does not support reply-in-thread), and to
skip straight to the fallback when incoming.MessageID is empty. This
keeps replies deliverable even when the reply API cannot be used.

Also add a debug log in convertEvent (longconn.go) that prints every
raw received event (msg_type/chat_id/msg_id/root_id/parent_id/thread_id/
content) to aid diagnosing intermittent event-delivery issues.

No interface signature changes; other platform adapters untouched.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-10 22:51:57 +08:00
wizardchen b4b63a0c1f feat(tenant): optional legacy auto-create API key on tenant creation
Add a tenant.auto_create_api_key setting (env
WEKNORA_TENANT_AUTO_CREATE_API_KEY, default false) that restores the
pre-break-change behaviour of minting a full_access API key when a
tenant is created and returning its plaintext token in the create
response (data.api_key).

- system_setting: register tenant.auto_create_api_key bool spec
- handler: gate auto key creation behind the setting; embed the token
  via tenantWithAPIKey without re-adding a persisted api_key column;
  key-creation failures log a warning and do not fail tenant creation
- client: add optional Tenant.APIKey field populated only on create
- docs/env: document the switch (.env.example, docker-compose,
  tenant.md, regenerated swagger)
2026-07-10 15:50:15 +08:00