Commit Graph

2164 Commits

Author SHA1 Message Date
Anky c007a897cf feat: support multi-tag for document knowledge (#1790)
* feat: support multi-tag for document knowledge

- Add knowledge_tag_relations join table for many-to-many tag-knowledge mapping
- Migrate existing single tag_id data to the new table
- Update Knowledge struct, filter, payload types (TagID -> Tags/TagIDs)
- Rewrite repository layer: SetKnowledgeTags, GetKnowledgeTags, ListIDsByTagIDs
- Update tag count references to query knowledge_tag_relations
- Update service/handler layers for multi-tag create/list/batch-update
- Frontend: multi-select tag sidebar, multi-tag chip display, tag toggle in popup
- FAQ tags remain single-tag model (unchanged)

* feat(knowledge): tag edit dialog with dynamic overflow and inline creation

- Add TagEditDialog component with search, multi-select, and confirm
- Replace inline t-popup tag selector with dialog-based editing
- Tags overflow with +N badge, visible count adapts to container width via ResizeObserver
- Date/card layout: fixed date width, tags flex to fill remaining space
- Add persistent new-tag input bar in dialog (Enter to create, auto-select)
- Tag creation emits callback to refresh parent tag list immediately
- Apply same overflow + dialog trigger to both card view and list view
- Add i18n keys: tagEditDialogTitle, tagEditSearch, tagSelectedCount, tagOverflowTip, tagNewPlaceholder

* fix: exclude soft-deleted knowledge from tag reference counts

CountReferences, BatchCountReferences, and DeleteUnusedTags now
JOIN with knowledges table to filter out soft-deleted records
when counting knowledge_tag_relations. Previously they queried
knowledge_tag_relations directly without the deleted_at check,
causing stale tag counts after knowledge deletion.

* fix: refresh tag counts after async knowledge deletion completes

Add a second delayed tag list refresh (2.5s) after tag deletion to capture
updated reference counts for remaining tags. The original immediate refresh
handles removing the deleted tag from the list, while this delayed refresh
ensures other tags' counts are correct after the async deletion finishes.

---------

Co-authored-by: lyingbug <xiayula2014@gmail.com>
2026-06-26 00:18:48 +08:00
wizardchen 5c0e97c12e fix(frontend): refresh batch reparse UI after async task enqueue
Optimistically mark submitted documents as pending and poll until the worker
updates the backend, preventing a stale completed state after list reload.
2026-06-25 23:57:54 +08:00
wizardchen 97b7bb9890 fix(knowledge): harden batch reparse UX and async processing
Move batch reparse work to an asynq task to avoid HTTP timeouts, add i18n
for all locales, guard in-flight documents, and prevent concurrent delete/reparse.
2026-06-25 23:57:54 +08:00
jw_mac a14c1d87aa feat(knowledge): add batch reparse capability
This commit introduces the Batch Reparse feature for the Knowledge Base.
Users can now select multiple documents and trigger a re-parsing pipeline in batches, streamlining the recovery from parse failures.
2026-06-25 23:43:27 +08:00
wizardchen 5f2ca6c975 fix(agent): reconcile agent stream mode and shared-agent readiness UX
Derive stream pipeline from selectedAgentId to fix quick-answer/isAgentEnabled drift, gate configure actions for shared agents, and polish selector detail panel positioning.
2026-06-25 23:33:39 +08:00
wizardchen 47478491c9 fix(agent): address readiness review findings
Tighten shared-agent readiness checks, surface not-ready toasts in the
selector, validate chat-model overrides in resolveChatModelID, and
improve configure deep-links for shared agents.
2026-06-25 23:33:39 +08:00
wizardchen 9ca4e1f23a feat(agent): enforce model readiness and polish selector UX
Block chat when agents lack an explicit chat or rerank model, guide users
through the selector and editor with field highlight, and align session QA
checks with the same readiness rules on backend and frontend.
2026-06-25 23:33:39 +08:00
wizardchen 1e16834d27 fix(frontend): reflect PDF scanned parsing in upload overview
Show the force-scanned mode in the parser summary line when enabled for
builtin PDF uploads, and add missing ko-KR/ru-RU i18n strings for the toggle.
2026-06-25 17:36:18 +08:00
杨聪聪 917feceb59 feat(docreader): support forcing PDF scanned-page OCR parsing 2026-06-25 17:30:22 +08:00
wizardchen 2c44a41f2a feat(models): add model test debugger for saved model configs
Let admins send real requests against configured chat, embedding, rerank,
VLM, and ASR models from settings, with thinking controls shown only when
the model supports them.
2026-06-25 17:05:33 +08:00
wizardchen 122408b283 feat(chat): unify resource chips and browse recent files in @ mention
Add recent=true to knowledge search so the mention picker can list recent
files without a keyword, and consolidate KB/FAQ/file chip styling across
chat input and message history.
2026-06-24 20:58:16 +08:00
wizardchen babe3f2315 fix(integrations): polish agent filter UX and embed session rebind
Add compact agent filter beside channel list titles with selected name
display, fix embed localStorage agentId read on rebind, add IM agent
rebind tests, and deep-link agent editor counts to the matching tab.
2026-06-24 20:09:19 +08:00
wizardchen ec5c3ac638 fix(frontend): polish integration channel card actions
Align add-card height with channel cards, show actions on hover,
and replace delete dialogs with in-place popconfirm like model settings.
2026-06-24 20:09:19 +08:00
wizardchen 635d7c9643 feat(integrations): polish embed/IM channel editors and agent rebind
Align embed channel setup with a step wizard, simplify deploy layout,
and allow changing bound agents on update for embed and IM channels.
2026-06-24 20:09:19 +08:00
wizardchen 2d6d1e3275 fix(frontend): correct integrations redirect tab and lazy-load panels
Map legacy section=integrations to tab=im instead of an invalid tab value,
and mount only the active IntegrationsModal tab to avoid duplicate API calls.
2026-06-24 20:09:19 +08:00
wizardchen 314b2a035c feat(frontend): add publish integrations center and streamline agent editor
Move IM and web embed channel management into a dedicated sidebar entry,
remove the user-menu IM overview submenu, and show a lightweight publish
channel status row in the agent editor basic settings instead of full panels.
2026-06-24 20:09:19 +08:00
zhouxun 4db727ac69 fix(excel): filter out image function strings from parsed content
WPS embeds images in cells using =DISPIMG("ID",mode) which may appear as plain text prefixed with "_xlfn." when processed by openpyxl. These function strings pollute the RAG-indexed document content, causing incorrect AI recall.

Add a regex filter (_is_image_function) that detects _xlfn.DISPIMG(...) and _xlfn.IMAGE(...) patterns and excludes them during row processing in ExcelParser.parse_into_text().

Refs: https://github.com/Tencent/WeKnora/issues/1779
2026-06-24 14:35:37 +08:00
wizardchen fa8e2bb087 fix(web-search): improve SearXNG test connection error messages
Return provider-specific diagnostics when connectivity tests return zero
results, including SearXNG unresponsive_engines details instead of a
misleading API key hint.
2026-06-24 14:28:49 +08:00
wizardchen 8242b6f298 refactor(frontend): update WeCom icon and styles in IM components
Replaced the WeCom SVG icon with a new design and updated the associated styles in the IMChannelPanel and SessionSourceFilter components to reflect the new color scheme. Adjusted layout properties for better responsiveness and visual consistency across the application.
2026-06-24 11:01:56 +08:00
wizardchen 7c8b15f022 fix(im): harden stream finalize and drop collapsed think header
Add agent complete wait timeout and answer fallbacks for cancel paths,
restore Telegram Markdown on finalize with plain-text retry, and show
answer-only final IM frames instead of the collapsed think summary.
2026-06-24 10:33:55 +08:00
wizardchen e6b3ef4f73 fix(im): unblock agent stream wait when QA fails before complete event
Close completeDone on pipeline errors and AgentQA early failures so
handleMessageStream and runQA do not hang waiting for EventAgentComplete.
2026-06-24 10:33:55 +08:00
wizardchen da3d65c6f2 fix(im): address stream display review feedback
Wait for agent complete without a short timeout, unify Telegram stream
parse mode, remove obsolete FinalAnswer handling, isolate per-section
newline tracking, add lifecycle tests, and update IM streaming docs.
2026-06-24 10:33:55 +08:00
wizardchen c3b036cb00 feat(im): align agent stream display with web thinking and tool progress
Mirror Web AgentStreamDisplay/RagPipelineProgress in IM text streams:
retract live answers into 思考过程, show Web-aligned tool titles with
query/summary from structured data, and collapse thinking on finalize
instead of dropping it entirely.
2026-06-24 10:33:55 +08:00
wizardchen ab3e472748 feat(frontend): implement session source filter component for improved session management
Added a new SessionSourceFilter component to allow users to filter session lists by source, enhancing the user experience. The component integrates with existing session management features and includes logic for displaying web and channel sources. Additionally, utility functions for managing session source options and visibility conditions have been introduced, along with corresponding unit tests to ensure functionality.
2026-06-24 10:30:10 +08:00
wizardchen 2cd1db859c fix(frontend): unify sidebar other-sources section typography
Channel folder headers and nested session rows now share 13px type,
consistent icon sizing, and aligned indentation so the menu no longer
mixes oversized session titles with undersized folder labels.
2026-06-24 10:30:10 +08:00
wizardchen 7855f36799 fix(model): block deletion when model is referenced by KB or agent
Reject model delete with HTTP 400 when the current tenant still has
knowledge bases or custom agents bound to the model ID.
2026-06-23 20:36:16 +08:00
wizardchen 775cc9ef11 fix(frontend): auto-scroll capped thinking block during streaming
Keep scrolling the thinking detail panel to the bottom while reasoning
content streams, not only before the first chunk arrives.
2026-06-23 20:17:19 +08:00
wizardchen bc9ad3aca0 fix(qa): wire agent thinking toggle and show reasoning in RAG timeline
Default unset agent thinking to explicit false so thinking_type models
receive disabled/enabled wire params. Render reasoning inside the quick-answer
pipeline timeline only when thinking events arrive, avoiding a flash of the
thinking step when thinking mode is off.
2026-06-23 20:17:19 +08:00
wizardchen be26563f86 fix(agent): remove unused RunnableByViewer gate blocking viewer chat
The runnable_by_viewer field was never populated for built-in agents,
so TenantRoleViewer users always hit 403 on agent-chat. Drop the field
and RBAC check so agent-chat matches knowledge-chat access for viewers.
2026-06-23 18:21:23 +08:00
wizardchen 35f717f7fa fix(datasource): optimize RSS incremental sync and surface partial failures
Skip article-page fetches when feed signals are unchanged, preserve sync
cursor across transient feed outages, and record partial multi-feed syncs.
2026-06-23 17:30:38 +08:00
wizardchen 11d461a4f7 fix(knowledgebase): delete data sources when knowledge base is removed
Stop cron schedules and cancel in-flight sync logs when a KB is deleted,
and skip datasource sync tasks if the target KB no longer exists.
2026-06-23 17:30:38 +08:00
wizardchen 3cb0e906a7 fix(datasource): harden RSS connector and editor UX after review
Scope auth headers to feed fetches, use feed-scoped external IDs and
content-based incremental fingerprints, tolerate partial feed failures,
store feed URLs in settings, and fix credential configured detection.
2026-06-23 17:30:38 +08:00
wizardchen e40ad802d7 feat(datasource): add RSS feed connector with full-text ingestion
Enable private RSS feeds as a knowledge base data source with custom
headers, article body fetching, and Markdown output for incremental sync.
2026-06-23 17:30:38 +08:00
wizardchen 94ce8fedb1 feat(mcp): add custom headers functionality to MCP service dialog
Introduce a new feature allowing users to add custom HTTP headers for MCP requests. This includes:

- A user interface for adding and managing custom headers within the MCP service dialog.
- Localization support for custom headers in English, Korean, Russian, and Chinese.
- Updates to the form submission process to include custom headers, ensuring they are serialized correctly.
- Removal of warnings for ignored headers, as all headers are now captured in the custom headers editor.

This enhancement improves the flexibility and usability of the MCP service configuration.
2026-06-22 21:08:06 +08:00
wizardchen 518add2641 feat(mcp): add code import functionality for MCP service configuration
Implement a new feature that allows users to import MCP service configurations via a standard JSON format. This includes:

- A toggle for importing from code with a user-friendly hint and placeholder.
- Error handling for various scenarios such as empty input, invalid JSON, and missing URLs.
- Toast notifications for successful imports and warnings about ignored headers.
- Updates to the MCP service dialog to support this new functionality, ensuring a seamless user experience during service creation and editing.
2026-06-22 20:45:50 +08:00
langcaiye 3eb49da746 feat(retriever): configure Tencent VectorDB replicas via env 2026-06-22 20:21:25 +08:00
wizardchen 7f3f219f58 feat(mcp): add OAuth2 authorization for MCP services
Support connecting to MCP servers that require OAuth2, with a Cursor-like
zero-config flow: the user provides only a URL and the system performs
metadata discovery, dynamic client registration (DCR), and the
authorization code flow with PKCE on first connect.

- Persist per-user access/refresh tokens and per-service DCR client
  credentials, encrypted at rest (AES-256-GCM)
- Store the PKCE verifier/state during the flow via Redis with an
  in-memory fallback for Lite mode
- Cache MCP clients per (service, user) so tokens stay user-isolated
- Add endpoints for authorize-url, callback, status, and token revoke;
  the callback is allowlisted for unauthenticated access
- Surface a structured "authorization required" hint when a tool call
  hits an unauthorized MCP service
- Frontend: add an auth-type selector with OAuth config, inline
  authorization status/actions, and render the connection test result
  inline in the editor drawer instead of a stacked centered dialog
2026-06-22 20:19:31 +08:00
Binks Riverton c25d762ed1 fix(organization): invalidate fetch cache after creating org
Reset organizationsLoadedAt to 0 after orgStore.create() succeeds,
so that the subsequent fetchOrganizations() call from
handleSettingsSaved is not blocked by the 60-second TTL cache.
This ensures resource_counts and other aggregated fields are
refreshed immediately after creating a shared space.
2026-06-22 20:17:18 +08:00
wizardchen 5d0d317a36 feat(datasource): reveal pre-existing selections in lazily-loaded picker
When editing a data source whose tree loads one level at a time (Feishu
wiki), a saved deep-node selection was invisible: only top-level spaces
were listed, so the selected node could not be seen and its ancestor
space rendered as unchecked.

- Add Connector.ResolveResourceAncestors: given selected resource IDs,
  return the ancestor IDs that must be expanded to reveal them. Feishu
  walks up via GetWikiNode (O(depth) per selection, no full-tree walk);
  Notion/Yuque return empty.
- Fix wikiNode.ParentNodeID JSON tag: the Feishu API field is
  parent_node_token, not parent_node_id, so GetWikiNode can return the
  parent and the ancestor walk works in production.
- Expose POST /datasource/{id}/resource-ancestors (service + handler +
  route) and a matching frontend API client.
- Edit-mode picker reveals hidden selections by expanding and lazily
  loading each ancestor level, so selections are visible and checked.
- Gate child fetching on treeFullyLoaded instead of a partial-children
  heuristic, avoiding a redundant request for full-tree connectors.
2026-06-22 19:24:08 +08:00
wizardchen d8f13ada4a fix(datasource): lazy-load Feishu wiki resources to avoid listing timeout (#1672)
Listing resources for a Feishu data source eagerly walked the entire wiki
tree (one API call per node across every space), which exceeded the request
timeout for large knowledge bases. ListResources now takes a parentID and
returns a single level at a time: empty lists the wiki spaces, a space/node
ID lists only its direct children. The frontend fetches children on demand
when a node is expanded. Notion (full tree in one call) and Yuque (flat)
keep their existing behavior by ignoring non-empty parentID.

Refs: https://github.com/Tencent/WeKnora/issues/1672
2026-06-22 17:53:30 +08:00
wizardchen 74c845a1f4 fix(docparser): normalize PaddleOCR-VL HTML tables (#1725)
PaddleOCR-VL renders tables as HTML with per-cell text-align styles.
These waste tokens on layout markup and bypass the chunker's
table-protection logic (which only recognizes Markdown pipe tables),
causing large tables to be split mid-row.

Convert inline HTML <table> blocks to GFM Markdown tables when possible,
and strip presentational attributes (style/class/align/...) for tables
using rowspan/colspan that Markdown cannot express.

Refs: https://github.com/Tencent/WeKnora/issues/1725
2026-06-22 17:31:26 +08:00
wizardchen abf6a99b90 fix(api): add POST for hybrid-search while keeping GET compatibility (#1727)
GET with a JSON body was non-standard and confused integrators. Register
POST as the preferred method and keep GET for existing clients; update SDK,
MCP server, and API docs accordingly.

Refs: https://github.com/Tencent/WeKnora/issues/1727
2026-06-22 17:22:35 +08:00
wizardchen 632cd55018 feat(im): integrate IM service for custom agent deletion
- Add IM service dependency to CustomAgentHandler for managing agent-related IM channels.
- Implement DeleteChannelsByAgent method to stop and soft-delete IM channels associated with a custom agent upon its removal.
- Update ListChannelsByTenant to exclude soft-deleted custom agents, ensuring consistent overview lists after agent removal.
2026-06-22 17:16:07 +08:00
wizardchen ddc5460988 fix(frontend): update UserMenu to use localized string for system management
Replace hardcoded "系统管理" text with the localized key for system management in UserMenu.vue, enhancing internationalization support.
2026-06-22 15:57:41 +08:00
wizardchen ecc27e9b0f fix(docparser): rename close variable to avoid shadowing builtin
The angle-bracket image target parser declared a local variable named
`close`, which shadows Go's built-in `close` function and is flagged by
revive's redefines-builtin-id rule, breaking `make lint`. Rename it to
`closeIdx`.
2026-06-22 15:57:41 +08:00
FFFFFFpy a6a20175e5 fix: preserve MHTML image resolution and Markdown structure (#1743)
* fix(docreader): preserve MHTML blocks and resolve titled images

* fix(docreader): harden MHTML markdown image resolution

* fix(docreader): preserve markdown hard break spaces

---------

Co-authored-by: FFFFFFpy <bushiyan@outlook.com>
2026-06-22 15:53:54 +08:00
wizardchen 5f7ed2ffe9 feat(i18n): add key descriptions for various settings in multiple languages
- Introduce detailed descriptions for authentication, SSRF protection, tenant management, and async task worker settings in English, Korean, Russian, and Chinese localization files.
- Enhance user understanding of configuration options by providing context-sensitive help directly within the settings interface.
- Update the Settings and SystemSettings components to utilize these new descriptions for improved user experience.
2026-06-22 15:21:50 +08:00
wizardchen 01819b5bc2 fix(im): prevent DingTalk stream connection from going stale
DingTalk websocket (Stream) channels stopped delivering @-mentions after
a day or two and only recovered when the IM channel was toggled off/on.
The factory fired the SDK's Start once and relied entirely on the SDK's
internal auto-reconnect, with no liveness supervision on WeKnora's side.
On long-running connections the SDK can enter a zombie state where the
connection object stays alive but no longer receives messages, leaving no
way to recover automatically. The cancel func was also ineffective because
the SDK ignores the context after the initial dial, so stopping a channel
leaked the old connection.

Add a reusable RunSupervised helper that keeps a long connection alive and
proactively recycles it every 6h on top of the SDK's own reconnect, which
bounds the worst-case outage and clears stuck connections. Wire DingTalk to
it, add LongConnClient.Close (disables AutoReconnect before closing so the
SDK does not resurrect the connection), and make the cancel func actually
tear the connection down. Drop the now-unused stream client reference from
the adapter, which only sends replies via sessionWebhook or OpenAPI.
2026-06-22 15:04:23 +08:00
wizardchen 10e63b2227 feat(frontend): add MCP config section and unify modal navigation styles
- Add dedicated MCP service configuration section in AgentEditorModal (agent mode only)
- Unify sidebar nav hover/active states across settings and editor modals
- Refine tool card and tag chip colors to use design tokens and color-mix
- Tighten suggested questions button padding and border-radius
2026-06-22 14:33:54 +08:00
wizardchen 8ada130ee1 fix(frontend): enhance repairFlankingEmphasis to handle opening delimiters correctly
- Update regex patterns to account for opening delimiters preceded by letters/numbers and followed by punctuation, ensuring proper rendering of emphasis in markdown.
- Add tests to validate new behavior for various edge cases, including punctuation-adjacent emphasis and exponent markers.
- Improve overall markdown rendering consistency in chat components.
2026-06-22 12:31:34 +08:00