Commit Graph

75 Commits

Author SHA1 Message Date
wolfkill 8f4e5a459f fix(session): scope session access by user 2026-05-14 15:56:43 +08:00
wizardchen 1f5970b67e refactor(agent): rebuild multi-turn history from DB, drop llmcontext layer
The Agent ran a parallel context cache (Redis/in-memory) on top of the
messages table to feed multi-turn history into the LLM. That dual-write
caused subtle drift (e.g. compression diverging from DB, system-prompt
swaps lost on restart) and required a separate ClearContext path on
session/IM clear.

Make the messages table the single source of truth:

- Add service.LoadAgentHistory: rebuilds chat.Message history per turn
  from the persisted messages, expanding AgentSteps into proper OpenAI
  assistant_with_tool_calls + tool messages and replaying the canonical
  final answer (with <think> blocks stripped). final_answer tool calls
  are filtered to avoid duplicating the trailing answer.
- Make AgentEngine stateless across turns: drop ContextManager / sessionID
  cache plumbing from the engine, agent service, and CreateAgentEngine
  signature. The engine only uses sessionID for logging/event emission.
- Wire AgentQA to load history from DB on demand using HistoryTurns
  (default 5) when MultiTurnEnabled, otherwise run with empty history.
- Delete the llmcontext package (ContextManager interface, Redis/memory
  storage, factory) and the SessionService.ClearContext API path; IM
  /clear and session message clear no longer need to invalidate cache.

Behavior preserved: KnowledgeQA-mode replay is unchanged (turns with
empty AgentSteps just produce the single canonical assistant message),
and Agent-mode turns now consistently see prior tool calls and results.
2026-05-12 12:26:24 +08:00
wizardchen fe0d24ae87 fix(sessions): review fixes for keyword search / pinning / IM titles
- Apply the session-list additions to the SQLite init migration (Lite build)
  and make QueryPaged dialect-aware: LOWER(..) LIKE on SQLite, ILIKE on
  Postgres, drop NULLS LAST on SQLite. Escape LIKE wildcards in the keyword
  via the existing escapeLikeKeyword helper.
- SetPinned now returns rowsAffected so the handler can respond 404 on
  unknown / unauthorized session IDs instead of a misleading 200.
- GET /sessions always returns the enriched shape (pin state + IM origin
  fields) so the frontend never needs a second roundtrip; the dual-path
  legacy branch in the handler is gone.
- Register pin routes with the wildcard name that matches each verb's
  existing radix tree (POST :session_id, DELETE :id) and accept either
  param name in the handler; avoids gin's "wildcard conflicts" panic.
- Drop the redundant [platform] prefix from IM session titles now that
  the list renders a platform icon alongside the title; add unit tests
  for shortID / buildUserSessionTitle / buildThreadSessionTitle.
- Frontend: remove the submenu search input and its i18n/keyword wiring
  (search lives elsewhere in the app); pin icon uses the TDesign `pin`
  glyph and inherits color so the active session turns green; optimistic
  pin moves the item to the top of the list so it shows up at the top
  of the Pinned group; IM text badge replaced by the platform SVG icons
  from assets/img/im, desaturated by default and full-color on hover
  or when the session is active.
2026-04-30 16:23:00 +08:00
wizardchen dbd804d6e3 feat(sessions): add keyword search, user-scoped pinning, and IM source visibility
Sessions today are a flat list ordered by updated_at. Two gaps showed up in
practice:
- Users cannot find specific chats as the list grows beyond a screen.
- IM-created sessions (WeCom/Feishu/Slack/...) are indistinguishable:
  every title was "IM-<platform>" or "IM-<platform>-<username>" and the
  list API hid the underlying im_channel_sessions mapping, so admins had
  no way to tell which Feishu group a session came from.

Backend
- Migration 000039 adds sessions.user_id (owner), is_pinned, pinned_at
  plus a composite index for the list query. Existing rows keep user_id
  NULL and stay visible at the tenant level for backward compatibility.
- CreateSession now writes the caller's user_id from auth context.
- GET /sessions accepts keyword / source / agent_id. When any filter is
  set, the response switches to enriched items that LEFT JOIN
  im_channel_sessions and expose im_platform / im_chat_id / im_thread_id
  / im_user_id / im_agent_id / im_channel_id. No filters => legacy shape,
  existing clients unaffected.
- Ordering: is_pinned DESC, pinned_at DESC NULLS LAST, updated_at DESC.
- POST/DELETE /sessions/:id/pin for user-scoped pin/unpin.
- IM session titles: "[platform] <user|chat|thread>" with short ID
  suffixes so group/DM/thread sessions are visually distinct without
  needing a round-trip to fetch a display name from the IM adapter.

Frontend
- Search input debounced at 300ms drives the keyword filter.
- Pinned chats render in a dedicated group above the time-based groups,
  with a pin icon and a pin/unpin entry in the per-chat dropdown.
- IM chats get a short [platform] badge in the list.
- Pin toggle is optimistic and guards against double-clicks.
- zh/en/ko/ru i18n keys added for the new strings.
2026-04-30 16:23:00 +08:00
wizardchen 492e92580b feat(observability): integrate Langfuse for LLM token tracking and tracing
Closes #620 #497. Add opt-in Langfuse observability covering all five
model types (chat, embedding, rerank, VLM, ASR) with HTTP-request-scoped
traces and Docker Compose support (both cloud and self-hosted).

Core package internal/tracing/langfuse:
- HTTP client with batched async ingestion (non-blocking in request path)
- Sampling, environment / release tagging, and graceful fallback when
  LANGFUSE_* env vars are absent (wrappers become no-ops)
- Gin middleware opens one trace per traced request and finishes it after
  the handler chain returns, attaching method / path / user / session
- Trace context is stored under a typed key exported from internal/types
  so logger.CloneContext can preserve it across handler / goroutine
  boundaries (otherwise each LLM call auto-created an orphan trace,
  fragmenting one request into many)

Per-model generation wrappers (opt-in via NewChat/NewEmbedder/...):
- chat: captures prompt, streaming output, token usage + TTFT
- embedding: approximates tokens when the provider omits usage
- rerank: previews query/docs, summarizes results to keep payload small
- vlm: records image count and total bytes, never uploads raw pixels
- asr: records file size and audio duration, never uploads audio bytes

Async title generation (GenerateTitleAsync) now forwards the trace key
into the goroutine so title calls appear under the parent chat trace.

Docker Compose:
- LANGFUSE_* env passthrough on the `app` service for cloud deployments
- Optional `langfuse` profile spins up a self-hosted Langfuse stack that
  reuses WeKnora's existing PostgreSQL (separate database via an idempotent
  init container that fixes ICU collation drift) and Redis (separate DB
  number), adding only ClickHouse, MinIO, web and worker containers
- web/worker entrypoints URL-encode DB_PASSWORD / REDIS_PASSWORD at start
  to avoid Prisma P1013 when passwords contain @ / # / etc.

Docs: docs/Langfuse集成.md covers cloud vs self-hosted, per-model usage
strategy, code map, and resource footprint.
2026-04-24 10:29:19 +08:00
wizardchen a5ca9a2784 feat(web-search): refactor web search provider
- Updated relevant files to include provider registration, implementation, and metadata.
- Enhanced frontend components to support provider management and configuration.
- Added localization for new provider settings and messages.
- Implemented backend repository methods for CRUD operations on web search providers.
2026-03-31 20:22:34 +08:00
wizardchen 052be53c42 feat(prompts): update assistant descriptions and add intent-specific templates
- Updated the assistant descriptions in various prompt templates to specify that they are developed by Tencent.
- Added new intent-specific system prompt templates for greeting, chitchat, follow-up, image analysis, summarization, and web search unavailability scenarios.
- Enhanced citation rules in the system prompts to ensure proper inline citation formatting.
- Refactored the rewrite prompt to classify user intent more accurately and output structured JSON responses.
2026-03-25 22:08:29 +08:00
wizardchen 17da967f92 refactor: update session title generation and language context handling
- Modified the session title generation template to focus on extracting only the intent of the user's question.
- Enhanced the session service to include language context when generating titles.
- Updated middleware to inject language into the request context.
- Added a new test suite for validating SSRF safety of URLs.
2026-03-17 22:23:24 +08:00
wizardchen 139a9c40ff feat: enhance localization and configuration support
- Added timezone and language settings to the environment configuration.
- Introduced built-in agent configurations with multilingual support for various agents.
- Updated Docker Compose to utilize new environment variables for timezone and language.
- Created new prompt templates for question generation, summary generation, and keywords extraction.
2026-03-17 22:23:24 +08:00
wizardchen c27740f739 feat: enhance input field and agent stream display with improved handling and sanitization
- Added blur handling for the textarea in the Input-field component to prevent errors when navigating away from the component.
- Improved the sanitization process in the AgentStreamDisplay component to preserve specific tags during markdown processing, ensuring accurate content display.
- Introduced new methods for handling FAQ-specific post-processing in the knowledge base service, optimizing retrieval and filtering of search results.

These changes enhance user experience by improving input handling and content rendering in the chat interface.
2026-03-16 02:26:47 +08:00
wizardchen 2984b21731 refactor: consolidate QA request handling and improve session service interface
- Introduced a new `QARequest` type to streamline parameters for KnowledgeQA and AgentQA service calls, replacing multiple parameters with a single structured request.
- Updated the `KnowledgeQA` and `AgentQA` methods in the session service interface to accept the new `QARequest` type, enhancing clarity and maintainability.
- Refactored the request parsing logic to utilize the new `buildQARequest` method, improving code organization and reducing redundancy.
- Implemented a new `resolveAgent` method to simplify custom agent resolution logic, enhancing readability and maintainability.

These changes improve the overall structure and usability of the QA request handling, facilitating easier updates and better code organization.
2026-03-16 02:26:47 +08:00
wizardchen 6896c7795d feat: add web search option to AgentQA functionality
- Introduced a new parameter `webSearchEnabled` to the `AgentQA` method, allowing for conditional web search capabilities.
- Updated the session service interface and handler to accommodate the new parameter, enhancing the flexibility of the AgentQA process.
- Adjusted the configuration to ensure that web search functionality can be toggled based on the provided input.

These changes improve the AgentQA feature by enabling users to control web search integration, enhancing the overall user experience.
2026-03-16 02:26:47 +08:00
wizardchen e4bb97ff67 feat: enhance chat model selection logic in session management
- Refactored the chat model ID selection process to prioritize the request's summaryModelID, followed by the custom agent's ModelID, and finally a fallback to the default model selection.
- Improved logging to provide clearer insights into which model ID is being used during the selection process.
- Removed the deprecated selectChatModelIDWithOverride function to streamline the codebase and enhance maintainability.

These changes improve the flexibility and clarity of chat model selection, ensuring a more robust session management experience.
2026-03-16 02:26:47 +08:00
wizardchen 421c83452c feat: enhance multimodal image processing and session management
- Updated configuration to support three tasks in the assistant's prompt: rewriting questions, classifying intent for knowledge base retrieval, and analyzing attached images.
- Implemented backend functionality to update image descriptions in user messages, ensuring that image context is preserved across conversation history.
- Modified session handling to return complete session data after updates, improving the accuracy of session information provided to users.
- Enhanced the chat pipeline to include detailed image descriptions and manage image captions effectively, enriching user interactions with visual content.

These improvements significantly enhance the application's multimodal capabilities, providing a more robust and user-friendly experience when handling image-based queries.
2026-03-16 02:26:47 +08:00
wizardchen 991bd29a14 feat: implement image upload and multimodal support
- Added functionality for image uploads in chat, allowing users to attach images for multimodal Q&A.
- Enhanced the input field to handle image selection via drag-and-drop and paste, with validation for file types and sizes.
- Updated the backend to process images alongside text queries, including support for image analysis.
- Introduced new UI components for image previews and management, improving user interaction with uploaded content.
- Added localization strings for image upload features and error messages, enhancing accessibility for users in multiple languages.

These changes significantly improve the chat experience by enabling users to incorporate images into their queries, facilitating richer interactions and responses.
2026-03-16 02:26:47 +08:00
wizardchen 1fb2bedd2b feat: implement chat history and retrieval settings
- Added new API endpoints for managing chat history configuration and retrieval settings, allowing tenants to enable message indexing and configure search parameters.
- Introduced new Vue components for ChatHistorySettings and RetrievalSettings, providing a user-friendly interface for managing these configurations.
- Updated localization files to include new settings descriptions and labels in multiple languages.
- Enhanced the KnowledgeSearch view to support searching across both knowledge bases and chat history, improving the overall search functionality.

These changes significantly enhance the application's capabilities in managing and retrieving chat history, contributing to a more robust user experience.
2026-03-09 11:46:56 +08:00
wizardchen 44f38ca6c3 feat: add fallback response handling and UI indicators
- Introduced a new `is_fallback` property to track when responses are generated without knowledge base matches.
- Updated the chat components to display fallback hints and indicators in the UI, enhancing user awareness of fallback responses.
- Added translations for fallback hints in multiple languages, improving accessibility for diverse users.
- Modified backend logic to emit fallback status in response events, ensuring consistent handling across the application.

These changes improve the user experience by providing clear feedback on fallback responses, enhancing the overall interaction with the chat system.
2026-03-05 18:14:14 +08:00
wizardchen 284d6c623e feat: add deleteAllSessions functionality and update batch delete logic
- Introduced a new API function `deleteAllSessions` to allow deletion of all sessions for a tenant.
- Updated the `BatchManageDialog` component to support a confirmation dialog for deleting all sessions.
- Enhanced the menu component to reflect the new delete all functionality with appropriate confirmation messages.
- Added internationalization support for the new delete all confirmation message in multiple languages.

These changes improve session management by providing users with the ability to delete all sessions at once, enhancing the overall user experience.
2026-03-02 21:21:49 +08:00
wizardchen 4a9977b11f feat: enhance document loading and state management in doc-content component
- Introduced a loading state (`loadingChunks`) to manage document chunk loading more effectively.
- Updated the scroll handling logic to prevent multiple requests while loading chunks.
- Adjusted the pagination logic to ensure correct page increments based on the total document count.

This update improves the user experience by ensuring smoother document loading and better state management during scrolling interactions.
2026-03-02 21:21:49 +08:00
wizardchen 469f320d10 feat: enhance storage engine configuration and UI updates
- Updated `.air.toml` to include additional CGO flags for improved build settings.
- Expanded `StorageEngineConfig` interface to support "tos" (火山引擎 TOS) as a new storage provider.
- Modified related components and views to accommodate the new storage engine, including updates to `ListSpaceSidebar`, `AgentList`, `KnowledgeBaseList`, and `OrganizationList`.
- Improved UI elements for batch management in the menu and sidebar components.
- Added internationalization support for new memory features and storage engine descriptions across multiple languages.

This update enhances the flexibility of storage options and improves user experience with better UI interactions.
2026-03-02 21:21:49 +08:00
joeyczheng 12baa885d2 feat: implement preview of user level memory graph
Signed-off-by: joeyczheng <joeyczheng@tencent.com>
2026-02-25 14:29:54 +08:00
AndyYang 172bf797ee feat(chat): add batch management for conversations 2026-02-25 14:23:05 +08:00
wizardchen ab1f15c97d feat(input-field): enhance knowledge base selection and UI components
- Updated the logic for cleaning invalid knowledge base IDs to include checks for shared agent knowledge bases, ensuring accurate selection retention.
- Refactored the mention chip component styles and structure for improved clarity and usability, including updated class names and enhanced visual elements.
- Introduced a new context key for session tenant ID to streamline tenant context handling in message and session services, improving the accuracy of operations in shared agent scenarios.

This update enhances the user experience by ensuring correct knowledge base selections and improving the visual representation of selected items.
2026-02-09 11:40:25 +08:00
wizardchen 03005d71fa refactor(session): streamline tenant context handling for async operations
- Updated session title generation and message completion to utilize the session's tenant ID directly, ensuring correct context is used in background operations.
- Removed unnecessary context value extraction and background context creation, simplifying the code and improving clarity.
- Enhanced the handling of stop events and message updates to consistently apply the session's tenant ID, improving the accuracy of operations in shared agent scenarios.

This update improves the reliability of async operations by ensuring the correct tenant context is consistently applied.
2026-02-09 11:40:25 +08:00
wizardchen dfaf862bc0 feat(agent): enhance agent sharing and selection features
- Updated the AgentSelector component to improve agent selection logic, ensuring accurate representation of selected agents.
- Introduced new AgentShareSettings component for managing agent sharing with organizations, including UI for selecting organizations and displaying shared agents.
- Enhanced API interactions for sharing agents, including new methods for listing and managing agent shares.
- Improved localization support for agent sharing features across multiple languages.
- Updated various components to reflect changes in agent capabilities and sharing status, enhancing user experience and clarity.

This update strengthens the agent management features, providing users with better tools for sharing and selecting agents within organizations.
2026-02-09 11:40:25 +08:00
wizardchen 4158773849 feat: add organization management features
- Introduced a new API for organization management, including creation, updating, and deletion of organizations.
- Implemented organization sharing functionality for knowledge bases, allowing users to share knowledge bases with specific organizations and set permissions.
- Added UI components for organization selection and sharing dialogs.
- Enhanced existing components to support organization-related features, including shared knowledge base handling and organization role management.

This update enhances collaboration capabilities within the application by allowing users to manage and share knowledge bases across different organizations.
2026-02-09 11:40:25 +08:00
wizardchen 2d6efec84f feat: enhance agent skills sandbox configuration and availability
- Updated .env.example to set default sandbox mode to 'docker' and added timeout and docker image variables.
- Modified docker-compose files to include a sandbox service for building and pulling the sandbox image.
- Adjusted frontend API to reflect sandbox availability for skills, ensuring UI elements are conditionally displayed based on sandbox status.
- Implemented backend logic to disable skills when the sandbox is not enabled, improving error handling and user experience.
2026-02-04 20:58:24 +08:00
nullkey 03446f35d3 feat: adjust sandbox setting of agentConfig 2026-02-04 20:08:49 +08:00
nullkey 43a2c647ea feat: optimize security and deployment of agent skills 2026-02-04 20:08:49 +08:00
nullkey 3f21345546 feat: adapt preloaded skills to smart-reasoning agent 2026-02-04 20:08:49 +08:00
wizardchen 7d272cc92c feat: add retrieve KB only when mentioned option 2026-01-26 10:44:09 +08:00
nullkey de06bb24d3 fix: cleanup conversation context stored in Redis 2026-01-23 10:33:01 +08:00
wizardchen aa4af5365e feat: add thinking mode support for agents 2026-01-20 18:19:44 +08:00
wizardchen 884153aeb8 feat: 增强模型选择和错误处理机制
优化模型ID传递逻辑,支持请求级别的模型覆盖;完善流式响应的错误处理
2025-12-31 15:36:20 +08:00
wizardchen 9beca5b688 feat: 添加Summary配置的Thinking字段 2025-12-31 15:36:20 +08:00
begoniezhao 907e9a5522 feat: Add DataSchema tool for retrieving schema information from CSV and Excel files 2025-12-29 20:03:51 +08:00
wizardchen 6c8e261a5f feat: 支持知识库选择模式和标题生成模型配置
- 移除对rerankModel的强制检查,支持无知识库场景
- 优化知识库解析逻辑,优先处理用户明确指定的知识库
2025-12-29 14:35:13 +08:00
wizardchen 6bc19ab082 feat: 统一系统提示词并优化智能体选择器界面
- 将系统提示词从分离的网络搜索开/关状态合并为统一提示词,使用 {{web_search_status}} 占位符动态控制行为
- 重构智能体选择器,从简单tooltip升级为内容丰富的popup,显示智能体详细信息、类型和能力
- 新增FAQ优先策略配置,支持FAQ答案优先于普通文档
- 优化知识库选择逻辑,智能体切换时自动同步配置的知识库
- 更新相关接口类型定义和国际化文本
2025-12-28 21:46:42 +08:00
wizardchen 0ca947981a refactor: 调整智能体配置,增强智能体选择与状态管理 2025-12-26 21:31:02 +08:00
wizardchen abfe87eaed feat: 重构提示词模板系统,新增多场景模板配置支持;优化智能体知识库选择模式,支持全部/指定/禁用三种模式 2025-12-25 18:22:36 +08:00
wizardchen dc1ab5b81d Merge branch 'master' into custom_agent 2025-12-24 15:53:04 +08:00
wizardchen e45b82e9be feat: 增强知识库搜索功能,支持多知识库和指定文件ID搜索;优化SearchKnowledgeRequest结构,向后兼容单知识库ID参数 2025-12-24 15:46:16 +08:00
wizardchen d0c5c4f6b0 feat: 新增智能体功能模块,支持自定义智能体创建、配置和选择 2025-12-24 10:34:41 +08:00
wizardchen 3c83cf9873 feat: 增强知识库和搜索功能的逻辑 2025-12-22 13:11:31 +08:00
wizardchen de1ffa7f7c feat: 支持输入框内选择知识库和文件,优化选择交互体验 2025-12-22 13:11:31 +08:00
wizardchen 6b4d17ec70 feat: Implement parallel search functionality combining chunk and entity searches, enhancing retrieval efficiency and result accuracy 2025-12-03 17:45:21 +08:00
wizardchen 93af4460d6 refactor: Remove agent_test command-line tool and associated files to streamline project structure and eliminate unused components 2025-12-01 17:24:57 +08:00
wizardchen eb8bfddfd1 refactor: Update function signatures across multiple files to improve readability by adding context parameters and enhancing code structure 2025-11-28 16:17:52 +08:00
wizardchen 135e33eca4 refactor: Clean up code by removing unnecessary blank lines and improving comments for better readability and maintainability 2025-11-28 16:10:37 +08:00
wizardchen 85caee044a refactor: Sanitize log inputs in agent and session handlers to enhance security 2025-11-26 19:08:19 +08:00