Commit Graph

494 Commits

Author SHA1 Message Date
ChenRussell bfa78cc933 fix(rerank): filter empty passages to prevent Aliyun API error
Aliyun rerank API rejects empty documents with error "document index:X
should not be empty". This fix validates passage content before building
the documents array, skipping empty chunks and logging them for data
quality monitoring.
2026-03-25 21:19:40 +08:00
nullkey 1bf6603271 feat(im): add AI card streaming for DingTalk and improve IM platform robustness
- Add DingTalk AI Card streaming support with create/deliver/update via OpenAPI
- Extract shared think-block rendering into reusable im.TransformThinkBlocks
- Add stream orphan reaper and edit throttling for DingTalk and Telegram
- Apply think-block transform on all reply paths (DingTalk, Telegram, Feishu)
- Add Card Template ID config to frontend IM channel panel with i18n
2026-03-24 22:12:08 +08:00
nullkey a1a384ec67 feat(im): add DingTalk platform integration
Add DingTalk bot adapter supporting both webhook (HmacSHA256 signature
verification) and Stream mode (via official dingtalk-stream-sdk-go),
with sessionWebhook/OpenAPI reply fallback and AccessToken caching.
2026-03-24 22:12:08 +08:00
nullkey aea0a97be3 feat(im): add Telegram platform integration
Add Telegram Bot API adapter supporting both webhook and long-polling
modes with streaming replies via editMessageText, file download via
getFile API, and timing-safe secret token verification.
2026-03-24 22:12:08 +08:00
Alex-wuhu f0c6d0752a feat(provider): add Novita AI as LLM provider
Add Novita AI as a new model provider supporting chat, embedding,
and VLLM model types via OpenAI-compatible API endpoint.

- Add NovitaProvider implementation with OpenAI-compatible API
- Register ProviderNovita in provider registry
- Add ModelSourceNovita constant
- Add Novita to frontend fallback provider options
- Add i18n translations for en-US, zh-CN, ru-RU, ko-KR
2026-03-24 22:10:42 +08:00
Andy Yang 62e6ae960f fix(chat): support reasoning content in streaming responses for vLLM models
- Parse reasoning field together with standard stream response using local struct
- Unify reasoning content extraction from standard and vLLM response paths
- Adapt processStreamDelta to accept reasoning content as a separate parameter
- Send reasoning content as thinking response in the stream when available
- Ensure compatibility with existing SDK logic by constructing standard choice object
2026-03-24 11:56:55 +08:00
lyingbug da9f7b43b6 fix: use service name for MCP tool naming with collision protection (#715)
- Use service.Name instead of service.ID (UUID) for stable tool names
  across MCP server reconnections
- Add (tenant_id, name) unique index on MCPService to prevent name
  collisions at the DB level
- Add collision detection logging in RegisterMCPTools for defense-in-depth
- Improve truncation logic to prioritize keeping the tool name intact

Security: first-wins policy in ToolRegistry + unique name constraint
prevents tool hijacking (GHSA-67q9-58vj-32qx).
2026-03-21 10:22:05 +08:00
lyingbug e3ab971430 fix: use service name instead of UUID for MCP tool naming (#715)
When an MCP server re-registers after long idle, a new UUID is generated,
causing tool names to change and agent tool calls to fail. Switch from
t.service.ID to t.service.Name so tool names remain stable across
re-registrations.
2026-03-21 10:22:05 +08:00
Dounx e11bd8c57d fix: use milvus result set length for empty search results 2026-03-20 18:22:40 +08:00
Andy Yang 5c31a9123f fix: Adjust temperature validation to allow zero as a valid setting and default only for negative values. 2026-03-20 18:21:39 +08:00
Dounx a4d3d69843 feat: expose builtin parser engine in settings 2026-03-20 15:52:33 +08:00
Dounx a9c2d61695 fix: add OpenRouter embedding provider support 2026-03-20 15:50:42 +08:00
nullkey f01dcd4129 feat(im): add multi-instance support with Redis-based distributed coordination
- config: add IMConfig for tunable workers, queue size, rate limit params
- qaqueue: add Redis global per-user limits and global concurrency gate
- ratelimit: add distributedLimiter with Redis ZSET sliding window + local fallback
- service: add Redis message dedup, WebSocket leader election, cross-instance /stop
2026-03-20 15:49:59 +08:00
nullkey fff9efb4f5 feat(im): integrate command dispatch, QA queue, rate limiting, and streaming improvements
- Wire slash-command registry into Service and HandleMessage dispatch
- Add bounded QA worker pool (qaQueue) with per-user backpressure
- Add sliding-window rate limiter to prevent message abuse
- Support /stop cancellation for queued and in-flight QA requests
- Fix WeCom group chat replies via appchat API with user fallback
- Fix WeCom longconn nil map panic and stream buffer memory leak
- Add Feishu stream reaper to clean orphaned streams (memory leak fix)
- Show "正在思考..." placeholder during Feishu streaming
- Handle empty-stream fallback when no visible content is produced
- Derive WebSearchEnabled from agent config for IM channels
2026-03-20 15:49:59 +08:00
nullkey f05f9b0e11 feat(im): add slash-command system with /help, /info, /search, /stop, /clear
Introduce a pluggable command framework for IM channels. Includes
Command interface, CommandRegistry for registration and dispatch,
and five built-in commands: /help (list commands), /info (inspect
knowledge bases), /search (search knowledge), /stop (cancel running
QA), and /clear (reset conversation context).
2026-03-20 15:49:59 +08:00
wizardchen be4f040aa6 feat(types): enhance FAQ metadata normalization and sanitization 2026-03-19 18:20:09 +08:00
wizardchen 8f1237d64f feat(agent): enhance event emission for final answers and fallback handling
- Updated the AgentEngine to emit final answer events even when the LLM stops without calling the final_answer tool, ensuring the frontend can render responses correctly.
- Added fallback logic in the AgentStreamHandler to emit final answers as fallback events when no answer events are streamed, improving robustness in edge cases.
2026-03-19 18:20:09 +08:00
wizardchen c25741673c feat(chunk): implement batch deletion for chunks to avoid MySQL placeholder limit
- Updated DeleteChunks method to split IDs into batches of 5000 to prevent MySQL Error 1390.
- Added unit tests for CalculateFAQContentHash to ensure consistent hashing behavior across various scenarios, including normalization and case sensitivity.
2026-03-19 18:20:09 +08:00
wizardchen 8db4b6cdea fix(chunk): update timestamp function in UpdateChunks method to use NOW() for consistency 2026-03-19 18:20:09 +08:00
wizardchen 4c3f083642 refactor(agent): streamline document info retrieval and enhance chunk search logic
- Removed the validation for a maximum of 10 knowledge IDs in GetDocumentInfoTool to allow more flexibility.
- Adjusted pagination settings in chunk retrieval to limit results to 1 per request for more granular control.
- Updated the searchChunks method to use case-insensitive matching based on the database type, improving compatibility across different systems.
- Enhanced query structure for better readability and maintainability.
2026-03-19 18:20:09 +08:00
wizardchen 2a63b58deb feat(im): enhance message handling and formatting for Feishu compatibility
- Updated the message streaming logic to handle <think> blocks more effectively, ensuring proper formatting for Feishu.
- Introduced a new transformThinkBlocks function to convert <think> content into Feishu-compatible markdown blockquotes.
- Improved the handling of tool call events to prevent duplicate processing and ensure correct message formatting.
- Enhanced documentation for permission configuration in the IM channel setup.
2026-03-18 21:56:34 +08:00
wizardchen 3de78dc3e8 refactor(im): improve tool invocation and result formatting
- Updated formatToolCallStart and formatToolCallResult functions to return plain-text lines instead of Markdown snippets for better integration within <think> blocks.
- Introduced briefToolSummary function to extract concise summaries from tool output.
- Enhanced handleMessageStream to manage <think> blocks more effectively during agent interactions.
- Added handling for server-side events in WeCom long connection management to improve reconnection logic.
2026-03-18 21:56:34 +08:00
wizardchen f3a3c72a4f fix(docparser): handle nested linked images and url parens 2026-03-18 21:56:34 +08:00
wizardchen 3450ed5e41 feat(knowledge): offload manual processing to async tasks 2026-03-18 21:56:34 +08:00
joeyczheng 49b5f1a739 feat(IM): support both websocket/webhook mode im channel for slack
Signed-off-by: joeyczheng <joeyczheng@tencent.com>
2026-03-18 21:26:53 +08:00
wizardchen 4e2583e58d feat: add language support to summary generation payload
- Enhanced the SummaryGenerationPayload struct to include a Language field for better localization.
- Updated enqueueSummaryGenerationTask and ProcessSummaryGeneration methods to handle language context in summary generation tasks.
2026-03-17 22:23:24 +08:00
wizardchen 17da967f92 refactor: update session title generation and language context handling
- Modified the session title generation template to focus on extracting only the intent of the user's question.
- Enhanced the session service to include language context when generating titles.
- Updated middleware to inject language into the request context.
- Added a new test suite for validating SSRF safety of URLs.
2026-03-17 22:23:24 +08:00
wizardchen f7750d65c1 fix: update default language setting to Chinese 2026-03-17 22:23:24 +08:00
wizardchen 2ad1faf85f chore: update fallback templates
- Added critical language rules to fallback response templates to ensure consistent language usage in responses.
- Updated logger context to include language and session-related keys for improved context management.
2026-03-17 22:23:24 +08:00
wizardchen 139a9c40ff feat: enhance localization and configuration support
- Added timezone and language settings to the environment configuration.
- Introduced built-in agent configurations with multilingual support for various agents.
- Updated Docker Compose to utilize new environment variables for timezone and language.
- Created new prompt templates for question generation, summary generation, and keywords extraction.
2026-03-17 22:23:24 +08:00
wizardchen 09d9fe5ddb feat(im): add file support and knowledge base integration 2026-03-17 22:23:24 +08:00
ChenRussell d89e5db0f6 git commit -m "fix: clarify MinIO endpoint format error message
Add specific error handling for 'Endpoint url cannot have fully qualified paths'
to guide users to remove http:// or https:// prefix from endpoint URL."
2026-03-16 21:50:07 +08:00
wizardchen 452db9f5c3 feat: add unit tests for InjectAndConditions function 2026-03-16 19:21:35 +08:00
wizardchen 2fd6e226a5 fix: sanitize tab chars and avoid double .md extension in manual download filename
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 18:18:23 +08:00
wizardchen 66c384823b test: add unit tests for manual knowledge download filename logic
Table-driven tests for sanitizeManualDownloadFilename covering normal titles,
each dangerous character replacement (/  \  "  \n  \r), blank/whitespace-only
titles falling back to "untitled.md", and safe passthrough of = ; characters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 18:01:16 +08:00
wizardchen e0c1b77b70 fix: quote Content-Disposition filename and add empty-title fallback
- Use mime.FormatMediaType for both attachment and inline Content-Disposition
  headers in DownloadKnowledgeFile and the preview handler, preventing header
  injection via a user-controlled knowledge title (e.g. containing ';' or '=')
- Extract inline sanitize logic into sanitizeManualDownloadFilename helper and
  add a blank/whitespace-only title fallback to "untitled" so the download
  never produces a degenerate ".md" filename

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 18:01:07 +08:00
wizardchen 63caf564ee feat: support manual knowledge download in GetKnowledgeFile
Add an early-return branch that streams manual knowledge content
(stored as Markdown in Metadata.content) directly as a .md file,
avoiding the runtime error that occurred when FilePath is empty.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 17:49:45 +08:00
wizardchen 160672ea9c feat: enhance message structure to support multi-content messages
- Introduced MessageContentPart and ImageURL types to allow for multi-content messages that can include both text and images.
- Updated the Message struct to include a MultiContent field for handling multiple message parts.
- Modified the ConvertMessages function to prioritize processing of multi-content messages, improving the handling of user messages with images.

These changes enhance the flexibility and functionality of chat messages, enabling richer content delivery.
2026-03-16 17:33:50 +08:00
wizardchen 6a7eb420a9 refactor: update conversation handling and image description requirements
- Removed outdated instructions regarding the `skip_kb_search` field in the conversation configuration.
- Clarified requirements for `image_description` to ensure it is non-empty when images are present.
- Adjusted the handling of message parts in the chat pipeline to improve image processing.
- Deleted unused migration files related to image handling in the database schema.

These changes enhance the clarity and functionality of the conversation and image processing logic.
2026-03-16 15:15:24 +08:00
wizardchen b6cbc412ba Merge remote-tracking branch 'wizard/main' 2026-03-16 11:13:53 +08:00
wizardchen 8b1febbe99 feat: implement IM channel sessions and update message handling 2026-03-16 02:40:42 +08:00
wizardchen 6159e8e3f7 feat: implement IM channel management in agent editor
- Introduced a new IMChannelPanel component for managing WeCom and Feishu channels.
- Added CRUD operations for IM channels, including create, update, delete, and list functionalities.
- Enhanced the backend with new API endpoints for IM channel management.
- Updated documentation to reflect changes in IM integration and channel management.
- Improved localization support for new IM-related UI elements across multiple languages.
2026-03-16 02:28:10 +08:00
wizardchen c27740f739 feat: enhance input field and agent stream display with improved handling and sanitization
- Added blur handling for the textarea in the Input-field component to prevent errors when navigating away from the component.
- Improved the sanitization process in the AgentStreamDisplay component to preserve specific tags during markdown processing, ensuring accurate content display.
- Introduced new methods for handling FAQ-specific post-processing in the knowledge base service, optimizing retrieval and filtering of search results.

These changes enhance user experience by improving input handling and content rendering in the chat interface.
2026-03-16 02:26:47 +08:00
wizardchen 2984b21731 refactor: consolidate QA request handling and improve session service interface
- Introduced a new `QARequest` type to streamline parameters for KnowledgeQA and AgentQA service calls, replacing multiple parameters with a single structured request.
- Updated the `KnowledgeQA` and `AgentQA` methods in the session service interface to accept the new `QARequest` type, enhancing clarity and maintainability.
- Refactored the request parsing logic to utilize the new `buildQARequest` method, improving code organization and reducing redundancy.
- Implemented a new `resolveAgent` method to simplify custom agent resolution logic, enhancing readability and maintainability.

These changes improve the overall structure and usability of the QA request handling, facilitating easier updates and better code organization.
2026-03-16 02:26:47 +08:00
wizardchen 6896c7795d feat: add web search option to AgentQA functionality
- Introduced a new parameter `webSearchEnabled` to the `AgentQA` method, allowing for conditional web search capabilities.
- Updated the session service interface and handler to accommodate the new parameter, enhancing the flexibility of the AgentQA process.
- Adjusted the configuration to ensure that web search functionality can be toggled based on the provided input.

These changes improve the AgentQA feature by enabling users to control web search integration, enhancing the overall user experience.
2026-03-16 02:26:47 +08:00
wizardchen 9face5920b feat: enhance knowledge search by resolving embedding model keys
- Introduced a new method to resolve embedding model IDs to their actual identity keys (name + endpoint), enabling optimal grouping of knowledge bases across different tenants.
- Updated the concurrent search functionality to utilize the resolved model keys, improving the efficiency of query embedding computations.
- Refactored search target grouping to reduce redundant API calls and enhance performance during hybrid searches.

These changes significantly improve the knowledge search process by ensuring that embeddings are computed efficiently and reused across multiple knowledge bases sharing the same underlying model.
2026-03-16 02:26:47 +08:00
wizardchen fc92b494b0 feat: optimize hybrid search by grouping targets and reusing query embeddings
- Enhanced the concurrent search functionality to group search targets by their embedding model, allowing for a single computation of query embeddings per model.
- Implemented batch fetching of knowledge base records to improve efficiency and reduce redundant API calls.
- Updated the HybridSearch method to support multiple knowledge base IDs, enabling combined retrieval for full-KB targets and improving overall search performance.
- Added a new GetQueryEmbedding method to compute embeddings for queries, facilitating reuse across multiple knowledge bases sharing the same model.

These changes significantly enhance the performance and efficiency of the knowledge search process, reducing API calls and improving response times.
2026-03-16 02:26:47 +08:00
wizardchen 6abc688ea2 refactor: update LLM configuration to model ID in API and frontend
- Replaced LLMConfig with model_id in TextRelationExtractionRequest and FabriTextRequest interfaces.
- Updated related API handlers to retrieve chat models based on model_id.
- Adjusted frontend components to pass model_id instead of LLMConfig, enhancing clarity and simplifying the configuration process.

These changes streamline the model configuration process across the application, improving maintainability and usability.
2026-03-16 02:26:47 +08:00
wizardchen 7f8409652f fix: update grep_chunks tool to support multiple patterns
- Renamed the input parameter from "pattern" to "patterns" to reflect that multiple text patterns can be provided.
- Updated the schema definition and input structure accordingly to ensure consistency in handling multiple patterns.
- Enhanced error logging to specify "patterns" when no valid input is provided, improving clarity for debugging.

These changes improve the functionality of the grep_chunks tool by allowing users to search for multiple patterns simultaneously.
2026-03-16 02:26:47 +08:00
wizardchen de0e93d9bb feat: add clear session messages functionality
- Implemented a new API endpoint to clear all messages in a session, enhancing session management capabilities.
- Updated the frontend to include a clear messages option in the session menu, allowing users to easily remove chat history.
- Added localization support for the new feature in English, Korean, Russian, and Chinese, improving accessibility for a broader audience.
- Enhanced the chat view to handle session message clearing events, ensuring a responsive user experience.

These changes significantly improve the user experience by providing a straightforward method to manage session messages effectively.
2026-03-16 02:26:47 +08:00