- Updated Elasticsearch repository methods to use ID fields without the ".keyword" suffix, aligning with the new index mapping requirements.
- Enhanced documentation for the VectorEmbedding struct to clarify expected index mapping and usage guidelines for ID fields.
- Ensured consistency across all repository methods that interact with ID fields, improving query performance and maintainability.
- Introduced a new `channel` field in the Knowledge struct and associated request types to track the source channel (e.g., "web", "api", "browser_extension").
- Updated various frontend components to display channel information and enhance user experience with channel labels.
- Enhanced localization files to support channel labels in English and Chinese.
- Modified backend services and database migrations to accommodate the new channel feature, ensuring consistent tracking across knowledge entries.
- Refactored related functions to integrate channel handling, improving overall knowledge management and context.
- Improved the message consolidation logic to preserve the current turn's user query and all subsequent assistant/tool messages, ensuring better context retention.
- Updated the CompressContext function to reflect the new consolidation strategy, maintaining the system prompt and relevant recent messages.
- Refactored the context manager to support optional message repository for improved context rebuilding from persistent storage.
- Added comprehensive tests to validate the new consolidation behavior and ensure correct message handling across various scenarios.
- Introduced support for JSON file types in English, Korean, Russian, and Chinese locale files.
- Updated the file type acceptance criteria in the document upload component to include JSON.
- Enhanced the knowledge base parser settings to recognize JSON as a valid file type.
- Implemented JSON conversion functionality in the document parser, allowing for JSON content to be processed and converted to markdown.
- Added comprehensive tests for JSON to markdown conversion to ensure functionality and reliability.
- Updated regex patterns for table separator and row matching to use [ \t] instead of \s, preventing newline consumption across rows.
- Enhanced comments for clarity on the changes made to the regex expressions.
- Updated the merging process in the chat pipeline to include a new step for re-merging overlapping ranges introduced by context expansion.
- Implemented a new function, removePartialOverlaps, to eliminate chunks that are largely contained within higher-scored chunks, improving deduplication accuracy.
- Enhanced normalization and overlap checking utilities to support the new deduplication logic.
- Updated the page size from 100 to 1000 in the buildTagMap function to enhance performance and efficiency when retrieving tags from the knowledge base.
- Changed the tag retrieval process to support pagination, allowing for more efficient handling of large sets of tags.
- Adjusted the page size to 100 for better performance and reduced memory usage.
- Ensured that the function continues fetching tags until all are retrieved, improving the completeness of the tag map.
- Improved the CreateAgentEngine method by breaking down the process into distinct steps for better readability and maintainability.
- Added a new registerMCPTools method to streamline the registration of MCP tools based on the agent configuration.
- Enhanced error handling and logging for MCP service registration, ensuring clearer feedback during the process.
- Introduced resolveKBAndDocInfos to load knowledge base metadata and selected document information for prompt generation.
- Added a confirmation dialog for rebuilding knowledge items in the KnowledgeBase component.
- Introduced new state variables to manage the rebuild process and item selection.
- Enhanced the processChunks function to support question generation options based on configuration.
- Updated the knowledge service to handle question generation settings during document processing.
- Added support for SO_REUSEPORT in the listenWithRetry function to improve port binding during hot-reloads.
- Implemented graceful shutdown by closing the listener immediately upon receiving a shutdown signal, allowing for quicker port release.
- Updated logging to provide clearer feedback during server shutdown and error handling.
- Updated the assistant descriptions in various prompt templates to specify that they are developed by Tencent.
- Added new intent-specific system prompt templates for greeting, chitchat, follow-up, image analysis, summarization, and web search unavailability scenarios.
- Enhanced citation rules in the system prompts to ensure proper inline citation formatting.
- Refactored the rewrite prompt to classify user intent more accurately and output structured JSON responses.
- Added functionality to resolve embedded data:image;base64 images in markdown, storing them and replacing references with stable URLs.
- Updated the knowledge service to process data URIs before handling remote images, ensuring all images are resolved consistently.
- Improved logging for resolved images, providing better insights during manual processing.
- Added unit tests to validate the new data URI resolution functionality and ensure reliability.
- Implemented a new method in the chunk repository to find FAQ chunks with overlapping standard or similar questions.
- Updated the knowledge service to utilize this method for checking duplicates during FAQ question creation.
- Improved error handling for duplicate questions, providing clearer feedback on conflicts.
- Added a new method to render graph extraction prompts with shared placeholders for language.
- Updated entity and relationship extraction methods to utilize the new rendering function, improving prompt customization.
- Enhanced question generation task to include language context, ensuring prompts are generated in the correct language.
- Improved error handling for empty prompt configurations in question generation, enhancing robustness.
- Removed the query display section from the DatabaseQuery component to streamline the UI.
- Updated the DatabaseQueryData interface to eliminate the query field, reflecting changes in tool result handling.
- Enhanced the DatabaseQuery tool execution to focus on returning structured results without exposing raw SQL queries.
- Added a new utility to strip <think> blocks from LLM outputs, improving content clarity and user experience.
- Introduced a memory consolidator to summarize older messages when the context window exceeds a specified limit, preserving key facts and tool results.
- Enhanced the AgentEngine to utilize the new token estimator and memory consolidator for improved context window management.
- Implemented logic to compress message history when approaching token limits, ensuring efficient use of context.
- Added unit tests for the memory consolidator and token estimator to validate functionality and performance.
- Updated tool execution to validate parameters against JSON schema before execution, improving error handling and robustness.
- Introduced a configurable LLM call timeout with a default value, allowing for better control over LLM call durations.
- Added logic to retry transient errors (e.g., timeouts, rate limits) up to a specified maximum number of retries, improving robustness in error handling.
- Implemented parameter casting for tool arguments to ensure correct types are used, addressing common LLM quirks.
- Enhanced tool execution error messages with guidance for retrying with different approaches.
- Added validation for configuration values to prevent runtime errors.
Previously, rag_stream relied entirely on the rewrite plugin's
internal loadHistory() to load conversation history. This meant that
when rewrite was disabled (enable_rewrite=false), RAG mode had NO
conversation history at all — multi-turn conversations silently
degraded to single-turn.
Changes:
1. Add LOAD_HISTORY as the first step in rag_stream pipeline,
ensuring history is always available regardless of rewrite config.
2. Make rewrite plugin reuse chatManage.History if already populated
by the upstream LOAD_HISTORY step, avoiding duplicate DB queries.
This brings rag_stream in line with chat_history_stream which already
has LOAD_HISTORY as an independent step.
Two related fixes for thinking mode streaming:
1. chat_completion_stream.go: When the response channel closes
unexpectedly (context cancelled, upstream error) while thinking
is in progress, the <think> tag was left unclosed. This caused
the stored message to have an incomplete tag, making the frontend
show a perpetual "thinking..." state on history reload.
Now we emit a closing </think> tag when the channel ends with
thinkingStarted=true but thinkingEnded=false.
2. remote_api.go: The thinking Done=true event was only sent when
the first answer content chunk arrived. If the model produced
only reasoning with no answer (e.g., finish_reason set with
empty content), the thinking-done signal was never emitted.
Now we also check isDone && state.hasThinking at the end of
processStreamDelta to ensure the done event is always sent.
The assistant message content may contain <think>...</think> tags from
extended thinking mode. Previously this raw thinking content was
indexed into the chat history knowledge base, polluting embeddings
and degrading RAG retrieval quality with intermediate reasoning text.
Now we strip thinking blocks before building the passage for
IndexMessageToKB, consistent with how load_history and rewrite
plugins already strip thinking from conversation history.
Enhanced cleanPassageForRerank() to convert markdown table data rows
(| col1 | col2 |) into plain text ("col1, col2") instead of leaving
them intact. Added filtering to skip empty/whitespace-only passages
before sending to the rerank API, preventing Baidu Qianfan 400 errors
caused by empty document fragments after markdown cleaning.
When an MCP tool returns image content items, the agent engine now
automatically generates text descriptions using the configured VLM
model and appends them to the tool result message. This makes image
content accessible to LLMs that receive tool results as text.
Images are not passed directly to the LLM because Chat Completions API
does not reliably support images in tool role messages across providers
(tested: gpt-5.4 silently ignores them, Qwen3.5 processes them).
Changes:
- Add Images field to ToolResult for preserving base64 data URIs
- Add extractContentAndImages() with MIME whitelist, size/count limits
- Add ImageDescriberFunc injection to AgentEngine (no vlm package dep)
- Add describeImages() with graceful error handling and ctx cancellation
- Add VLM model injection in CreateAgentEngine (same pattern as
SetAppConfig/SetSkillsManager)
- Redact image base64 from Data map to prevent log/SSE exposure
When a tenant setting is updated via PUT /tenants/kv/{key} (e.g.,
retrieval-config, parser-engine-config), the GORM AfterFind-decrypted
plaintext api_key was silently written back to the database because
db.Updates() does not trigger the BeforeSave hook.
Guard the repository's UpdateTenant to skip writing plaintext api_key
back to the database. Pre-encrypted values (enc:v1:… from CreateTenant
/ UpdateAPIKey) are written as-is; AfterFind-decrypted plaintext is
blanked so GORM skips the column entirely.
Closes#798
Aliyun rerank API rejects empty documents with error "document index:X
should not be empty". This fix validates passage content before building
the documents array, skipping empty chunks and logging them for data
quality monitoring.
- Updated DeleteChunks method to split IDs into batches of 5000 to prevent MySQL Error 1390.
- Added unit tests for CalculateFAQContentHash to ensure consistent hashing behavior across various scenarios, including normalization and case sensitivity.
- Enhanced the SummaryGenerationPayload struct to include a Language field for better localization.
- Updated enqueueSummaryGenerationTask and ProcessSummaryGeneration methods to handle language context in summary generation tasks.
- Modified the session title generation template to focus on extracting only the intent of the user's question.
- Enhanced the session service to include language context when generating titles.
- Updated middleware to inject language into the request context.
- Added a new test suite for validating SSRF safety of URLs.
- Added timezone and language settings to the environment configuration.
- Introduced built-in agent configurations with multilingual support for various agents.
- Updated Docker Compose to utilize new environment variables for timezone and language.
- Created new prompt templates for question generation, summary generation, and keywords extraction.
Table-driven tests for sanitizeManualDownloadFilename covering normal titles,
each dangerous character replacement (/ \ " \n \r), blank/whitespace-only
titles falling back to "untitled.md", and safe passthrough of = ; characters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Use mime.FormatMediaType for both attachment and inline Content-Disposition
headers in DownloadKnowledgeFile and the preview handler, preventing header
injection via a user-controlled knowledge title (e.g. containing ';' or '=')
- Extract inline sanitize logic into sanitizeManualDownloadFilename helper and
add a blank/whitespace-only title fallback to "untitled" so the download
never produces a degenerate ".md" filename
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add an early-return branch that streams manual knowledge content
(stored as Markdown in Metadata.content) directly as a .md file,
avoiding the runtime error that occurred when FilePath is empty.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Removed outdated instructions regarding the `skip_kb_search` field in the conversation configuration.
- Clarified requirements for `image_description` to ensure it is non-empty when images are present.
- Adjusted the handling of message parts in the chat pipeline to improve image processing.
- Deleted unused migration files related to image handling in the database schema.
These changes enhance the clarity and functionality of the conversation and image processing logic.
- Added blur handling for the textarea in the Input-field component to prevent errors when navigating away from the component.
- Improved the sanitization process in the AgentStreamDisplay component to preserve specific tags during markdown processing, ensuring accurate content display.
- Introduced new methods for handling FAQ-specific post-processing in the knowledge base service, optimizing retrieval and filtering of search results.
These changes enhance user experience by improving input handling and content rendering in the chat interface.
- Introduced a new `QARequest` type to streamline parameters for KnowledgeQA and AgentQA service calls, replacing multiple parameters with a single structured request.
- Updated the `KnowledgeQA` and `AgentQA` methods in the session service interface to accept the new `QARequest` type, enhancing clarity and maintainability.
- Refactored the request parsing logic to utilize the new `buildQARequest` method, improving code organization and reducing redundancy.
- Implemented a new `resolveAgent` method to simplify custom agent resolution logic, enhancing readability and maintainability.
These changes improve the overall structure and usability of the QA request handling, facilitating easier updates and better code organization.
- Introduced a new parameter `webSearchEnabled` to the `AgentQA` method, allowing for conditional web search capabilities.
- Updated the session service interface and handler to accommodate the new parameter, enhancing the flexibility of the AgentQA process.
- Adjusted the configuration to ensure that web search functionality can be toggled based on the provided input.
These changes improve the AgentQA feature by enabling users to control web search integration, enhancing the overall user experience.
- Introduced a new method to resolve embedding model IDs to their actual identity keys (name + endpoint), enabling optimal grouping of knowledge bases across different tenants.
- Updated the concurrent search functionality to utilize the resolved model keys, improving the efficiency of query embedding computations.
- Refactored search target grouping to reduce redundant API calls and enhance performance during hybrid searches.
These changes significantly improve the knowledge search process by ensuring that embeddings are computed efficiently and reused across multiple knowledge bases sharing the same underlying model.
- Enhanced the concurrent search functionality to group search targets by their embedding model, allowing for a single computation of query embeddings per model.
- Implemented batch fetching of knowledge base records to improve efficiency and reduce redundant API calls.
- Updated the HybridSearch method to support multiple knowledge base IDs, enabling combined retrieval for full-KB targets and improving overall search performance.
- Added a new GetQueryEmbedding method to compute embeddings for queries, facilitating reuse across multiple knowledge bases sharing the same model.
These changes significantly enhance the performance and efficiency of the knowledge search process, reducing API calls and improving response times.
- Implemented a new API endpoint to clear all messages in a session, enhancing session management capabilities.
- Updated the frontend to include a clear messages option in the session menu, allowing users to easily remove chat history.
- Added localization support for the new feature in English, Korean, Russian, and Chinese, improving accessibility for a broader audience.
- Enhanced the chat view to handle session message clearing events, ensuring a responsive user experience.
These changes significantly improve the user experience by providing a straightforward method to manage session messages effectively.
- Refactored the chat model ID selection process to prioritize the request's summaryModelID, followed by the custom agent's ModelID, and finally a fallback to the default model selection.
- Improved logging to provide clearer insights into which model ID is being used during the selection process.
- Removed the deprecated selectChatModelIDWithOverride function to streamline the codebase and enhance maintainability.
These changes improve the flexibility and clarity of chat model selection, ensuring a more robust session management experience.
- Added support for updating the description field in knowledge records, improving the detail and context of knowledge entries.
- Refactored knowledge base update logic to ensure configuration updates are only applied when provided, enhancing robustness and preventing unnecessary overwrites.
- Modified the request structure to make the configuration field optional, allowing for more flexible updates.
These changes improve the knowledge management system by allowing for more comprehensive updates and better handling of configuration data.