- Replaced LLMConfig with model_id in TextRelationExtractionRequest and FabriTextRequest interfaces.
- Updated related API handlers to retrieve chat models based on model_id.
- Adjusted frontend components to pass model_id instead of LLMConfig, enhancing clarity and simplifying the configuration process.
These changes streamline the model configuration process across the application, improving maintainability and usability.
- Renamed the input parameter from "pattern" to "patterns" to reflect that multiple text patterns can be provided.
- Updated the schema definition and input structure accordingly to ensure consistency in handling multiple patterns.
- Enhanced error logging to specify "patterns" when no valid input is provided, improving clarity for debugging.
These changes improve the functionality of the grep_chunks tool by allowing users to search for multiple patterns simultaneously.
- Implemented a new API endpoint to clear all messages in a session, enhancing session management capabilities.
- Updated the frontend to include a clear messages option in the session menu, allowing users to easily remove chat history.
- Added localization support for the new feature in English, Korean, Russian, and Chinese, improving accessibility for a broader audience.
- Enhanced the chat view to handle session message clearing events, ensuring a responsive user experience.
These changes significantly improve the user experience by providing a straightforward method to manage session messages effectively.
- Refactored the chat model ID selection process to prioritize the request's summaryModelID, followed by the custom agent's ModelID, and finally a fallback to the default model selection.
- Improved logging to provide clearer insights into which model ID is being used during the selection process.
- Removed the deprecated selectChatModelIDWithOverride function to streamline the codebase and enhance maintainability.
These changes improve the flexibility and clarity of chat model selection, ensuring a more robust session management experience.
- Added support for updating the description field in knowledge records, improving the detail and context of knowledge entries.
- Refactored knowledge base update logic to ensure configuration updates are only applied when provided, enhancing robustness and preventing unnecessary overwrites.
- Modified the request structure to make the configuration field optional, allowing for more flexible updates.
These changes improve the knowledge management system by allowing for more comprehensive updates and better handling of configuration data.
- Updated configuration to support three tasks in the assistant's prompt: rewriting questions, classifying intent for knowledge base retrieval, and analyzing attached images.
- Implemented backend functionality to update image descriptions in user messages, ensuring that image context is preserved across conversation history.
- Modified session handling to return complete session data after updates, improving the accuracy of session information provided to users.
- Enhanced the chat pipeline to include detailed image descriptions and manage image captions effectively, enriching user interactions with visual content.
These improvements significantly enhance the application's multimodal capabilities, providing a more robust and user-friendly experience when handling image-based queries.
- Updated configuration to improve handling of multimodal queries, including clearer guidelines for when to use [NO_SEARCH] in responses.
- Enhanced the user interface to include dedicated sections for multimodal configuration, allowing users to manage image uploads and related settings more effectively.
- Added localization support for new multimodal features in both English and Chinese, improving accessibility for a broader audience.
- Implemented backend changes to support image analysis and extraction of OCR text, enriching the interaction experience for users.
These enhancements significantly improve the application's capability to process and respond to multimodal queries, fostering a more interactive and user-friendly environment.
- Added functionality for image uploads in chat, allowing users to attach images for multimodal Q&A.
- Enhanced the input field to handle image selection via drag-and-drop and paste, with validation for file types and sizes.
- Updated the backend to process images alongside text queries, including support for image analysis.
- Introduced new UI components for image previews and management, improving user interaction with uploaded content.
- Added localization strings for image upload features and error messages, enhancing accessibility for users in multiple languages.
These changes significantly improve the chat experience by enabling users to incorporate images into their queries, facilitating richer interactions and responses.
- Updated the summary length requirement for document summarization to 100-300 words.
- Added new localization strings for missing storage engine notifications in English, Korean, Russian, and Chinese.
- Implemented UI warnings for users regarding storage engine configuration when files exist, including confirmation dialogs for storage engine changes.
- Enhanced backend validation to ensure a storage engine is configured before allowing content uploads.
These changes improve user experience by providing clearer guidance on storage engine requirements and enhancing the overall functionality of the knowledge base management system.
- add StreamSender interface and streaming output for WeCom/Feishu
- refactor WeCom adapter into separate webhook and websocket files
- add output_mode config for stream/full toggle per platform
- add stream flush batching, dedup cleanup, and think-block filtering
- improve context template to handle irrelevant retrieved info gracefully
- add IM streaming integration tests
- add IM integration documentation with quick start guide
- support webhook and websocket modes for both platforms
- add im_channel_sessions migration for channel-session mapping
- register IM adapters and callback routes
- update config and docker-compose for IM env vars
Prevents 400 Bad Request errors when uploading documents with large
tables or code blocks by adding hard limits in buildUnitsWithProtection
and mergeUnits functions.
- Added API endpoints for moving knowledge items between knowledge bases, including validation for source and target compatibility.
- Implemented asynchronous task handling for knowledge moves, tracking progress and status updates.
- Introduced new localization strings for move-related UI elements in English, Korean, Russian, and Chinese.
- Enhanced the frontend to support knowledge move operations, including target selection and progress monitoring.
These changes significantly improve the application's capabilities for managing knowledge across different bases, providing users with a seamless experience in organizing their knowledge assets.
- Revised the storage size calculation logic for Milvus, Qdrant, and Weaviate repositories to provide more accurate estimates based on the actual data structure and indexing methods.
- Updated comments to clarify the relationship between vector dimensions and index storage requirements, enhancing code readability and maintainability.
- Added support for AWS S3 as a storage engine, including configuration options for endpoint, region, access key, secret key, bucket name, and path prefix.
- Updated the StorageEngineConfig interface to include S3 configuration.
- Enhanced the UI components to allow users to configure S3 settings and test connections.
- Updated localization files to include new S3-related labels and descriptions in multiple languages.
These changes significantly enhance the storage capabilities of the application, providing users with more options for managing their data.
- Added new API endpoints for managing chat history configuration and retrieval settings, allowing tenants to enable message indexing and configure search parameters.
- Introduced new Vue components for ChatHistorySettings and RetrievalSettings, providing a user-friendly interface for managing these configurations.
- Updated localization files to include new settings descriptions and labels in multiple languages.
- Enhanced the KnowledgeSearch view to support searching across both knowledge bases and chat history, improving the overall search functionality.
These changes significantly enhance the application's capabilities in managing and retrieving chat history, contributing to a more robust user experience.
- Introduced the final_answer tool to ensure agents submit their complete responses as the last action, improving response accuracy and consistency.
- Updated the agent engine to log detailed information about tool calls and responses, including the total duration of agent execution.
- Enhanced the chat message model to include agent_duration_ms, tracking the total execution time from query start to answer delivery.
- Implemented JSON field extraction for streaming responses, allowing for more efficient handling of tool call outputs.
These changes significantly improve the agent's response handling and ensure that final answers are consistently delivered, enhancing overall user experience.
- Replace hardcoded tag options with English equivalents (Content,
Culture, Person, etc.) so random tag generation works across locales
- Replace default extract example with English text (Romeo and Juliet)
matching the existing config.yaml extract_graph examples
- Update graph database status string to use locale-independent value
and align frontend comparison accordingly
- Convert error messages in fabri text handler to English
- Add crypto utility (internal/utils/crypto.go) with AES-256-GCM encrypt/decrypt
using SYSTEM_AES_KEY env var, with "enc:v1:" prefix for versioned ciphertext
- Encrypt tenant API key via GORM BeforeSave/AfterFind hooks and manual
encryption in CreateTenant/UpdateAPIKey (db.Updates bypasses hooks)
- Encrypt model API key in ModelParameters Value/Scan (driver.Valuer)
- Widen api_key column from varchar(64) to varchar(256) across all DB dialects
(MySQL, ParadeDB, SQLite) and add versioned migration 000018
- Propagate SYSTEM_AES_KEY through docker-compose, Helm secrets and values
- Fix migration 000017 PL/pgSQL dollar-quoting syntax ($ -> $$)
Convert all LLM-facing prompts, templates, and tool output labels to
language-neutral English so the model responds in the user's input
language. Update frontend relevance/match-type mappings to match.
Output language strategy:
- User-facing: reply in user's input language
- Vector-indexed: match source document language
- LLM-internal: fixed English
- Implemented ListChunksByParentIDs in chunkRepository to retrieve chunks based on multiple parent IDs, improving data retrieval efficiency.
- Enhanced the chunk merging logic in PluginMerge to better handle overlapping chunks and collect merged ImageInfo for parent chunks.
- Updated the ChunkRepository interface to include the new ListChunksByParentIDs method, ensuring consistency across the application.
These changes optimize chunk management and improve the handling of image information during merges, enhancing overall application functionality.
- Implemented GetUserByTenantID method in userRepository to retrieve the first user associated with a given tenant ID.
- Added corresponding GetUserByTenantID method in userService to facilitate user retrieval by tenant ID.
- Updated UserService and UserRepository interfaces to include the new method, ensuring consistency across the application.
- Enhanced auth middleware to store user information in context based on tenant ID, improving user management.
These changes enhance user retrieval capabilities by allowing access to user data based on tenant associations, improving overall application functionality.
- Increased default sizes for parent and child chunks in the knowledge base editor and settings, enhancing data processing capabilities.
- Adjusted slider settings in the UI to reflect new minimum and maximum values for chunk sizes, improving user control over configurations.
- Updated backend logic to ensure consistent handling of chunk sizes across the application.
These changes optimize the chunking process, allowing for more efficient data handling and improved performance in knowledge base operations.
- Introduced a new API endpoint to toggle the pin status of knowledge bases, allowing users to pin important entries to the top of the list.
- Updated the frontend to include UI elements for pinning and unpinning knowledge bases, with corresponding success and error messages.
- Enhanced localization support by adding translations for pinning actions in English, Korean, Russian, and Chinese.
- Modified the database schema to include `is_pinned` and `pinned_at` fields for knowledge bases, enabling persistent pinning status.
These changes improve the organization and accessibility of knowledge bases, enhancing user experience by allowing quick access to prioritized entries.
- Introduced a constant `maxFunctionNameLength` set to 64 characters to comply with OpenAI API constraints on tool names.
- Updated the `Name` method in `MCPTool` to truncate tool names exceeding the maximum length, ensuring unique identification while preventing potential collisions.
- Enhanced the logic to handle long service IDs by using only the first 8 characters when necessary, maintaining the integrity of tool naming conventions.
These changes improve compliance with API requirements and enhance the robustness of tool name generation.
- Introduced a new `is_fallback` property to track when responses are generated without knowledge base matches.
- Updated the chat components to display fallback hints and indicators in the UI, enhancing user awareness of fallback responses.
- Added translations for fallback hints in multiple languages, improving accessibility for diverse users.
- Modified backend logic to emit fallback status in response events, ensuring consistent handling across the application.
These changes improve the user experience by providing clear feedback on fallback responses, enhancing the overall interaction with the chat system.
- Updated the StorageCheckResponse to include a new field `bucket_created` to indicate if a bucket was automatically created during the connectivity check.
- Modified the MinIO connectivity check logic to attempt auto-creation of the bucket if it does not exist, setting a public-read policy if successful.
- Adjusted the frontend to reflect the new response structure, updating the UI to display a 'created' status when a bucket is auto-created.
These changes improve the user experience by providing clear feedback on bucket creation during storage checks, enhancing the overall functionality of the storage engine settings.
- Added support for parsing both `results.document` and `results.files` in the MinerU API response, allowing for more flexible handling of markdown content and images.
- Implemented fallback logic to prefer `results.document` when available, improving the robustness of document processing.
- Added debug and warning logging to track response path usage and potential issues with missing content.
These changes improve the reliability and clarity of document parsing from the MinerU API, ensuring better handling of varying response formats.
- Introduced a new function `isIconImage` to identify small images and decorative elements based on pixel dimensions and file size.
- Updated the image resolution logic to filter out identified icon images from markdown content, enhancing the quality of stored images.
- Added unit tests for icon image detection to ensure accurate functionality across various image sizes.
These changes improve image handling by effectively filtering out non-essential images, leading to cleaner document processing.
- Introduced a new function `cleanPassageForRerank` to strip markdown and structural noise from text, enhancing semantic clarity for rerank models.
- Implemented comprehensive regex patterns to remove images, links, code blocks, LaTeX, HTML tags, and other formatting elements.
- Added unit tests to validate the cleaning process across various input scenarios, ensuring expected output consistency.
- Updated the `getEnrichedPassage` function to utilize the new cleaning method, improving the quality of text passed to the rerank model.
These changes significantly enhance the preprocessing of text data, leading to better performance in reranking tasks.
- Added support for parent-child chunking in the knowledge base, allowing for a two-level chunking strategy that improves context retrieval.
- Introduced new configuration options for enabling parent-child chunking, including parent and child chunk sizes.
- Updated relevant components and services to handle parent chunk data, ensuring proper linking and retrieval of child chunks.
- Enhanced user interface elements to reflect the new chunking options and provide better user guidance.
These changes significantly improve the knowledge extraction process by providing richer context through hierarchical chunking.
- Added automatic filtering for `deleted_at IS NULL` in database queries to ensure only active records are returned.
- Updated the `database_query.go` documentation to reflect the new soft-delete feature and its implications for query construction.
- Enhanced the SQL validation process to include soft-delete conditions, improving data integrity and security.
These changes enhance the robustness of database interactions by enforcing soft-delete logic across relevant queries.
- Added a new method `resolveFileServiceForKnowledge` to dynamically resolve the appropriate file service based on the tenant's storage configuration and the knowledge file's provider.
- Updated the `extractionResources` struct to include a reference to the tenant, enabling the new resolution logic.
- Modified the `processTableData` method to utilize the resolved file service, improving the handling of file operations during data extraction.
These changes enhance the flexibility and reliability of file service management in the data extraction process.
- Removed the image storage handling from the `_resolve_images` function, which now only decodes images and returns them as inline bytes.
- Updated the function's docstring to clarify that image persistence is managed by the Go App, and the return value for `image_dir_path` is always empty.
- Adjusted related gRPC and protobuf files to remove the `ConvertToPDF` method and its associated types, streamlining the API.
- Enhanced security measures in the frontend by implementing a placeholder for provider images and ensuring proper hydration of protected file images.
These changes improve the clarity and efficiency of image processing and API interactions within the application.
- Eliminated the `mineru_api_base_url` field from the `ParserEngineConfig` interface and its usage across the application, simplifying the configuration structure.
- Updated the `ParserEngineSettings` view to remove the corresponding input field, enhancing the user interface.
- Adjusted the `PingMinerUCloud` function to no longer require the base URL parameter, defaulting to a predefined value instead.
- Modified database migration comments to reflect the removal of the base URL from the configuration.
These changes streamline the configuration process for the MinerU Cloud integration, improving clarity and reducing potential misconfigurations.
- Introduced `TagID` and `IsRecommended` fields across various repository implementations (Elasticsearch, Milvus, Postgres, Qdrant, SQLite) to enhance document categorization and recommendation capabilities.
- Updated the logic for handling `IsEnabled` to ensure it reflects the actual state from the source data.
- Adjusted the knowledge service to incorporate the new `IsRecommended` field, improving the indexing process for knowledge chunks.
These changes enhance the flexibility and functionality of document retrieval and indexing across the application.
- Added `IsEnabled` and `IsRecommended` fields to the document indexing process in both ElasticFaiss and Elasticsearch repositories, improving the flexibility and functionality of document retrieval.
- Updated the data extraction process to include these new fields, ensuring they are correctly populated from the source data.
- Adjusted the indexing logic in the knowledge service to reflect these changes, maintaining consistency across the application.
- Removed hardcoded IP address from blocked hosts in `system.go`.
- Integrated SSRF safety checks in `MinerUCloudReader` for base URL and zip URL validation.
- Updated HTTP client initialization to use a new utility function for SSRF-safe HTTP clients, improving security during API requests.
These changes enhance the security of the MinerUCloudReader by ensuring that potentially unsafe URLs are properly validated before use.
- Introduced a new API function `deleteAllSessions` to allow deletion of all sessions for a tenant.
- Updated the `BatchManageDialog` component to support a confirmation dialog for deleting all sessions.
- Enhanced the menu component to reflect the new delete all functionality with appropriate confirmation messages.
- Added internationalization support for the new delete all confirmation message in multiple languages.
These changes improve session management by providing users with the ability to delete all sessions at once, enhancing the overall user experience.
wizardchen