Commit Graph

809 Commits

Author SHA1 Message Date
ChenRussell 166f106b7f fix: ensure builtin engine handles simple formats correctly
Builtin engine was incorrectly falling back to simple engine for md/txt/csv files. Added explicit builtin engine check in selection logic.

Fixes #755
2026-03-15 23:58:49 +08:00
Manx98 447fac4b54 fix: NVIDIA VLM model API default BaseURL 2026-03-13 18:14:08 +08:00
Manx98 a355ac889e feat: NVIDIA chat model API support custom endpoint 2026-03-13 18:14:08 +08:00
Manx98 5afc81d00b feat: support NVIDIA model API 2026-03-13 18:14:08 +08:00
ChenRussell b7ed54f8de fix: add missing UserInfo in organization.go 2026-03-13 18:13:10 +08:00
qunbo.guo fb9e70fd16 feat: 改进文档预览组件的响应式设计
- 优化doc-content组件抽屉宽度,在小屏幕上自适应调整
- 增强document-preview组件的响应式布局,防止内容溢出
- 添加媒体查询支持,提升移动端用户体验
- 修复DOCX文档预览在不同屏幕尺寸下的显示问题
2026-03-12 10:31:48 +08:00
Andy Yang e57deb218c feat(preview): add localized document fullscreen toggle
This feature introduces a localized fullscreen toggle capability directly to the document preview component. It adds translations across 4 UI languages (zh-CN, en-US, ko-KR, ru-RU) avoiding hardcoded strings. It also sets proper responsive height scaling for DOCX views expanding to full viewport.
2026-03-09 20:26:43 +08:00
wizardchen daa9ef500c feat: add agent management functionality
- Introduced a new package for managing custom agents, including CRUD operations for agent creation, retrieval, updating, and deletion.
- Implemented API endpoints for listing agents and retrieving agent placeholders.
- Added data structures for agent configuration and requests, enhancing the overall agent management capabilities.
- Enhanced the client with methods to interact with the new agent management features, improving user experience in managing agents.

These changes significantly expand the application's functionality for handling custom agents, providing users with a comprehensive toolset for agent management.
2026-03-09 17:34:13 +08:00
wizardchen 125d3e5e4a feat: implement knowledge move functionality
- Added API endpoints for moving knowledge items between knowledge bases, including validation for source and target compatibility.
- Implemented asynchronous task handling for knowledge moves, tracking progress and status updates.
- Introduced new localization strings for move-related UI elements in English, Korean, Russian, and Chinese.
- Enhanced the frontend to support knowledge move operations, including target selection and progress monitoring.

These changes significantly improve the application's capabilities for managing knowledge across different bases, providing users with a seamless experience in organizing their knowledge assets.
2026-03-09 13:23:16 +08:00
wizardchen 33ccc41bea refactor: update storage size calculations for vector embeddings
- Revised the storage size calculation logic for Milvus, Qdrant, and Weaviate repositories to provide more accurate estimates based on the actual data structure and indexing methods.
- Updated comments to clarify the relationship between vector dimensions and index storage requirements, enhancing code readability and maintainability.
2026-03-09 13:23:16 +08:00
wizardchen dd39fbc018 chore: remove obsolete localization rule file 2026-03-09 11:46:56 +08:00
wizardchen 3f340771e9 feat(storage): integrate AWS S3 support in storage settings
- Added support for AWS S3 as a storage engine, including configuration options for endpoint, region, access key, secret key, bucket name, and path prefix.
- Updated the StorageEngineConfig interface to include S3 configuration.
- Enhanced the UI components to allow users to configure S3 settings and test connections.
- Updated localization files to include new S3-related labels and descriptions in multiple languages.

These changes significantly enhance the storage capabilities of the application, providing users with more options for managing their data.
2026-03-09 11:46:56 +08:00
wizardchen b5edc4aa39 chore: update AWS SDK dependencies and add database migrations
- Updated AWS SDK dependencies to versions v1.29.14 for config and v1.83.0 for S3, ensuring compatibility and access to the latest features.
- Added migration scripts to introduce  column in the messages table for tracking agent execution duration.
- Added migration scripts to introduce  column in the messages table and new JSONB columns in the tenants table for chat history configuration.
2026-03-09 11:46:56 +08:00
wizardchen 1fb2bedd2b feat: implement chat history and retrieval settings
- Added new API endpoints for managing chat history configuration and retrieval settings, allowing tenants to enable message indexing and configure search parameters.
- Introduced new Vue components for ChatHistorySettings and RetrievalSettings, providing a user-friendly interface for managing these configurations.
- Updated localization files to include new settings descriptions and labels in multiple languages.
- Enhanced the KnowledgeSearch view to support searching across both knowledge bases and chat history, improving the overall search functionality.

These changes significantly enhance the application's capabilities in managing and retrieving chat history, contributing to a more robust user experience.
2026-03-09 11:46:56 +08:00
wizardchen c516d1f9bd style: update AgentStreamDisplay component for improved UI consistency 2026-03-09 11:46:56 +08:00
wizardchen a4b255a456 feat: implement final_answer tool and enhance agent duration tracking
- Introduced the final_answer tool to ensure agents submit their complete responses as the last action, improving response accuracy and consistency.
- Updated the agent engine to log detailed information about tool calls and responses, including the total duration of agent execution.
- Enhanced the chat message model to include agent_duration_ms, tracking the total execution time from query start to answer delivery.
- Implemented JSON field extraction for streaming responses, allowing for more efficient handling of tool call outputs.

These changes significantly improve the agent's response handling and ensure that final answers are consistently delivered, enhancing overall user experience.
2026-03-09 11:46:56 +08:00
wizardchen bedb8ceb08 refactor: standardize loading and avatar sizes across components
- Changed loading component size from large to medium in document preview, agent list, knowledge base list, and organization list for improved consistency.
- Updated avatar sizes from large to medium in agent editor modal, organization editor modal, organization settings modal, and other relevant components to maintain uniformity in UI design.
- Adjusted switch component sizes from large to medium in various settings to enhance visual coherence.

These changes contribute to a more consistent and polished user interface across the application.
2026-03-09 11:46:56 +08:00
DaWesen e309e0bed8 feat(storage): 集成S3存储适配器
添加对AWS S3及兼容存储服务的支持:
- 实现完整的S3FileService接口
- 支持文件上传、下载、删除功能
- 添加配置支持和环境变量检查
- 实现连接测试功能
- 遵循与其他存储适配器相同的代码风格
2026-03-09 10:39:46 +08:00
MaoMengww e898d62141 refactor: change Tokenization Method 2026-03-09 10:36:16 +08:00
MaoMengww c4a5a4d99b feat: support weaviate vectordb for knowledge retrieve 2026-03-09 10:36:16 +08:00
ochan.kwon 5583bf0eb4 fix(i18n): enable multilingual support for Knowledge Graph settings
- Replace hardcoded tag options with English equivalents (Content,
  Culture, Person, etc.) so random tag generation works across locales
- Replace default extract example with English text (Romeo and Juliet)
  matching the existing config.yaml extract_graph examples
- Update graph database status string to use locale-independent value
  and align frontend comparison accordingly
- Convert error messages in fabri text handler to English
2026-03-09 10:35:41 +08:00
ochan.kwon 10614df567 fix(i18n): complete multilingual support for TDesign components and locale key synchronization
- Synchronize 2641 i18n keys across all 4 locales (en-US, zh-CN, ko-KR, ru-RU)
- Add TDesign ConfigProvider with dynamic locale to ensure all UI
  components (select, input, empty states, dialogs) display in the
  user's chosen language
- Fix provider dropdown in model editor: use i18n keys with API fallback
- Fix connection test messages: always use i18n for user-facing feedback
- Add explicit placeholders to t-select components in settings pages
- Add missing i18n keys: theme.selectTheme, organization role selectors,
  lkeap provider translations
2026-03-09 10:35:41 +08:00
AndyYang 6c69de2df1 feat(security): add AES-256-GCM encryption for API keys at rest
- Add crypto utility (internal/utils/crypto.go) with AES-256-GCM encrypt/decrypt
  using SYSTEM_AES_KEY env var, with "enc:v1:" prefix for versioned ciphertext
- Encrypt tenant API key via GORM BeforeSave/AfterFind hooks and manual
  encryption in CreateTenant/UpdateAPIKey (db.Updates bypasses hooks)
- Encrypt model API key in ModelParameters Value/Scan (driver.Valuer)
- Widen api_key column from varchar(64) to varchar(256) across all DB dialects
  (MySQL, ParadeDB, SQLite) and add versioned migration 000018
- Propagate SYSTEM_AES_KEY through docker-compose, Helm secrets and values
- Fix migration 000017 PL/pgSQL dollar-quoting syntax ($ -> $$)
2026-03-09 10:35:07 +08:00
Manx98 1d1d3de76a fix: make dev-app command error on Linux 2026-03-09 10:33:52 +08:00
ochan.kwon 111bb8b83a fix(i18n): fix vue-i18n placeholder escaping, computed ref accessor, and missing ru-RU keys
- Escape literal {{ and }} in locale strings using vue-i18n's {'{{'}
  syntax to prevent "Not allowed nest placeholder" compilation errors
  (ko-KR: 10, en-US: 10, ru-RU: 2, zh-CN: 1)
- Add missing .value accessor on allTools computed ref in
  AgentEditorModal, fixing "allTools.map is not a function" error
- Add missing menu.agents and menu.organizations keys to ru-RU locale

All issues are follow-ups from #724 (frontend i18n).
2026-03-06 21:08:28 +08:00
ochan.kwon edcf7116cd feat: use language-neutral English prompts for multilingual LLM responses
Convert all LLM-facing prompts, templates, and tool output labels to
language-neutral English so the model responds in the user's input
language. Update frontend relevance/match-type mappings to match.

Output language strategy:
- User-facing: reply in user's input language
- Vector-indexed: match source document language
- LLM-internal: fixed English
2026-03-06 21:08:12 +08:00
ochan.kwon db7cef5f17 feat(i18n): replace hardcoded Chinese strings with i18n calls across frontend
Replace all user-visible hardcoded Chinese strings in Vue/TS source files
with proper i18n function calls and add corresponding translation keys
to all 4 locale files (zh-CN, en-US, ko-KR, ru-RU).

Changes:
- Replace 80+ hardcoded Chinese strings across 30+ source files
- Add ~450 new translation keys per locale file
- Cover components, views, utils, hooks, and API modules
- Remove Chinese fallback patterns (|| '中文') from existing t() calls
- Add mermaid toolbar and ollama date format translations

Intentionally preserved (backend API values / backward compatibility):
- FAQ CSV column headers and parsing constants
- RelevanceLevel type and Record key mappings
- Graph database engine status comparison ('未启用')
- Language name labels ('简体中文')
2026-03-06 10:34:08 +08:00
DaWesen 65743da754 fix: 修复 Milvus retriever 相关问题
- 修正 envMilvusCollection 字段名拼写错误
- 将 log.Errorf 中的 %w 改为 %v,修复日志格式化问题
- 保留错误包装功能,确保错误处理的完整性
2026-03-06 10:33:09 +08:00
AndyYang 6dde1330c2 feat: add built-in MCP service support 2026-03-05 21:57:01 +08:00
wizardchen ead6712955 chore: update CHANGELOG and version to 0.3.3 2026-03-05 18:14:14 +08:00
wizardchen 43394a35bb refactor: adjust component styles for improved consistency
- Modified border-radius values in botmsg and usermsg components for a more uniform appearance.
- Increased font size in the ai-markdown-template class for better readability.
- Removed hover scale effect in AgentStreamDisplay, botmsg, and usermsg components to streamline interactions.

These changes enhance the visual consistency and usability of the chat components, contributing to a more polished user interface.
2026-03-05 18:14:14 +08:00
wizardchen 927e5d4750 feat: add ListChunksByParentIDs method and enhance chunk merging logic
- Implemented ListChunksByParentIDs in chunkRepository to retrieve chunks based on multiple parent IDs, improving data retrieval efficiency.
- Enhanced the chunk merging logic in PluginMerge to better handle overlapping chunks and collect merged ImageInfo for parent chunks.
- Updated the ChunkRepository interface to include the new ListChunksByParentIDs method, ensuring consistency across the application.

These changes optimize chunk management and improve the handling of image information during merges, enhancing overall application functionality.
2026-03-05 18:14:14 +08:00
wizardchen 4ddc468fea feat: add GetUserByTenantID functionality to user repository and service
- Implemented GetUserByTenantID method in userRepository to retrieve the first user associated with a given tenant ID.
- Added corresponding GetUserByTenantID method in userService to facilitate user retrieval by tenant ID.
- Updated UserService and UserRepository interfaces to include the new method, ensuring consistency across the application.
- Enhanced auth middleware to store user information in context based on tenant ID, improving user management.

These changes enhance user retrieval capabilities by allowing access to user data based on tenant associations, improving overall application functionality.
2026-03-05 18:14:14 +08:00
wizardchen 371002e085 fix: standardize border styles across components
- Updated border styles in multiple components to use a consistent thickness of 0.5px instead of 1px, enhancing visual uniformity.
- Adjusted styles in components such as AgentSelector, Input-field, KnowledgeBaseSelector, and others to improve overall UI consistency.
- Refined CSS variables for borders to ensure maintainability and adherence to design specifications.

These changes improve the aesthetic coherence of the application, providing a more polished user interface.
2026-03-05 18:14:14 +08:00
wizardchen b697d4945e feat: update chunk size configurations for knowledge base processing
- Increased default sizes for parent and child chunks in the knowledge base editor and settings, enhancing data processing capabilities.
- Adjusted slider settings in the UI to reflect new minimum and maximum values for chunk sizes, improving user control over configurations.
- Updated backend logic to ensure consistent handling of chunk sizes across the application.

These changes optimize the chunking process, allowing for more efficient data handling and improved performance in knowledge base operations.
2026-03-05 18:14:14 +08:00
wizardchen 2e48bc0e63 feat: update at-icon SVG and enhance input field component
- Replaced the image source for the at-icon with inline SVG for better scalability and styling flexibility.
- Updated the SVG to use `currentColor` for stroke, allowing it to adapt to the current text color.
- Enhanced the input field component with improved styling and responsiveness, including adjustments to color transitions and hover effects.
- Refined the mention selector component to improve popup placement and scrolling behavior, ensuring a smoother user experience.

These changes enhance the visual consistency and usability of the components, providing a more modern and responsive interface.
2026-03-05 18:14:14 +08:00
wizardchen 5d90019fd0 feat: enhance sidebar functionality and UI responsiveness
- Implemented sidebar collapse and expand functionality, allowing users to toggle the sidebar visibility for a more streamlined interface.
- Updated the `uiStore` to manage sidebar state, persisting the collapsed state in local storage for user preference.
- Enhanced the `menu.vue` and `UserMenu.vue` components to adapt their layout based on the sidebar state, improving usability.
- Added corresponding translations for sidebar actions in English, Korean, Russian, and Chinese, ensuring accessibility for diverse users.
- Refined CSS styles to support the new sidebar behavior, enhancing overall UI consistency and responsiveness.

These changes significantly improve the user experience by providing a more flexible and user-friendly navigation system.
2026-03-05 18:14:14 +08:00
wizardchen 00bf0460cc feat: add pinning functionality for knowledge bases
- Introduced a new API endpoint to toggle the pin status of knowledge bases, allowing users to pin important entries to the top of the list.
- Updated the frontend to include UI elements for pinning and unpinning knowledge bases, with corresponding success and error messages.
- Enhanced localization support by adding translations for pinning actions in English, Korean, Russian, and Chinese.
- Modified the database schema to include `is_pinned` and `pinned_at` fields for knowledge bases, enabling persistent pinning status.

These changes improve the organization and accessibility of knowledge bases, enhancing user experience by allowing quick access to prioritized entries.
2026-03-05 18:14:14 +08:00
wizardchen a1c0682aa3 feat: enforce maximum length for tool names in MCPTool
- Introduced a constant `maxFunctionNameLength` set to 64 characters to comply with OpenAI API constraints on tool names.
- Updated the `Name` method in `MCPTool` to truncate tool names exceeding the maximum length, ensuring unique identification while preventing potential collisions.
- Enhanced the logic to handle long service IDs by using only the first 8 characters when necessary, maintaining the integrity of tool naming conventions.

These changes improve compliance with API requirements and enhance the robustness of tool name generation.
2026-03-05 18:14:14 +08:00
wizardchen 90c32b5926 feat: enhance Docker setup with entrypoint script and skill management
- Added a new entrypoint script to manage ownership of bind-mounted directories and merge built-in skills into preloaded ones.
- Updated the Dockerfile to include the `gosu` package for privilege management and to set the entrypoint to the new script.
- Ensured built-in skills are preserved and accessible after bind-mounting user directories, improving the application's flexibility and usability.

These changes streamline the container initialization process and enhance the management of skills within the application.
2026-03-05 18:14:14 +08:00
wizardchen dbc7c32874 feat: update theme and UI styles across components
- Refactored CSS variables for consistent theming, replacing hardcoded colors with CSS variables for better maintainability.
- Updated various components including `AgentAvatar`, `AgentSelector`, and `AgentShareSettings` to utilize new theme variables for colors, backgrounds, and shadows.
- Enhanced SVG icons to use `currentColor` for better adaptability to theme changes.
- Improved overall UI consistency by aligning styles with the new theming approach, enhancing user experience across the application.

These changes significantly improve the visual coherence and maintainability of the UI, ensuring a more seamless user experience.
2026-03-05 18:14:14 +08:00
wizardchen 44f38ca6c3 feat: add fallback response handling and UI indicators
- Introduced a new `is_fallback` property to track when responses are generated without knowledge base matches.
- Updated the chat components to display fallback hints and indicators in the UI, enhancing user awareness of fallback responses.
- Added translations for fallback hints in multiple languages, improving accessibility for diverse users.
- Modified backend logic to emit fallback status in response events, ensuring consistent handling across the application.

These changes improve the user experience by providing clear feedback on fallback responses, enhancing the overall interaction with the chat system.
2026-03-05 18:14:14 +08:00
wizardchen 5ae076e22c feat: add reparse functionality for knowledge documents
- Introduced a new API function `reparseKnowledge` to trigger the re-parsing of knowledge documents.
- Updated the KnowledgeBase component to include a user interface option for document re-parsing, with confirmation prompts and status messages.
- Added corresponding translations for the reparse feature in both English and Chinese locales.

These changes enhance the document management capabilities by allowing users to rebuild knowledge documents, improving the overall user experience.
2026-03-05 18:14:14 +08:00
wizardchen 76a5b329f3 feat: enhance storage engine connectivity check with auto-creation of buckets
- Updated the StorageCheckResponse to include a new field `bucket_created` to indicate if a bucket was automatically created during the connectivity check.
- Modified the MinIO connectivity check logic to attempt auto-creation of the bucket if it does not exist, setting a public-read policy if successful.
- Adjusted the frontend to reflect the new response structure, updating the UI to display a 'created' status when a bucket is auto-created.

These changes improve the user experience by providing clear feedback on bucket creation during storage checks, enhancing the overall functionality of the storage engine settings.
2026-03-05 18:14:14 +08:00
wizardchen fd742fea42 feat: enhance MinerU response handling for document parsing
- Added support for parsing both `results.document` and `results.files` in the MinerU API response, allowing for more flexible handling of markdown content and images.
- Implemented fallback logic to prefer `results.document` when available, improving the robustness of document processing.
- Added debug and warning logging to track response path usage and potential issues with missing content.

These changes improve the reliability and clarity of document parsing from the MinerU API, ensuring better handling of varying response formats.
2026-03-05 18:14:14 +08:00
wizardchen 45eba325e2 feat: add image icon detection and filtering functionality
- Introduced a new function `isIconImage` to identify small images and decorative elements based on pixel dimensions and file size.
- Updated the image resolution logic to filter out identified icon images from markdown content, enhancing the quality of stored images.
- Added unit tests for icon image detection to ensure accurate functionality across various image sizes.

These changes improve image handling by effectively filtering out non-essential images, leading to cleaner document processing.
2026-03-05 18:14:14 +08:00
wizardchen 1b8d7bd4de feat: add passage cleaning functionality for rerank model
- Introduced a new function `cleanPassageForRerank` to strip markdown and structural noise from text, enhancing semantic clarity for rerank models.
- Implemented comprehensive regex patterns to remove images, links, code blocks, LaTeX, HTML tags, and other formatting elements.
- Added unit tests to validate the cleaning process across various input scenarios, ensuring expected output consistency.
- Updated the `getEnrichedPassage` function to utilize the new cleaning method, improving the quality of text passed to the rerank model.

These changes significantly enhance the preprocessing of text data, leading to better performance in reranking tasks.
2026-03-05 18:14:14 +08:00
wizardchen b333499078 feat: implement parent-child chunking strategy for enhanced context management
- Added support for parent-child chunking in the knowledge base, allowing for a two-level chunking strategy that improves context retrieval.
- Introduced new configuration options for enabling parent-child chunking, including parent and child chunk sizes.
- Updated relevant components and services to handle parent chunk data, ensuring proper linking and retrieval of child chunks.
- Enhanced user interface elements to reflect the new chunking options and provide better user guidance.

These changes significantly improve the knowledge extraction process by providing richer context through hierarchical chunking.
2026-03-05 18:14:14 +08:00
hylaz 8399527022 fix: cleanupCtx 在启动时创建,可能在 shutdown 前过期 2026-03-05 11:17:51 +08:00
wizardchen a0952edc07 feat: release v0.3.2 with new features, improvements, and bug fixes 2026-03-04 11:07:21 +08:00