Commit Graph

140 Commits

Author SHA1 Message Date
wizardchen 3450ed5e41 feat(knowledge): offload manual processing to async tasks 2026-03-18 21:56:34 +08:00
wizardchen 4e2583e58d feat: add language support to summary generation payload
- Enhanced the SummaryGenerationPayload struct to include a Language field for better localization.
- Updated enqueueSummaryGenerationTask and ProcessSummaryGeneration methods to handle language context in summary generation tasks.
2026-03-17 22:23:24 +08:00
wizardchen 139a9c40ff feat: enhance localization and configuration support
- Added timezone and language settings to the environment configuration.
- Introduced built-in agent configurations with multilingual support for various agents.
- Updated Docker Compose to utilize new environment variables for timezone and language.
- Created new prompt templates for question generation, summary generation, and keywords extraction.
2026-03-17 22:23:24 +08:00
wizardchen 2fd6e226a5 fix: sanitize tab chars and avoid double .md extension in manual download filename
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 18:18:23 +08:00
wizardchen e0c1b77b70 fix: quote Content-Disposition filename and add empty-title fallback
- Use mime.FormatMediaType for both attachment and inline Content-Disposition
  headers in DownloadKnowledgeFile and the preview handler, preventing header
  injection via a user-controlled knowledge title (e.g. containing ';' or '=')
- Extract inline sanitize logic into sanitizeManualDownloadFilename helper and
  add a blank/whitespace-only title fallback to "untitled" so the download
  never produces a degenerate ".md" filename

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 18:01:07 +08:00
wizardchen 63caf564ee feat: support manual knowledge download in GetKnowledgeFile
Add an early-return branch that streams manual knowledge content
(stored as Markdown in Metadata.content) directly as a .md file,
avoiding the runtime error that occurred when FilePath is empty.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 17:49:45 +08:00
wizardchen a6f77cec14 feat: enhance knowledge update functionality
- Added support for updating the description field in knowledge records, improving the detail and context of knowledge entries.
- Refactored knowledge base update logic to ensure configuration updates are only applied when provided, enhancing robustness and preventing unnecessary overwrites.
- Modified the request structure to make the configuration field optional, allowing for more flexible updates.

These changes improve the knowledge management system by allowing for more comprehensive updates and better handling of configuration data.
2026-03-16 02:26:47 +08:00
wizardchen cc8a99f0a3 feat: enhance storage engine configuration and user notifications
- Updated the summary length requirement for document summarization to 100-300 words.
- Added new localization strings for missing storage engine notifications in English, Korean, Russian, and Chinese.
- Implemented UI warnings for users regarding storage engine configuration when files exist, including confirmation dialogs for storage engine changes.
- Enhanced backend validation to ensure a storage engine is configured before allowing content uploads.

These changes improve user experience by providing clearer guidance on storage engine requirements and enhancing the overall functionality of the knowledge base management system.
2026-03-16 02:26:47 +08:00
ChenRussell 166f106b7f fix: ensure builtin engine handles simple formats correctly
Builtin engine was incorrectly falling back to simple engine for md/txt/csv files. Added explicit builtin engine check in selection logic.

Fixes #755
2026-03-15 23:58:49 +08:00
wizardchen 125d3e5e4a feat: implement knowledge move functionality
- Added API endpoints for moving knowledge items between knowledge bases, including validation for source and target compatibility.
- Implemented asynchronous task handling for knowledge moves, tracking progress and status updates.
- Introduced new localization strings for move-related UI elements in English, Korean, Russian, and Chinese.
- Enhanced the frontend to support knowledge move operations, including target selection and progress monitoring.

These changes significantly improve the application's capabilities for managing knowledge across different bases, providing users with a seamless experience in organizing their knowledge assets.
2026-03-09 13:23:16 +08:00
wizardchen 1fb2bedd2b feat: implement chat history and retrieval settings
- Added new API endpoints for managing chat history configuration and retrieval settings, allowing tenants to enable message indexing and configure search parameters.
- Introduced new Vue components for ChatHistorySettings and RetrievalSettings, providing a user-friendly interface for managing these configurations.
- Updated localization files to include new settings descriptions and labels in multiple languages.
- Enhanced the KnowledgeSearch view to support searching across both knowledge bases and chat history, improving the overall search functionality.

These changes significantly enhance the application's capabilities in managing and retrieving chat history, contributing to a more robust user experience.
2026-03-09 11:46:56 +08:00
ochan.kwon edcf7116cd feat: use language-neutral English prompts for multilingual LLM responses
Convert all LLM-facing prompts, templates, and tool output labels to
language-neutral English so the model responds in the user's input
language. Update frontend relevance/match-type mappings to match.

Output language strategy:
- User-facing: reply in user's input language
- Vector-indexed: match source document language
- LLM-internal: fixed English
2026-03-06 21:08:12 +08:00
wizardchen b697d4945e feat: update chunk size configurations for knowledge base processing
- Increased default sizes for parent and child chunks in the knowledge base editor and settings, enhancing data processing capabilities.
- Adjusted slider settings in the UI to reflect new minimum and maximum values for chunk sizes, improving user control over configurations.
- Updated backend logic to ensure consistent handling of chunk sizes across the application.

These changes optimize the chunking process, allowing for more efficient data handling and improved performance in knowledge base operations.
2026-03-05 18:14:14 +08:00
wizardchen b333499078 feat: implement parent-child chunking strategy for enhanced context management
- Added support for parent-child chunking in the knowledge base, allowing for a two-level chunking strategy that improves context retrieval.
- Introduced new configuration options for enabling parent-child chunking, including parent and child chunk sizes.
- Updated relevant components and services to handle parent chunk data, ensuring proper linking and retrieval of child chunks.
- Enhanced user interface elements to reflect the new chunking options and provide better user guidance.

These changes significantly improve the knowledge extraction process by providing richer context through hierarchical chunking.
2026-03-05 18:14:14 +08:00
wizardchen 4192cfd072 refactor: simplify image handling in _resolve_images function
- Removed the image storage handling from the `_resolve_images` function, which now only decodes images and returns them as inline bytes.
- Updated the function's docstring to clarify that image persistence is managed by the Go App, and the return value for `image_dir_path` is always empty.
- Adjusted related gRPC and protobuf files to remove the `ConvertToPDF` method and its associated types, streamlining the API.
- Enhanced security measures in the frontend by implementing a placeholder for provider images and ensuring proper hydration of protected file images.

These changes improve the clarity and efficiency of image processing and API interactions within the application.
2026-03-03 22:23:23 +08:00
wizardchen 931bd38138 feat(repository): add TagID and IsRecommended fields to vector embeddings
- Introduced `TagID` and `IsRecommended` fields across various repository implementations (Elasticsearch, Milvus, Postgres, Qdrant, SQLite) to enhance document categorization and recommendation capabilities.
- Updated the logic for handling `IsEnabled` to ensure it reflects the actual state from the source data.
- Adjusted the knowledge service to incorporate the new `IsRecommended` field, improving the indexing process for knowledge chunks.

These changes enhance the flexibility and functionality of document retrieval and indexing across the application.
2026-03-02 21:43:30 +08:00
wizardchen 7b8a78aa2a feat(repository): enhance document indexing with isEnabled and isRecommended fields
- Added `IsEnabled` and `IsRecommended` fields to the document indexing process in both ElasticFaiss and Elasticsearch repositories, improving the flexibility and functionality of document retrieval.
- Updated the data extraction process to include these new fields, ensuring they are correctly populated from the source data.
- Adjusted the indexing logic in the knowledge service to reflect these changes, maintaining consistency across the application.
2026-03-02 21:43:30 +08:00
wizardchen dbab7b0405 feat: introduce storage provider configuration and enhance backward compatibility
- Added a new `StorageProviderConfig` struct to the `KnowledgeBase` for improved storage provider management, separating provider selection from storage credentials.
- Updated the `UnmarshalJSON` method to auto-populate `StorageProviderConfig` from the legacy `StorageConfig` for backward compatibility.
- Refactored various service methods to utilize the new storage provider configuration, ensuring a smooth transition from the legacy system.
- Implemented database migrations to add the `storage_provider_config` column and migrate existing data, maintaining compatibility with historical knowledge bases.

This update enhances the clarity and flexibility of storage configurations while ensuring compatibility with existing data structures.
2026-03-02 21:21:49 +08:00
wizardchen 393ebf9f92 feat: standardize storage configuration and enhance backward compatibility
- Updated the `KnowledgeBase` struct to rename `cos_config` to `storage_config` for consistency across the application.
- Implemented a custom `UnmarshalJSON` method to maintain backward compatibility with legacy responses that still use `cos_config`.
- Modified various frontend components to reflect the new `storage_config` naming, ensuring seamless integration with the updated backend structure.
- Enhanced Dockerfiles and service configurations to support the new storage engine setup.

This update improves the clarity of storage configurations and ensures compatibility with existing data structures.
2026-03-02 21:21:49 +08:00
wizardchen 397689d2f3 feat: introduce WeKnora Lite edition with lightweight configuration and deployment
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.

This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
2026-03-02 21:21:49 +08:00
begoniezhao daf7dee4d2 refactor: Unify /url route and file URL handling 2026-02-26 19:46:08 +08:00
begoniezhao e24bfb05a2 feat: Add knowledge creation support from remote file URLs 2026-02-26 19:46:08 +08:00
begoniezhao bce968833e feat: Add async knowledge re-parse API and examples 2026-02-11 12:27:55 +08:00
wizardchen dfaf862bc0 feat(agent): enhance agent sharing and selection features
- Updated the AgentSelector component to improve agent selection logic, ensuring accurate representation of selected agents.
- Introduced new AgentShareSettings component for managing agent sharing with organizations, including UI for selecting organizations and displaying shared agents.
- Enhanced API interactions for sharing agents, including new methods for listing and managing agent shares.
- Improved localization support for agent sharing features across multiple languages.
- Updated various components to reflect changes in agent capabilities and sharing status, enhancing user experience and clarity.

This update strengthens the agent management features, providing users with better tools for sharing and selecting agents within organizations.
2026-02-09 11:40:25 +08:00
wizardchen 4158773849 feat: add organization management features
- Introduced a new API for organization management, including creation, updating, and deletion of organizations.
- Implemented organization sharing functionality for knowledge bases, allowing users to share knowledge bases with specific organizations and set permissions.
- Added UI components for organization selection and sharing dialogs.
- Enhanced existing components to support organization-related features, including shared knowledge base handling and organization role management.

This update enhances collaboration capabilities within the application by allowing users to manage and share knowledge bases across different organizations.
2026-02-09 11:40:25 +08:00
wizardchen 14ad38fd83 fix: tighten SSRF protection for URL imports 2026-01-27 11:09:53 +08:00
wizardchen faf1c9a37b feat: add matched question field to FAQ search results 2026-01-26 16:32:18 +08:00
wizardchen 2a99e2458d feat: add SSRF protection for URL fetching 2026-01-21 19:13:14 +08:00
wizardchen e55cbaa057 fix: switch to append mode on retry to prevent data loss 2026-01-21 19:13:14 +08:00
wizardchen f183b5c598 feat: offload large FAQ imports to object storage 2026-01-21 19:13:14 +08:00
wizardchen a6ba12f389 fix: switch to append mode on retry to prevent data loss 2026-01-20 18:19:44 +08:00
wizardchen 4636bca2b7 perf: optimize FAQ security check with concurrency 2026-01-20 18:19:44 +08:00
wizardchen 5665fb00cb refactor: store FAQ import task info instead of ID 2026-01-20 18:19:44 +08:00
wizardchen b74545b007 fix: check existing progress before saving 2026-01-20 18:19:44 +08:00
wizardchen b969681269 fix: only cleanup unused tags in replace mode 2026-01-20 18:19:44 +08:00
wizardchen cb9369b16b feat: support large FAQ imports via object storage 2026-01-20 18:19:44 +08:00
wizardchen a9df80b5db feat: offload large FAQ imports to object storage 2026-01-20 18:19:44 +08:00
wizardchen 0ea9acfc2f feat: 支持删除文档类型标签 2026-01-15 19:38:16 +08:00
wizardchen 48a0f9b508 Merge remote-tracking branch 'github_public/main' 2026-01-15 14:39:35 +08:00
wizardchen d530a02cfe fix: 修复FAQ更新时的重复检查逻辑 2026-01-15 14:34:24 +08:00
Xia Dong cec722aa43 feat(knowledge): Add tag_id support for knowledge creation and file uploads
- Introduced tag_id parameter in various knowledge-related API functions to allow categorization of knowledge entries.
- Updated frontend components to handle tag_id during knowledge creation and file uploads.
- Enhanced UI store to manage the selected tag ID for better user experience in categorizing knowledge.
2026-01-14 19:39:35 +08:00
wizardchen 3f25d36c46 perf: 优化FAQ索引更新和删除性能 2026-01-14 17:30:04 +08:00
wizardchen e3ac490c8e fix: 修复 FAQ 导入时清理 Tag 的逻辑 2026-01-14 17:19:21 +08:00
wizardchen 1c899a6f2c fix: 修复FAQ条目标签变更时未更新的问题 2026-01-14 17:19:05 +08:00
wizardchen dae772dd17 fix: 确保"未分类"标签排在最前 2026-01-14 17:18:45 +08:00
wizardchen 858d8875e8 fix: 修复切片越界导致的潜在崩溃 2026-01-14 17:18:35 +08:00
wizardchen 6c9d31496f feat: 添加FAQ导入成功条目详情展示 2026-01-14 17:16:53 +08:00
wizardchen 9338603c20 feat: 支持向 FAQ 条目添加相似问 2026-01-14 17:14:06 +08:00
wizardchen 14d7d3d995 feat: 为 FAQ 和标签引入 seq_id 支持 2026-01-14 17:13:44 +08:00
wizardchen 84a4ec62de refactor: 简化FAQ导入结果获取逻辑
将导入结果字段合并到FAQImportProgress结构体中,当状态为completed时自动填充。删除了独立的GetLastFAQImportResult接口和相关方法,客户端改为通过进度接口获取已完成任务的结果,并使用localStorage记录最后完成的任务ID。
2026-01-14 17:09:19 +08:00