Commit Graph

43 Commits

Author SHA1 Message Date
wizardchen 33da010605 feat: Implement deadlock retry mechanism for chunk creation
Added a new utility function, WithDeadlockRetry, to handle MySQL deadlock errors during chunk creation. This function retries the operation up to three times with exponential backoff. Updated the CreateChunks method to utilize this new function, ensuring robust handling of concurrent write scenarios and preventing potential deadlocks. Additionally, improved documentation for CreateChunks to clarify the use of Omit for avoiding auto-increment column issues.
2026-04-22 21:17:21 +08:00
wizardchen 4f55994266 feat(prompt): enhance question rewriting and intent classification in prompt templates
- Updated the prompt template to clarify the task of understanding and rewriting user questions, improving the instructions for coreference resolution and ellipsis completion.
- Renamed sections for better clarity, changing "Rewriting Goals" to "Query Understanding."
- Added critical notes for intent classification regarding image and document attachments, ensuring accurate processing based on the presence of attachments.
- Improved examples to reflect the updated intent classification logic, enhancing the overall user experience in query handling.
2026-04-16 23:15:02 +08:00
wizardchen 53e8e17df4 feat(logger): implement LLM debug logging functionality
- Added a new logger for LLM calls, enabling detailed logging of model interactions, including request and response data.
- Introduced configuration options for enabling and specifying the log directory for LLM debug logs.
- Implemented cleanup for old log files to manage disk space effectively.
- Wrapped existing chat implementations to log calls when debug logging is enabled, enhancing traceability for model interactions.
2026-04-16 23:15:02 +08:00
swim2sun d2c6814f22 fix: SQLite/Lite mode compatibility for FAQ import and chunk operations
Multiple fixes for WeKnora Lite mode (SQLite, no Redis):

1. Redis nil pointer panic on FAQ import (knowledge.go)
   - Add in-memory fallback (sync.Map) for FAQ import progress tracking
   - Add nil guards for running task locks and progress queries

2. seq_id not auto-incrementing in SQLite (chunk.go, tag.go)
   - GORM autoIncrement on non-PK columns doesn't work in SQLite
   - Add AssignChunkSeqIDs() to pre-assign seq_ids before batch insert
   - Add BeforeCreate hook for KnowledgeTag
   - Use Unscoped() to include soft-deleted records in MAX(seq_id) query

3. NOW() function not available in SQLite (chunk.go)
   - Replace raw SQL NOW() with time.Now() or datetime('now')
   - Use dialector check for raw SQL that must differ between PG and SQLite

4. FAQ KB should reject file uploads (knowledge.go)
   - Add kb.Type == "faq" check in CreateKnowledgeFromFile

5. Linux/macOS build compatibility (Makefile)
   - Conditionally apply macOS-only linker flag based on uname

6. Add SQLite integration tests (chunk_sqlite_test.go)
   - TestCreateChunks_SQLite_SeqIDAutoAssigned
   - TestCreateChunks_SQLite_SeqIDContinuesFromExisting
   - TestCreateChunks_SQLite_SeqIDUniqueAcrossKBs
   - TestCreateChunks_SQLite_SeqIDAfterSoftDelete
   - TestKnowledgeTag_SQLite_SeqIDAutoAssigned
   - TestUpdateChunk_SQLite_NoNOWError

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:37:10 +08:00
ochan.kwon 33e919cc80 fix(knowledge): delete extracted images from storage when knowledge is removed
Previously, only the original uploaded file was deleted via knowledge.FilePath.
Images extracted by the builtin parser (stored as individual S3/MinIO/local files
via ImageResolver) were never cleaned up, causing orphan files to accumulate.

- Add ListImageInfoByKnowledgeIDs repository method (no chunk_type filter)
- Add collectImageURLs utility with URL deduplication and JSON parse logging
- Add deleteExtractedImages helper (standalone, best-effort with error logging)
- Fix DeleteKnowledge: collect image URLs before errgroup, delete in file goroutine
- Fix DeleteKnowledgeList: bulk collect with KB-level fileSvc mapping
- Fix cleanupKnowledgeResources: collect before chunk deletion, delete after
- Fix ProcessKBDelete: collect before chunk deletion for KB-wide cleanup
2026-04-02 12:04:04 +08:00
wizardchen 72dfb9ce75 feat(prompt): enhance context and intent classification in templates
- Updated context template to include runtime metadata such as current time and week, improving contextual awareness in user queries.
- Enhanced rewrite template with critical instructions for intent classification, ensuring that rewritten questions preserve essential entities and keywords.
- Refined intent classification logic to prioritize user intents more effectively, improving the accuracy of responses based on user queries.
- Added examples to clarify expected input and output formats for intent classification, enhancing usability for developers.
2026-03-26 18:57:12 +08:00
wizardchen 1960fc2d90 feat(suggested-questions): implement suggested questions feature for chat interface
- Added a new API endpoint to fetch suggested questions based on the agent's associated knowledge bases.
- Implemented frontend components to display suggested questions in both chat and create chat views, enhancing user interaction.
- Updated localization files to include translations for suggested questions in English, Korean, Russian, and Chinese.
- Introduced debounce logic for fetching suggestions to optimize performance during knowledge base changes.
- Enhanced backend services to support fetching and returning suggested questions, improving the overall chat experience.
2026-03-25 22:08:29 +08:00
wizardchen 9650188a35 feat(chunk): add FindFAQChunkWithDuplicateQuestion method for enhanced duplicate question checking
- Implemented a new method in the chunk repository to find FAQ chunks with overlapping standard or similar questions.
- Updated the knowledge service to utilize this method for checking duplicates during FAQ question creation.
- Improved error handling for duplicate questions, providing clearer feedback on conflicts.
2026-03-25 22:08:29 +08:00
wizardchen c25741673c feat(chunk): implement batch deletion for chunks to avoid MySQL placeholder limit
- Updated DeleteChunks method to split IDs into batches of 5000 to prevent MySQL Error 1390.
- Added unit tests for CalculateFAQContentHash to ensure consistent hashing behavior across various scenarios, including normalization and case sensitivity.
2026-03-19 18:20:09 +08:00
wizardchen 8db4b6cdea fix(chunk): update timestamp function in UpdateChunks method to use NOW() for consistency 2026-03-19 18:20:09 +08:00
wizardchen 125d3e5e4a feat: implement knowledge move functionality
- Added API endpoints for moving knowledge items between knowledge bases, including validation for source and target compatibility.
- Implemented asynchronous task handling for knowledge moves, tracking progress and status updates.
- Introduced new localization strings for move-related UI elements in English, Korean, Russian, and Chinese.
- Enhanced the frontend to support knowledge move operations, including target selection and progress monitoring.

These changes significantly improve the application's capabilities for managing knowledge across different bases, providing users with a seamless experience in organizing their knowledge assets.
2026-03-09 13:23:16 +08:00
wizardchen 927e5d4750 feat: add ListChunksByParentIDs method and enhance chunk merging logic
- Implemented ListChunksByParentIDs in chunkRepository to retrieve chunks based on multiple parent IDs, improving data retrieval efficiency.
- Enhanced the chunk merging logic in PluginMerge to better handle overlapping chunks and collect merged ImageInfo for parent chunks.
- Updated the ChunkRepository interface to include the new ListChunksByParentIDs method, ensuring consistency across the application.

These changes optimize chunk management and improve the handling of image information during merges, enhancing overall application functionality.
2026-03-05 18:14:14 +08:00
wizardchen 397689d2f3 feat: introduce WeKnora Lite edition with lightweight configuration and deployment
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.

This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
2026-03-02 21:21:49 +08:00
wizardchen 4158773849 feat: add organization management features
- Introduced a new API for organization management, including creation, updating, and deletion of organizations.
- Implemented organization sharing functionality for knowledge bases, allowing users to share knowledge bases with specific organizations and set permissions.
- Added UI components for organization selection and sharing dialogs.
- Enhanced existing components to support organization-related features, including shared knowledge base handling and organization role management.

This update enhances collaboration capabilities within the application by allowing users to manage and share knowledge bases across different organizations.
2026-02-09 11:40:25 +08:00
wizardchen 14d7d3d995 feat: 为 FAQ 和标签引入 seq_id 支持 2026-01-14 17:13:44 +08:00
wizardchen b71e7084fd feat: 将未分类FAQ条目转为"未分类"标签 2026-01-07 11:08:59 +08:00
wizardchen 3905b6fe5d feat: 优化知识块分页查询功能,支持按知识类型区分搜索和排序逻辑 2025-12-29 17:18:26 +08:00
wizardchen 0e5370c7c4 fix: 修复标签ID处理逻辑,正确处理空字符串与UntaggedTagID的判断条件 2025-12-25 22:07:42 +08:00
wizardchen a025ebe7fb feat: 优化标签ID处理逻辑,新增批量更新TagID功能 2025-12-25 22:07:42 +08:00
wizardchen 4c0846f9c7 feat: 增强标签删除功能,支持排除指定ID;新增异步索引删除任务优化删除流程 2025-12-25 22:07:42 +08:00
wizardchen 87ead9dec4 feat: 新增FAQ条目批量更新排除功能,支持在ByTag操作中排除指定ID列表 2025-12-25 22:07:42 +08:00
wizardchen 557081c8f8 fix: 修复不同数据库类型的JSON查询兼容性问题,支持MySQL和PostgreSQL的差异处理 2025-12-25 22:07:42 +08:00
wizardchen 1c1033101d feat: 增强FAQ搜索功能,支持指定搜索字段(标准问题/相似问法/答案) 2025-12-24 20:47:08 +08:00
wizardchen ec9d6ee961 fix: 修复GORM批量插入时零值字段被忽略的问题,确保IsEnabled和Flags字段正确插入 2025-12-24 20:47:08 +08:00
wizardchen 8430b9fa2d feat: 新增FAQ列表排序功能,支持按更新时间升序/降序排列;优化标签名称批量查询,提升查询效率 2025-12-24 20:47:08 +08:00
wizardchen 509582a4d5 feat: 知识库异步复制功能支持进度追踪和增量同步 2025-12-17 22:04:18 +08:00
wizardchen ad6bd157fd feat: 新增FAQ条目导出功能,支持CSV格式导出 2025-12-17 22:04:18 +08:00
wizardchen c461a59ecd feat: 新增FAQ推荐标签和批量操作支持,优化标签删除逻辑 2025-12-12 19:57:08 +08:00
wizardchen 16562eebbb feat: 添加AI问题生成功能,支持配置生成数量并在前端展示生成的问题 2025-12-03 01:42:31 +08:00
wizardchen f26cbe02e1 feat: Add synchronous creation of single FAQ entry and duplicate question detection, update API documentation and frontend handling 2025-12-02 21:14:24 +08:00
wizardchen 845d990a48 feat: Add tag management and FAQ management APIs, enhance knowledge base functionality with keyword search and custom file naming support 2025-12-02 20:39:59 +08:00
wizardchen eb8bfddfd1 refactor: Update function signatures across multiple files to improve readability by adding context parameters and enhancing code structure 2025-11-28 16:17:52 +08:00
wizardchen 64c44c9e27 refactor: Update tenant ID handling across the application 2025-11-25 16:30:54 +08:00
wizardchen 55d6bc7274 fix: Update chunk repository queries to use indexed chunk status 2025-11-24 15:50:55 +08:00
wizardchen e56233ef11 feat: Enhance chunk management and FAQ import functionality
- Updated chunk repository methods to support deletion of unindexed chunks and listing all FAQ chunks by knowledge ID.
- Introduced batch update functionality for chunks in the chunk service.
- Enhanced FAQ import logic to handle append and replace operations, improving efficiency and data integrity.
- Added content hash calculation for FAQ entries to facilitate quick matching and deduplication.
- Refactored related methods to streamline the import process and improve logging for better tracking of import tasks.
2025-11-24 15:47:42 +08:00
wizardchen 4a2be2b878 feat: Enhance file upload functionality with progress tracking
- Updated the `uploadKnowledgeFile` function to accept an optional progress callback, enabling real-time upload progress tracking.
- Refactored the document upload handling in the menu component to support multiple file uploads with individual progress events.
- Removed deprecated document processing state management from the KnowledgeBase view, streamlining the upload process.
- Improved user feedback during file uploads by dispatching events for upload start, progress, and completion.
2025-11-24 00:21:42 +08:00
wizardchen 057cfd9eb0 feat: Enhance FAQ import functionality and UI improvements
- Added support for tracking the status of FAQ import tasks, including progress indicators for ongoing imports.
- Introduced new localization strings for processing tasks and batch operations in multiple languages.
- Updated the FAQ entry manager to display import progress and improved user feedback during the import process.
- Enhanced the KnowledgeBase component to reflect the processing state of knowledge items, improving user experience during document uploads.
- Implemented batch deletion functionality for chunks, streamlining the management of knowledge base content.
2025-11-23 17:32:50 +08:00
wizardchen 9e98848d69 feat: Add statistics localization and enhance knowledge base functionality
- Introduced new localization strings for document count, FAQ entries, and chunk count in English, Russian, and Chinese.
- Updated the KnowledgeBase and KnowledgeBaseList components to display knowledge and chunk counts.
- Implemented backend functions to count knowledge items and chunks associated with each knowledge base, improving data management and user insights.
2025-11-21 01:39:43 +08:00
wizardchen fdf5d9d781 feat: Implement knowledge tag management and batch updates for FAQ entries
- Added API functions for listing, creating, updating, and deleting knowledge base tags.
- Introduced batch update functionality for enabling/disabling FAQ entries and updating their tags.
- Enhanced the KnowledgeBase.vue and FAQEntryManager.vue components to support tag management and batch operations.
- Updated localization files to include new strings related to tag management and batch operations.
2025-11-18 11:48:37 +08:00
wizardchen 822041d995 feat: add knowldege tags 2025-11-17 18:15:02 +08:00
wizardchen 25fdaed8ca feat: Implement FAQ functionality in Knowledge Base
- Added support for creating and managing FAQ knowledge bases, including new API endpoints for listing, upserting, updating, and deleting FAQ entries.
- Enhanced the frontend to allow users to create FAQ knowledge bases and manage FAQ entries with a dedicated interface.
- Introduced new types and configurations for FAQ handling in the backend, ensuring proper indexing and retrieval of FAQ entries.
- Updated localization files to include new strings related to FAQ management in multiple languages.
- Refactored existing components to accommodate the new FAQ features, improving user experience and functionality.
2025-11-15 01:30:17 +08:00
wizardchen 1eed11fc4f fix: ignore non-utf8 words when saving to db 2025-08-21 19:53:54 +08:00
wizardchen 56eb2bce33 init commit 2025-08-05 15:08:07 +08:00