Commit Graph

129 Commits

Author SHA1 Message Date
wizardchen a655a25cf8 fix: revert to neo4j v6 and workaround Go 1.24 Windows compiler bug with -p=1
Go 1.24 has an internal compiler race condition on Windows that causes
'package without types' errors during parallel compilation. Setting
GOFLAGS="-p=1" serializes the build and avoids triggering the bug.
2026-04-11 22:08:02 +08:00
wizardchen 9525f08038 fix: downgrade neo4j driver from v6 to v5 to fix Go 1.24 Windows internal compiler bug 2026-04-11 21:59:03 +08:00
wizardchen 9a2b54536d fix: update environment configuration for debugging and logging, enhance SQLite FTS5 table management, and improve API base URL handling 2026-04-11 18:51:16 +08:00
ochan.kwon 69785ad0a3 feat: add VectorStore entity, repository, and migrations
Introduce the VectorStore domain model as a foundation for multi-store
vector DB architecture (Phase 1, PR 1/4).

- VectorStore entity with ConnectionConfig (AES-GCM encrypted) and IndexConfig
- VectorStoreRepository interface and GORM implementation (CRUD + duplicate check)
- PostgreSQL migration (000031) and SQLite migration (000001)
- Unit tests for types, encryption, validation, and helpers
2026-04-09 01:39:51 +08:00
wizardchen 9d021a29b7 feat(elasticsearch): enhance ID field handling with dynamic suffix detection
- Added logic to determine whether to append the ".keyword" suffix to ID fields based on index mapping.
- Introduced `detectFieldTypes` method to inspect index mappings and set the `useKeywordSuffix` flag accordingly.
- Updated methods to utilize the new `idField` function for consistent ID field querying across the repository.
- Improved logging for better visibility into mapping detection and suffix usage.
2026-04-02 15:26:34 +08:00
ochan.kwon 33e919cc80 fix(knowledge): delete extracted images from storage when knowledge is removed
Previously, only the original uploaded file was deleted via knowledge.FilePath.
Images extracted by the builtin parser (stored as individual S3/MinIO/local files
via ImageResolver) were never cleaned up, causing orphan files to accumulate.

- Add ListImageInfoByKnowledgeIDs repository method (no chunk_type filter)
- Add collectImageURLs utility with URL deduplication and JSON parse logging
- Add deleteExtractedImages helper (standalone, best-effort with error logging)
- Fix DeleteKnowledge: collect image URLs before errgroup, delete in file goroutine
- Fix DeleteKnowledgeList: bulk collect with KB-level fileSvc mapping
- Fix cleanupKnowledgeResources: collect before chunk deletion, delete after
- Fix ProcessKBDelete: collect before chunk deletion for KB-wide cleanup
2026-04-02 12:04:04 +08:00
ochan.kwon 258a786129 Revert "refactor(elasticsearch): remove ".keyword" suffix from ID fields in Elasticsearch queries"
This reverts commit 3daaabfc4a.
2026-04-01 20:41:24 +08:00
wizardchen a5ca9a2784 feat(web-search): refactor web search provider
- Updated relevant files to include provider registration, implementation, and metadata.
- Enhanced frontend components to support provider management and configuration.
- Added localization for new provider settings and messages.
- Implemented backend repository methods for CRUD operations on web search providers.
2026-03-31 20:22:34 +08:00
wizardchen 1a340a8374 refactor(datasource): remove WeCom document integration and related components
- Deleted WeCom document connector and callback handling code to streamline the datasource management.
- Removed associated localization entries and UI elements related to WeCom integration.
- Updated data source management logic to eliminate references to WeCom, enhancing maintainability and clarity.
- Adjusted error handling and logging to reflect the removal of WeCom-specific functionality.
2026-03-30 20:46:07 +08:00
wizardchen 53bd412cf6 fix(datasource): enhance error handling in ProcessSync method
- Improved logging for missing data sources, providing clearer warnings when a data source is not found.
- Updated sync log status to 'canceled' when a data source is deleted, ensuring accurate tracking of sync operations.
- Adjusted error handling to prevent unnecessary error returns, enhancing overall process stability.
2026-03-30 20:46:07 +08:00
wizardchen adabf8e089 feat(datasource): implement data source management functionality
- Added a new API for managing data sources, including CRUD operations and connection validation.
- Introduced UI components for data source settings, sync logs, and editor dialog, enhancing user experience in managing external data sources.
- Updated localization files to support new data source features in multiple languages.
- Implemented cron expression humanization for better scheduling visibility.
- Enhanced knowledge base editor to include data source management options.
2026-03-30 20:46:07 +08:00
wizardchen 1a954099f0 feat(knowledge): add SQL LIKE keyword escaping and metadata search functionality
- Introduced `escapeLikeKeyword` function to escape SQL LIKE wildcards in search queries, ensuring accurate keyword matching.
- Updated search queries to utilize the new escaping function for filtering by file name and title.
- Added `FindByMetadataKey` method to retrieve knowledge items based on key-value pairs in the metadata JSON column, enhancing search capabilities.
2026-03-27 16:30:07 +08:00
wizardchen 72dfb9ce75 feat(prompt): enhance context and intent classification in templates
- Updated context template to include runtime metadata such as current time and week, improving contextual awareness in user queries.
- Enhanced rewrite template with critical instructions for intent classification, ensuring that rewritten questions preserve essential entities and keywords.
- Refined intent classification logic to prioritize user intents more effectively, improving the accuracy of responses based on user queries.
- Added examples to clarify expected input and output formats for intent classification, enhancing usability for developers.
2026-03-26 18:57:12 +08:00
wizardchen 1960fc2d90 feat(suggested-questions): implement suggested questions feature for chat interface
- Added a new API endpoint to fetch suggested questions based on the agent's associated knowledge bases.
- Implemented frontend components to display suggested questions in both chat and create chat views, enhancing user interaction.
- Updated localization files to include translations for suggested questions in English, Korean, Russian, and Chinese.
- Introduced debounce logic for fetching suggestions to optimize performance during knowledge base changes.
- Enhanced backend services to support fetching and returning suggested questions, improving the overall chat experience.
2026-03-25 22:08:29 +08:00
wizardchen ae89871ac5 feat(knowledge): enhance search functionality to include title filtering and URL handling
- Updated search queries to filter by both file name and title when a keyword is provided, improving search accuracy.
- Added support for filtering by URL and HTML file types in search queries, enhancing the flexibility of the search functionality.
- Refactored query construction for file types to streamline the inclusion of additional conditions, ensuring robust handling of various file types.
2026-03-25 22:08:29 +08:00
wizardchen 3daaabfc4a refactor(elasticsearch): remove ".keyword" suffix from ID fields in Elasticsearch queries
- Updated Elasticsearch repository methods to use ID fields without the ".keyword" suffix, aligning with the new index mapping requirements.
- Enhanced documentation for the VectorEmbedding struct to clarify expected index mapping and usage guidelines for ID fields.
- Ensured consistency across all repository methods that interact with ID fields, improving query performance and maintainability.
2026-03-25 22:08:29 +08:00
wizardchen 9650188a35 feat(chunk): add FindFAQChunkWithDuplicateQuestion method for enhanced duplicate question checking
- Implemented a new method in the chunk repository to find FAQ chunks with overlapping standard or similar questions.
- Updated the knowledge service to utilize this method for checking duplicates during FAQ question creation.
- Improved error handling for duplicate questions, providing clearer feedback on conflicts.
2026-03-25 22:08:29 +08:00
ochan.kwon 8d8cc051be fix(tenant): prevent api_key encryption loss on tenant settings update
When a tenant setting is updated via PUT /tenants/kv/{key} (e.g.,
retrieval-config, parser-engine-config), the GORM AfterFind-decrypted
plaintext api_key was silently written back to the database because
db.Updates() does not trigger the BeforeSave hook.

Guard the repository's UpdateTenant to skip writing plaintext api_key
back to the database. Pre-encrypted values (enc:v1:… from CreateTenant
/ UpdateAPIKey) are written as-is; AfterFind-decrypted plaintext is
blanked so GORM skips the column entirely.

Closes #798
2026-03-25 21:25:21 +08:00
Dounx 8df12aeee2 fix: make Milvus vector metric type configurable via MILVUS_METRIC_TYPE 2026-03-25 21:20:49 +08:00
Dounx e11bd8c57d fix: use milvus result set length for empty search results 2026-03-20 18:22:40 +08:00
wizardchen c25741673c feat(chunk): implement batch deletion for chunks to avoid MySQL placeholder limit
- Updated DeleteChunks method to split IDs into batches of 5000 to prevent MySQL Error 1390.
- Added unit tests for CalculateFAQContentHash to ensure consistent hashing behavior across various scenarios, including normalization and case sensitivity.
2026-03-19 18:20:09 +08:00
wizardchen 8db4b6cdea fix(chunk): update timestamp function in UpdateChunks method to use NOW() for consistency 2026-03-19 18:20:09 +08:00
wizardchen 890dfb4bb3 feat: add clear session messages functionality
- Implemented a new API endpoint to clear all messages in a session, enhancing session management capabilities.
- Updated the frontend to include a clear messages option in the session menu, allowing users to easily remove chat history.
- Added localization support for the new feature in English, Korean, Russian, and Chinese, improving accessibility for a broader audience.
- Enhanced the chat view to handle session message clearing events, ensuring a responsive user experience.

These changes significantly improve the user experience by providing a straightforward method to manage session messages effectively.
2026-03-12 21:47:12 +08:00
wizardchen 4eed96f901 feat: enhance multimodal image processing and session management
- Updated configuration to support three tasks in the assistant's prompt: rewriting questions, classifying intent for knowledge base retrieval, and analyzing attached images.
- Implemented backend functionality to update image descriptions in user messages, ensuring that image context is preserved across conversation history.
- Modified session handling to return complete session data after updates, improving the accuracy of session information provided to users.
- Enhanced the chat pipeline to include detailed image descriptions and manage image captions effectively, enriching user interactions with visual content.

These improvements significantly enhance the application's multimodal capabilities, providing a more robust and user-friendly experience when handling image-based queries.
2026-03-12 13:17:47 +08:00
wizardchen bc2cb96d70 feat: enhance multimodal support and image handling
- Updated configuration to improve handling of multimodal queries, including clearer guidelines for when to use [NO_SEARCH] in responses.
- Enhanced the user interface to include dedicated sections for multimodal configuration, allowing users to manage image uploads and related settings more effectively.
- Added localization support for new multimodal features in both English and Chinese, improving accessibility for a broader audience.
- Implemented backend changes to support image analysis and extraction of OCR text, enriching the interaction experience for users.

These enhancements significantly improve the application's capability to process and respond to multimodal queries, fostering a more interactive and user-friendly environment.
2026-03-12 10:32:26 +08:00
wizardchen 125d3e5e4a feat: implement knowledge move functionality
- Added API endpoints for moving knowledge items between knowledge bases, including validation for source and target compatibility.
- Implemented asynchronous task handling for knowledge moves, tracking progress and status updates.
- Introduced new localization strings for move-related UI elements in English, Korean, Russian, and Chinese.
- Enhanced the frontend to support knowledge move operations, including target selection and progress monitoring.

These changes significantly improve the application's capabilities for managing knowledge across different bases, providing users with a seamless experience in organizing their knowledge assets.
2026-03-09 13:23:16 +08:00
wizardchen 33ccc41bea refactor: update storage size calculations for vector embeddings
- Revised the storage size calculation logic for Milvus, Qdrant, and Weaviate repositories to provide more accurate estimates based on the actual data structure and indexing methods.
- Updated comments to clarify the relationship between vector dimensions and index storage requirements, enhancing code readability and maintainability.
2026-03-09 13:23:16 +08:00
wizardchen 1fb2bedd2b feat: implement chat history and retrieval settings
- Added new API endpoints for managing chat history configuration and retrieval settings, allowing tenants to enable message indexing and configure search parameters.
- Introduced new Vue components for ChatHistorySettings and RetrievalSettings, providing a user-friendly interface for managing these configurations.
- Updated localization files to include new settings descriptions and labels in multiple languages.
- Enhanced the KnowledgeSearch view to support searching across both knowledge bases and chat history, improving the overall search functionality.

These changes significantly enhance the application's capabilities in managing and retrieving chat history, contributing to a more robust user experience.
2026-03-09 11:46:56 +08:00
MaoMengww e898d62141 refactor: change Tokenization Method 2026-03-09 10:36:16 +08:00
MaoMengww c4a5a4d99b feat: support weaviate vectordb for knowledge retrieve 2026-03-09 10:36:16 +08:00
DaWesen 65743da754 fix: 修复 Milvus retriever 相关问题
- 修正 envMilvusCollection 字段名拼写错误
- 将 log.Errorf 中的 %w 改为 %v,修复日志格式化问题
- 保留错误包装功能,确保错误处理的完整性
2026-03-06 10:33:09 +08:00
AndyYang 6dde1330c2 feat: add built-in MCP service support 2026-03-05 21:57:01 +08:00
wizardchen 927e5d4750 feat: add ListChunksByParentIDs method and enhance chunk merging logic
- Implemented ListChunksByParentIDs in chunkRepository to retrieve chunks based on multiple parent IDs, improving data retrieval efficiency.
- Enhanced the chunk merging logic in PluginMerge to better handle overlapping chunks and collect merged ImageInfo for parent chunks.
- Updated the ChunkRepository interface to include the new ListChunksByParentIDs method, ensuring consistency across the application.

These changes optimize chunk management and improve the handling of image information during merges, enhancing overall application functionality.
2026-03-05 18:14:14 +08:00
wizardchen 4ddc468fea feat: add GetUserByTenantID functionality to user repository and service
- Implemented GetUserByTenantID method in userRepository to retrieve the first user associated with a given tenant ID.
- Added corresponding GetUserByTenantID method in userService to facilitate user retrieval by tenant ID.
- Updated UserService and UserRepository interfaces to include the new method, ensuring consistency across the application.
- Enhanced auth middleware to store user information in context based on tenant ID, improving user management.

These changes enhance user retrieval capabilities by allowing access to user data based on tenant associations, improving overall application functionality.
2026-03-05 18:14:14 +08:00
wizardchen 00bf0460cc feat: add pinning functionality for knowledge bases
- Introduced a new API endpoint to toggle the pin status of knowledge bases, allowing users to pin important entries to the top of the list.
- Updated the frontend to include UI elements for pinning and unpinning knowledge bases, with corresponding success and error messages.
- Enhanced localization support by adding translations for pinning actions in English, Korean, Russian, and Chinese.
- Modified the database schema to include `is_pinned` and `pinned_at` fields for knowledge bases, enabling persistent pinning status.

These changes improve the organization and accessibility of knowledge bases, enhancing user experience by allowing quick access to prioritized entries.
2026-03-05 18:14:14 +08:00
wizardchen 931bd38138 feat(repository): add TagID and IsRecommended fields to vector embeddings
- Introduced `TagID` and `IsRecommended` fields across various repository implementations (Elasticsearch, Milvus, Postgres, Qdrant, SQLite) to enhance document categorization and recommendation capabilities.
- Updated the logic for handling `IsEnabled` to ensure it reflects the actual state from the source data.
- Adjusted the knowledge service to incorporate the new `IsRecommended` field, improving the indexing process for knowledge chunks.

These changes enhance the flexibility and functionality of document retrieval and indexing across the application.
2026-03-02 21:43:30 +08:00
wizardchen 7b8a78aa2a feat(repository): enhance document indexing with isEnabled and isRecommended fields
- Added `IsEnabled` and `IsRecommended` fields to the document indexing process in both ElasticFaiss and Elasticsearch repositories, improving the flexibility and functionality of document retrieval.
- Updated the data extraction process to include these new fields, ensuring they are correctly populated from the source data.
- Adjusted the indexing logic in the knowledge service to reflect these changes, maintaining consistency across the application.
2026-03-02 21:43:30 +08:00
wizardchen 284d6c623e feat: add deleteAllSessions functionality and update batch delete logic
- Introduced a new API function `deleteAllSessions` to allow deletion of all sessions for a tenant.
- Updated the `BatchManageDialog` component to support a confirmation dialog for deleting all sessions.
- Enhanced the menu component to reflect the new delete all functionality with appropriate confirmation messages.
- Added internationalization support for the new delete all confirmation message in multiple languages.

These changes improve session management by providing users with the ability to delete all sessions at once, enhancing the overall user experience.
2026-03-02 21:21:49 +08:00
wizardchen 397689d2f3 feat: introduce WeKnora Lite edition with lightweight configuration and deployment
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.

This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
2026-03-02 21:21:49 +08:00
joeyczheng 1b56e99cac feat: support milvus vectordb for knowledge retrieve
Signed-off-by: joeyczheng <joeyczheng@tencent.com>
2026-02-27 09:51:17 +08:00
joeyczheng 12baa885d2 feat: implement preview of user level memory graph
Signed-off-by: joeyczheng <joeyczheng@tencent.com>
2026-02-25 14:29:54 +08:00
AndyYang 172bf797ee feat(chat): add batch management for conversations 2026-02-25 14:23:05 +08:00
wizardchen cbe66aba56 feat(api): integrate per-organization resource counts into organization listing
- Added `resource_counts` to the `ListOrganizationsResponse` to include per-organization knowledge base and agent counts, enhancing the sidebar display.
- Removed the deprecated `getMeResourceCounts` API endpoint, consolidating resource count retrieval into the organization listing.
- Updated the organization store to manage and utilize the new resource counts effectively.
- Adjusted UI components to leverage the new resource counts structure, improving data retrieval and display for agents and knowledge bases.

This update streamlines resource count management and enhances the user experience by providing relevant data in a single API call.
2026-02-09 11:40:25 +08:00
wizardchen 9a073d62bf feat(api): enhance organization shared knowledge base and agent functionalities
- Introduced new interfaces for handling shared knowledge bases and agents, including visibility and selection modes for agents.
- Added API functions to list shared knowledge bases and agents within an organization, improving data retrieval for UI components.
- Updated existing components to utilize the new API endpoints and interfaces, ensuring accurate representation of shared resources.
- Enhanced error handling in API calls to provide clearer feedback on failures.

This update improves the functionality and user experience by providing better access to shared resources and enhancing the overall data structure within the application.
2026-02-09 11:40:25 +08:00
wizardchen c806eaf7f9 feat(agent): enhance agent and skill management features
- Updated the API to include disabled agent IDs for better management of user-specific agent visibility.
- Introduced a new function to set the disabled state for shared agents, allowing users to manage their shared agent preferences.
- Added a new Skills API to retrieve available skills, enhancing the overall agent capabilities.
- Improved the AgentSelector and related components to filter out disabled agents, ensuring a cleaner user experience.
- Updated localization strings to reflect new features and improve clarity in agent and skill management.

This update enhances the user experience by providing better control over agent visibility and introducing new skill management functionalities.
2026-02-09 11:40:25 +08:00
wizardchen dfaf862bc0 feat(agent): enhance agent sharing and selection features
- Updated the AgentSelector component to improve agent selection logic, ensuring accurate representation of selected agents.
- Introduced new AgentShareSettings component for managing agent sharing with organizations, including UI for selecting organizations and displaying shared agents.
- Enhanced API interactions for sharing agents, including new methods for listing and managing agent shares.
- Improved localization support for agent sharing features across multiple languages.
- Updated various components to reflect changes in agent capabilities and sharing status, enhancing user experience and clarity.

This update strengthens the agent management features, providing users with better tools for sharing and selecting agents within organizations.
2026-02-09 11:40:25 +08:00
wizardchen b3d743a0a9 feat(kbshare): enhance sharing functionality with soft-delete handling and shared access
- Updated ListByOrganization and ListSharedKBsForUser methods to exclude soft-deleted knowledge bases and organizations, improving data integrity.
- Modified knowledge fetching methods to include documents from shared knowledge bases the user has access to, enhancing user experience.
- Improved comments for clarity on functionality and permission checks in UpdateSharePermission method.

This update strengthens the sharing capabilities and ensures users only see relevant knowledge bases, enhancing overall application usability.
2026-02-09 11:40:25 +08:00
wizardchen 342e56735f feat(avatar): enhance SpaceAvatar component with emoji support
- Updated SpaceAvatar component to allow emoji-based avatars, enabling users to select emojis as their space avatar.
- Added new props and computed properties to handle emoji logic and display.
- Enhanced UI to include an emoji picker and clear option for avatars.
- Updated localization strings in English and Chinese for new avatar features.
- Adjusted styles for emoji display and interaction.

This update improves user personalization options for avatars, enhancing the overall user experience in the application.
2026-02-09 11:40:25 +08:00
wizardchen 72173432c9 feat(organization): enhance searchable organizations with member limit and search by ID
- Added a member limit field to the SearchableOrganizationItem interface, allowing organizations to specify a maximum number of members.
- Updated API and service layers to support searching organizations by name, description, or ID.
- Enhanced UI components to display member limits and updated localization strings in English and Chinese for new features.
- Included migration files to add the searchable column to the organizations table.

This update improves the organization search functionality and membership management, enhancing user experience and administrative control.
2026-02-09 11:40:25 +08:00
wizardchen ca1b3da1df feat(organization): add member limit feature
- Introduced a new member limit feature for organizations, allowing a maximum number of members to be set (default is 50; 0 means unlimited).
- Updated API endpoints to handle member limit during organization creation and updates.
- Implemented validation to prevent adding members when the limit is reached.
- Enhanced UI components to display and manage member limits, including localization updates for English and Chinese.
- Added migration files to support the new member limit column in the organizations table.

This update improves organization management by allowing administrators to control membership size.
2026-02-09 11:40:25 +08:00