- Adjusted the expanded TopK limit to a maximum of 200 candidates to enhance HNSW efficiency.
- Updated the vector retrieval query to ensure the ORDER BY expression matches the indexed expression, improving index usage.
- Implemented local settings for HNSW's ef_search and iterative_scan within a transaction to optimize candidate retrieval.
- Added fallback logic to handle unsupported GUCs gracefully, ensuring query execution continues even with older pgvector versions.
These changes improve the performance and reliability of vector retrieval in the PostgreSQL implementation.
- Extend RetrieveEngineRegistry with byStoreID map for DB store engines
- Add StoreRegistry interface and EngineFactory function type
- Create engine factory with per-driver SDK client construction
- Load VectorStore records from DB into registry at startup
- Wire Create/Delete to dynamically register/unregister engines
- Add comprehensive unit tests for registry and service
- Updated the prompt template to clarify the task of understanding and rewriting user questions, improving the instructions for coreference resolution and ellipsis completion.
- Renamed sections for better clarity, changing "Rewriting Goals" to "Query Understanding."
- Added critical notes for intent classification regarding image and document attachments, ensuring accurate processing based on the presence of attachments.
- Improved examples to reflect the updated intent classification logic, enhancing the overall user experience in query handling.
- Added a new logger for LLM calls, enabling detailed logging of model interactions, including request and response data.
- Introduced configuration options for enabling and specifying the log directory for LLM debug logs.
- Implemented cleanup for old log files to manage disk space effectively.
- Wrapped existing chat implementations to log calls when debug logging is enabled, enhancing traceability for model interactions.
- Implemented a new method to validate the uniqueness of storage bucket names across tenants, ensuring no two tenants can use the same bucket name.
- Integrated this validation into the CreateTenant and UpdateTenant methods, enhancing data isolation and preventing potential conflicts.
- Improved error logging for better traceability when validation fails.
This update strengthens the integrity of tenant storage configurations.
- Added a check to set expandedTopK to params.TopK if it is lower, ensuring the subquery limit is not less than the final limit.
- Removed unnecessary type casting in the SQL query for embedding distance calculations, simplifying the query structure.
This update improves the accuracy of vector retrieval operations in the PostgreSQL repository.
Introduce KnowledgePostProcess task to decouple image multimodal
processing from downstream operations like summary, question
generation, and knowledge graph extraction.
This allows unifying the processing logic across documents and
images and ensures all text content, including from images,
is properly extracted before post-processing.
Multiple fixes for WeKnora Lite mode (SQLite, no Redis):
1. Redis nil pointer panic on FAQ import (knowledge.go)
- Add in-memory fallback (sync.Map) for FAQ import progress tracking
- Add nil guards for running task locks and progress queries
2. seq_id not auto-incrementing in SQLite (chunk.go, tag.go)
- GORM autoIncrement on non-PK columns doesn't work in SQLite
- Add AssignChunkSeqIDs() to pre-assign seq_ids before batch insert
- Add BeforeCreate hook for KnowledgeTag
- Use Unscoped() to include soft-deleted records in MAX(seq_id) query
3. NOW() function not available in SQLite (chunk.go)
- Replace raw SQL NOW() with time.Now() or datetime('now')
- Use dialector check for raw SQL that must differ between PG and SQLite
4. FAQ KB should reject file uploads (knowledge.go)
- Add kb.Type == "faq" check in CreateKnowledgeFromFile
5. Linux/macOS build compatibility (Makefile)
- Conditionally apply macOS-only linker flag based on uname
6. Add SQLite integration tests (chunk_sqlite_test.go)
- TestCreateChunks_SQLite_SeqIDAutoAssigned
- TestCreateChunks_SQLite_SeqIDContinuesFromExisting
- TestCreateChunks_SQLite_SeqIDUniqueAcrossKBs
- TestCreateChunks_SQLite_SeqIDAfterSoftDelete
- TestKnowledgeTag_SQLite_SeqIDAutoAssigned
- TestUpdateChunk_SQLite_NoNOWError
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Modified the SQL clause in the KeywordsRetrieve function to utilize ParadeDB's ||| operator for improved token matching.
- Enhanced the handling of retrieval results in the fuseOrDeduplicate function to maintain original scores for keyword-only results, ensuring better performance in FAQ scenarios.
This update optimizes the keyword retrieval process and improves the accuracy of search results.
- Added WeKnoraCloudVLM struct to handle interactions with the WeKnoraCloud API, including image and text processing capabilities.
- Enhanced model configuration to include AppID and AppSecret for secure API access.
- Updated modelService to resolve WeKnoraCloud credentials and pass them to the VLM configuration.
- Implemented Predict method for sending requests to the WeKnoraCloud API and processing responses.
This update introduces a new integration with WeKnoraCloud, expanding the application's capabilities in handling multimedia content.
- Replaced references to Docreader credentials with WeKnoraCloud credentials across multiple services and handlers.
- Introduced a new CredentialsConfig structure to manage third-party provider credentials, specifically for WeKnoraCloud.
- Updated database schema to include a credentials column for storing WeKnoraCloud AppID and AppSecret.
- Enhanced methods for retrieving and utilizing WeKnoraCloud credentials, ensuring proper encryption and decryption during storage and retrieval.
This update improves the management of WeKnoraCloud credentials, streamlining the integration and enhancing security measures.
- Added tenantService to modelService for retrieving WeKnoraCloud credentials when not provided in model parameters.
- Implemented resolveWeKnoraCloudCredentials method to automatically fetch AppID and AppSecret from tenant configuration.
- Updated GetEmbeddingModel, GetEmbeddingModelForTenant, GetRerankModel, and GetChatModel methods to utilize the new credential resolution logic.
- Modified ModelParameters to encrypt and decrypt both APIKey and AppSecret for improved security during database operations.
This update streamlines credential management for WeKnoraCloud models, enhancing security and usability.
- Removed support for video file uploads across the application, including updates to the knowledge base and document processing services.
- Updated internationalization files to reflect the removal of video-related terminology, changing labels and descriptions to focus solely on audio processing.
- Adjusted file type handling to exclude video formats, ensuring users receive accurate feedback regarding supported file types.
- Enhanced error handling to provide clear messages when users attempt to upload unsupported video files.
This update streamlines the application by focusing on audio processing capabilities and improving user experience through clearer messaging.
- Changed 'Audio Processing' to 'Audio & Video' in English, Korean, Russian, and Chinese translations to reflect expanded functionality.
- Updated ASR configuration titles and descriptions to include video processing capabilities, enhancing user understanding of the feature.
- Renamed file type labels from 'Audio Files' to 'Audio & Video' across multiple languages for consistency and clarity.
- Enhanced document upload handling in the KnowledgeBase to filter unsupported file types, providing user feedback for filtered images, videos, and audio files.
This update improves the internationalization of the application and clarifies the capabilities of the ASR feature.
- Updated the query content to include metadata for uploaded images and attachments, improving the context provided to the assistant.
- Modified the attachment prompt format to use XML-like tags for better structure and readability, including metadata for file type and size.
- Enhanced error handling for unsupported file content extraction, providing clearer feedback to users.
This update improves the assistant's ability to process and respond to queries involving multimedia content.
- Introduced a new WeKnoraCloudSettings component for managing APPID and APPSECRET.
- Added functionality to save WeKnoraCloud credentials without automatically creating models.
- Implemented credential status checks across various components to provide user feedback on configuration status.
- Updated API endpoints to handle credential saving and status retrieval.
- Enhanced UI elements to guide users in configuring WeKnoraCloud settings.
This update improves the integration of WeKnoraCloud by allowing users to manage their credentials more effectively and receive real-time feedback on their configuration status.
Add VectorStoreService with CRUD validation, duplicate checking (DB + env
stores), and engine-specific health checks for 6 vector database types.
Include VectorStoreResponse DTO, env store builder, engine type metadata,
and comprehensive unit tests.
- Remove infrastructure/crypto package (CryptoService, config, PBKDF2-based encryption)
- Replace with lightweight utils.GetAESKey/EncryptAESGCM/DecryptAESGCM helpers
- Remove cryptoSvc dependency from modelService, tenantService, weKnoraCloudService
- Remove crypto state persistence logic from container initialization
- Add TruncatePromptTokens field to WeKnoraCloud embed request
- Update frontend hint text for WeKnoraCloud credential errors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add oss.go with full FileService implementation (SaveFile, SaveBytes,
GetFile, DeleteFile, GetFileURL, CheckConnectivity) using aws-sdk-go-v2
with path-style addressing for Aliyun OSS. Includes temp bucket support
and unit tests for parseOssFilePath and client creation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Go 1.24 has an internal compiler race condition on Windows that causes
'package without types' errors during parallel compilation. Setting
GOFLAGS="-p=1" serializes the build and avoids triggering the bug.
Introduce the VectorStore domain model as a foundation for multi-store
vector DB architecture (Phase 1, PR 1/4).
- VectorStore entity with ConnectionConfig (AES-GCM encrypted) and IndexConfig
- VectorStoreRepository interface and GORM implementation (CRUD + duplicate check)
- PostgreSQL migration (000031) and SQLite migration (000001)
- Unit tests for types, encryption, validation, and helpers
- Implement Validate, ListResources, FetchAll, FetchIncremental
- Per-record database sync with individual knowledge items
- Incremental sync with record-level change detection for databases
- Tree-structured resource listing with parent-child relationships
- Markdown renderer for all Notion block types including nested content
- Register NotionConnector in DI container
- Add unit tests for connector and markdown
- Enhanced the logic for replacing child content with parent content to only apply to text chunks whose parent is a parent_text chunk.
- Added checks to skip non-text chunks and ensure that only appropriate parent chunks are used for replacement, improving content quality and integrity.
- Implemented a new function to check ASR model connectivity via the /v1/audio/transcriptions endpoint.
- Updated ModelEditorDialog.vue to integrate ASR model checks and adjust UI elements accordingly.
- Enhanced localization files to reflect changes in ASR terminology and descriptions.
- Added ASR configuration options to the knowledge base and updated related migration scripts.
- Refactored model handling to support ASR as a distinct model type in the application.
jw_mac