Added logic to the analyzeResponse function to handle cases where the LLM response is blocked by the content filter. This includes emitting events for the final answer and stopping the agent loop to prevent infinite responses. The implementation ensures that users receive a clear message when content is filtered, enhancing the agent's response management.
Implemented three critical fixes to enable wiki_config persistence during
KB creation and updates:
1. Add WikiConfig field to KnowledgeBaseConfig struct
- File: internal/types/knowledgebase.go
- Enables JSON unmarshaling of wiki_config from update requests
- Previously the field was silently dropped during deserialization
2. Update UpdateKnowledgeBase service to persist wiki_config
- File: internal/application/service/knowledgebase.go
- Added logic to apply wiki_config changes when config parameter is provided
- Mirrors existing logic for FAQConfig persistence
3. Update frontend TypeScript types for IDE completion
- File: frontend/src/api/knowledge-base/index.ts
- Added WikiConfig type to createKnowledgeBase function signature
- Added structured WikiConfig type to updateKnowledgeBase config parameter
- Enables TypeScript IDE autocomplete and type checking
Flow:
- Frontend: updateKnowledgeBase({name, description, config: {wiki_config}})
- Handler: Parses into UpdateKnowledgeBaseRequest via KnowledgeBaseConfig
- Service: Applies wiki_config to KB entity via UpdateKnowledgeBase
- Database: Persists wiki_config JSONB field to postgres
Root cause analysis:
Previously, the KnowledgeBaseConfig struct was missing the WikiConfig field,
causing JSON unmarshaling to silently drop any wiki_config data sent in
PUT requests. KB creation worked because it used the full KnowledgeBase struct.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit introduces a new "Recent Activity" section in the WikiBrowser component, allowing users to view the latest changes made to the wiki. The implementation includes:
- Addition of a toggleable recent activity log displaying the last five entries.
- Backend support for fetching the activity log from the knowledge base.
- Updates to the wiki ingest process to ensure draft pages are published post-ingest.
- Improvements to the deduplication process for entities and concepts during wiki page creation.
These enhancements aim to improve user engagement and provide better insights into wiki updates.
This commit introduces the complete wiki feature for WeKnora, enabling AI-powered wiki page generation and management. The implementation includes:
**Backend Changes:**
- Wiki data model: WikiPage type with support for multiple page types (Summary, Entity, Concept, Index, Log)
- Database schema: wiki_pages table with full migration support
- WikiPageService: CRUD operations and page management
- WikiPageRepository: GORM-based persistence layer
- Wiki ingest pipeline: Automated generation of wiki pages from knowledge documents
* Summary page generation using LLM
* Entity and concept extraction in a single LLM call
* Synthesis opportunity detection
* Index page rebuilding
* Log page maintenance
- Wiki boost feature: Enhance chat retrieval with wiki context
- Wiki linting: Maintenance and validation utilities
- Agent wiki tools: Enable agents to query and interact with wiki pages
- Wiki prompts: Comprehensive LLM prompt templates for all wiki generation tasks
- Language support: Reuse existing middleware language infrastructure for LLM prompts
**Frontend Changes:**
- Wiki browser UI: View all wiki pages with filtering and search
- Wiki API client: Knowledge base wiki management endpoints
- Knowledge base editor: Configure wiki settings (language, auto-ingest, synthesis model)
- i18n updates: Support for English, Korean, Russian, and Chinese interfaces
**Configuration:**
- Container DI: Wire up all wiki services
- Router: Register wiki API endpoints
- Task handling: Support async wiki ingest tasks
**Testing:**
- Unit tests for wiki page types
- Service layer tests
- Endpoint tests for wiki operations
- Integration tests with LLM mocking
**Documentation:**
- Language refactoring analysis and guides
- Implementation completion reports
- Quick reference guides for developers
**Key Features:**
✅ LLM-powered wiki page generation from documents
✅ Multi-language support (9+ languages)
✅ Automatic extraction of entities and concepts
✅ Synthesis opportunity detection
✅ Index and log page maintenance
✅ Progressive wiki building across multiple documents
✅ Agent-based wiki interaction
✅ Chat retrieval enhancement with wiki context
✅ Full frontend UI for wiki browsing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Raw byte truncation at 200 bytes splits multi-byte codepoints (Chinese
characters are 3 bytes), producing invalid UTF-8 that downstream filename
validation (utf8.ValidString) rejects with "文件名包含非法字符". Peel
trailing bytes via DecodeLastRuneInString until the last rune is complete.
Same latent bug and fix as the Yuque connector's sanitizeFileName.
Syncs personal and team Yuque knowledge bases into WeKnora as Markdown,
with full and incremental sync + deletion detection. Mirrors the Feishu
connector pattern; fetches V2Doc.body directly (no async export step).
Config: api_token (required) + base_url (optional, defaults to public cloud).
401/403 wraps ErrInvalidCredentials; 429 honors Retry-After; filters to
status=1 Doc-type at list stage to conserve API quota. Body is trusted as
Markdown for format=markdown|lake — empirically verified against the v2
API: Lake XML lives separately in body_lake, body itself is the Markdown
view. Other formats (e.g. html) are skipped with a placeholder item so
they surface in sync logs instead of silently feeding HTML to the parser.
v2Doc.status accepts both int and string shapes — Yuque's OpenAPI spec
declares string but the live API returns int. sanitizeFileName truncates
at a UTF-8 rune boundary so multi-byte titles don't emit invalid UTF-8
that downstream filename validation rejects.
Also adds logger.SetOutput test hook, used by the token-redaction test.
limit and offset are optional parameters with default values (limit=20,
offset=0). Marking them as required in the JSON Schema forces LLM clients
to always supply them, breaking calls that rely on the defaults.
Fixes#1003
- Adjusted the expanded TopK limit to a maximum of 200 candidates to enhance HNSW efficiency.
- Updated the vector retrieval query to ensure the ORDER BY expression matches the indexed expression, improving index usage.
- Implemented local settings for HNSW's ef_search and iterative_scan within a transaction to optimize candidate retrieval.
- Added fallback logic to handle unsupported GUCs gracefully, ensuring query execution continues even with older pgvector versions.
These changes improve the performance and reliability of vector retrieval in the PostgreSQL implementation.
- Updated the formatKnowledgeBaseList function to return knowledge base information as XML, improving data structure for integration.
- Modified the formatOutput function in GrepChunksTool to format search results as XML, enhancing readability and consistency.
- Adjusted the formatOutput method in KnowledgeSearchTool to output search results in XML format, providing a structured response.
- Refactored the buildOutput function in ListKnowledgeChunksTool to generate XML output, improving the clarity of chunk information.
These changes standardize output formats across multiple tools, facilitating better data handling and integration.
- Extend RetrieveEngineRegistry with byStoreID map for DB store engines
- Add StoreRegistry interface and EngineFactory function type
- Create engine factory with per-driver SDK client construction
- Load VectorStore records from DB into registry at startup
- Wire Create/Delete to dynamically register/unregister engines
- Add comprehensive unit tests for registry and service
- Updated the prompt template to clarify the task of understanding and rewriting user questions, improving the instructions for coreference resolution and ellipsis completion.
- Renamed sections for better clarity, changing "Rewriting Goals" to "Query Understanding."
- Added critical notes for intent classification regarding image and document attachments, ensuring accurate processing based on the presence of attachments.
- Improved examples to reflect the updated intent classification logic, enhancing the overall user experience in query handling.
- Added a new logger for LLM calls, enabling detailed logging of model interactions, including request and response data.
- Introduced configuration options for enabling and specifying the log directory for LLM debug logs.
- Implemented cleanup for old log files to manage disk space effectively.
- Wrapped existing chat implementations to log calls when debug logging is enabled, enhancing traceability for model interactions.
- Updated the SplitTextParentChild function to maintain a newParents slice, ensuring accurate parent indexing when splitting text into child chunks.
- This change enhances the integrity of parent-child relationships by correctly associating child chunks with their respective parents, improving overall chunk processing.
- Implemented a new method to validate the uniqueness of storage bucket names across tenants, ensuring no two tenants can use the same bucket name.
- Integrated this validation into the CreateTenant and UpdateTenant methods, enhancing data isolation and preventing potential conflicts.
- Improved error logging for better traceability when validation fails.
This update strengthens the integrity of tenant storage configurations.
- Added a check to set expandedTopK to params.TopK if it is lower, ensuring the subquery limit is not less than the final limit.
- Removed unnecessary type casting in the SQL query for embedding distance calculations, simplifying the query structure.
This update improves the accuracy of vector retrieval operations in the PostgreSQL repository.
Introduce KnowledgePostProcess task to decouple image multimodal
processing from downstream operations like summary, question
generation, and knowledge graph extraction.
This allows unifying the processing logic across documents and
images and ensures all text content, including from images,
is properly extracted before post-processing.
- Added isValidURL function to validate URLs before rendering links in the document content component, preventing invalid links from being clickable.
- Refactored timeout handling in KnowledgeBase.vue to use setTimeout instead of setInterval, improving performance and resource management.
- Ensured that timeouts are cleared appropriately to avoid memory leaks and unnecessary polling.
This update improves the robustness of URL handling and optimizes the component's performance.
- Added validation for bucket name format to ensure it meets specified criteria, returning an error message for invalid formats.
- Simplified error messages for MinIO client creation and bucket auto-creation failures, enhancing clarity for users.
- Updated success messages to reflect the correct status of bucket creation without public-read policy settings.
This update improves the robustness of the MinIO integration and enhances user feedback during storage checks.
Multiple fixes for WeKnora Lite mode (SQLite, no Redis):
1. Redis nil pointer panic on FAQ import (knowledge.go)
- Add in-memory fallback (sync.Map) for FAQ import progress tracking
- Add nil guards for running task locks and progress queries
2. seq_id not auto-incrementing in SQLite (chunk.go, tag.go)
- GORM autoIncrement on non-PK columns doesn't work in SQLite
- Add AssignChunkSeqIDs() to pre-assign seq_ids before batch insert
- Add BeforeCreate hook for KnowledgeTag
- Use Unscoped() to include soft-deleted records in MAX(seq_id) query
3. NOW() function not available in SQLite (chunk.go)
- Replace raw SQL NOW() with time.Now() or datetime('now')
- Use dialector check for raw SQL that must differ between PG and SQLite
4. FAQ KB should reject file uploads (knowledge.go)
- Add kb.Type == "faq" check in CreateKnowledgeFromFile
5. Linux/macOS build compatibility (Makefile)
- Conditionally apply macOS-only linker flag based on uname
6. Add SQLite integration tests (chunk_sqlite_test.go)
- TestCreateChunks_SQLite_SeqIDAutoAssigned
- TestCreateChunks_SQLite_SeqIDContinuesFromExisting
- TestCreateChunks_SQLite_SeqIDUniqueAcrossKBs
- TestCreateChunks_SQLite_SeqIDAfterSoftDelete
- TestKnowledgeTag_SQLite_SeqIDAutoAssigned
- TestUpdateChunk_SQLite_NoNOWError
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Modified the SQL clause in the KeywordsRetrieve function to utilize ParadeDB's ||| operator for improved token matching.
- Enhanced the handling of retrieval results in the fuseOrDeduplicate function to maintain original scores for keyword-only results, ensuring better performance in FAQ scenarios.
This update optimizes the keyword retrieval process and improves the accuracy of search results.
- Added WeKnoraCloudVLM struct to handle interactions with the WeKnoraCloud API, including image and text processing capabilities.
- Enhanced model configuration to include AppID and AppSecret for secure API access.
- Updated modelService to resolve WeKnoraCloud credentials and pass them to the VLM configuration.
- Implemented Predict method for sending requests to the WeKnoraCloud API and processing responses.
This update introduces a new integration with WeKnoraCloud, expanding the application's capabilities in handling multimedia content.
- Replaced references to Docreader credentials with WeKnoraCloud credentials across multiple services and handlers.
- Introduced a new CredentialsConfig structure to manage third-party provider credentials, specifically for WeKnoraCloud.
- Updated database schema to include a credentials column for storing WeKnoraCloud AppID and AppSecret.
- Enhanced methods for retrieving and utilizing WeKnoraCloud credentials, ensuring proper encryption and decryption during storage and retrieval.
This update improves the management of WeKnoraCloud credentials, streamlining the integration and enhancing security measures.
- Added tenantService to modelService for retrieving WeKnoraCloud credentials when not provided in model parameters.
- Implemented resolveWeKnoraCloudCredentials method to automatically fetch AppID and AppSecret from tenant configuration.
- Updated GetEmbeddingModel, GetEmbeddingModelForTenant, GetRerankModel, and GetChatModel methods to utilize the new credential resolution logic.
- Modified ModelParameters to encrypt and decrypt both APIKey and AppSecret for improved security during database operations.
This update streamlines credential management for WeKnoraCloud models, enhancing security and usability.
- Removed support for video file uploads across the application, including updates to the knowledge base and document processing services.
- Updated internationalization files to reflect the removal of video-related terminology, changing labels and descriptions to focus solely on audio processing.
- Adjusted file type handling to exclude video formats, ensuring users receive accurate feedback regarding supported file types.
- Enhanced error handling to provide clear messages when users attempt to upload unsupported video files.
This update streamlines the application by focusing on audio processing capabilities and improving user experience through clearer messaging.
- Changed 'Audio Processing' to 'Audio & Video' in English, Korean, Russian, and Chinese translations to reflect expanded functionality.
- Updated ASR configuration titles and descriptions to include video processing capabilities, enhancing user understanding of the feature.
- Renamed file type labels from 'Audio Files' to 'Audio & Video' across multiple languages for consistency and clarity.
- Enhanced document upload handling in the KnowledgeBase to filter unsupported file types, providing user feedback for filtered images, videos, and audio files.
This update improves the internationalization of the application and clarifies the capabilities of the ASR feature.
- Improved error handling in the attachment processing logic by providing detailed XML-like error messages for text, audio, and document processing failures.
- Updated the audio file content format to use XML tags for better structure, replacing the previous placeholder format.
This update enhances the clarity of error reporting and improves the overall structure of attachment content.
- Updated the query content to include metadata for uploaded images and attachments, improving the context provided to the assistant.
- Modified the attachment prompt format to use XML-like tags for better structure and readability, including metadata for file type and size.
- Enhanced error handling for unsupported file content extraction, providing clearer feedback to users.
This update improves the assistant's ability to process and respond to queries involving multimedia content.
- Introduced a new prompt template for document analysis, allowing the assistant to provide detailed responses based on attached documents.
- Updated the rewrite template descriptions to clarify the handling of image and document attachments, ensuring better user understanding of intent classification.
- Added a new intent type for document-only queries to improve response accuracy.
This update enhances the assistant's capabilities in processing and analyzing documents while refining the user experience in intent classification.
- Introduced a new WeKnoraCloudSettings component for managing APPID and APPSECRET.
- Added functionality to save WeKnoraCloud credentials without automatically creating models.
- Implemented credential status checks across various components to provide user feedback on configuration status.
- Updated API endpoints to handle credential saving and status retrieval.
- Enhanced UI elements to guide users in configuring WeKnoraCloud settings.
This update improves the integration of WeKnoraCloud by allowing users to manage their credentials more effectively and receive real-time feedback on their configuration status.
Wire VectorStoreService to HTTP with 8 endpoints: types metadata, CRUD
(create/list/get/update/delete), and connection testing (raw + by ID).
Register routes, DI container bindings, and add API documentation.
Remove the hardcoded TruncatePromptTokens: 512 limit from WeKnoraCloud
BatchEmbed to allow the API to use its default truncation behavior,
avoiding unexpected truncation of longer input texts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add VectorStoreService with CRUD validation, duplicate checking (DB + env
stores), and engine-specific health checks for 6 vector database types.
Include VectorStoreResponse DTO, env store builder, engine type metadata,
and comprehensive unit tests.