Closes#620#497. Add opt-in Langfuse observability covering all five
model types (chat, embedding, rerank, VLM, ASR) with HTTP-request-scoped
traces and Docker Compose support (both cloud and self-hosted).
Core package internal/tracing/langfuse:
- HTTP client with batched async ingestion (non-blocking in request path)
- Sampling, environment / release tagging, and graceful fallback when
LANGFUSE_* env vars are absent (wrappers become no-ops)
- Gin middleware opens one trace per traced request and finishes it after
the handler chain returns, attaching method / path / user / session
- Trace context is stored under a typed key exported from internal/types
so logger.CloneContext can preserve it across handler / goroutine
boundaries (otherwise each LLM call auto-created an orphan trace,
fragmenting one request into many)
Per-model generation wrappers (opt-in via NewChat/NewEmbedder/...):
- chat: captures prompt, streaming output, token usage + TTFT
- embedding: approximates tokens when the provider omits usage
- rerank: previews query/docs, summarizes results to keep payload small
- vlm: records image count and total bytes, never uploads raw pixels
- asr: records file size and audio duration, never uploads audio bytes
Async title generation (GenerateTitleAsync) now forwards the trace key
into the goroutine so title calls appear under the parent chat trace.
Docker Compose:
- LANGFUSE_* env passthrough on the `app` service for cloud deployments
- Optional `langfuse` profile spins up a self-hosted Langfuse stack that
reuses WeKnora's existing PostgreSQL (separate database via an idempotent
init container that fixes ICU collation drift) and Redis (separate DB
number), adding only ClickHouse, MinIO, web and worker containers
- web/worker entrypoints URL-encode DB_PASSWORD / REDIS_PASSWORD at start
to avoid Prisma P1013 when passwords contain @ / # / etc.
Docs: docs/Langfuse集成.md covers cloud vs self-hosted, per-model usage
strategy, code map, and resource footprint.
The share-time validation forced every agent bound to a knowledge base to
have a rerank model configured, which blocked Wiki-type agents
(`wiki_search` / `wiki_read_page` / …) from being shared even though they
never invoke the reranker at runtime.
Align the check with the runtime logic in `session_agent_qa.go`: only
require a rerank model when the agent's allowed_tools include
`knowledge_search` (or rely on the default tool set, which contains it).
Wiki-only agents can now be shared without a rerank model; hybrid / RAG
agents still enforce the requirement.
Also update the handler error message to reflect the new rule.
Introduce per-model `custom_headers` config (similar to OpenAI Python SDK's
`extra_headers`) so users can inject gateway auth tokens, trace IDs, etc.
into every outbound API request for chat / embedding / rerank / VLLM / ASR
models. Reserved headers like Authorization / Content-Type are always
preserved to avoid breaking auth or signing flows.
Along the way, unify production and "test connection" paths onto a single
`ConfigFromModel(*types.Model, appID, appSecret)` constructor in each
model package. Both `service.modelService.GetXxxModel` and the four
`handler.initialization.Check*Model` / `TestEmbeddingModel` endpoints now
go through the same field mapping, so new parameters only need to be
added in one place.
Backend:
- Add `CustomHeaders map[string]string` to `types.ModelParameters` with
JSON/YAML `omitempty` tags for forward compatibility.
- New `internal/utils/extraheaders.go` providing `ApplyCustomHeaders`
(for hand-rolled HTTP paths) and `WrapHTTPClientWithHeaders` /
`CustomHeadersRoundTripper` (for SDK paths like go-openai). Reserved
headers (Authorization, api-key, Content-Type, Accept, Host,
Content-Length, User-Agent) are filtered out.
- Inject headers in every remote model path: chat (SDK + raw HTTP),
embedding (openai/aliyun/jina/volcengine/nvidia/azure_openai), rerank
(remote_api/aliyun/jina/nvidia/zhipu — via shared `customHeaderSetter`
interface), VLM and ASR (via http.Client wrapping).
- Add `ConfigFromModel` to each of chat/embedding/rerank/vlm/asr,
consolidating field mapping (ExtraConfig, CustomHeaders, InterfaceType
defaulting, WeKnoraCloud credentials, etc.) and cover it with
dedicated unit tests per package.
- Refactor `service/model.go`: replace ~100 lines of hand-written Config
literals with one-line `ConfigFromModel` calls; drop the now-unused
`stringMapToAnyMap` helper.
- Refactor `handler/initialization.go` test-connection endpoints: merge
four ad-hoc request structs into one `ModelTestRequest`, extract
`buildTestModel` and `resolveTenantWeKnoraCloudCreds` helpers, and
route all four endpoints through `ConfigFromModel` + `NewXxx` so the
test path is now behaviorally identical to production.
Frontend:
- Add `custom_headers` to the `ModelConfig` API type.
- `ModelEditorDialog.vue`: new Key-Value header editor (add/remove rows),
auto-converts map <-> array when loading/saving, and passes the
current header set to the `Test Connection` button so the preview
reflects exactly what production will send.
- `ModelSettings.vue`: serialize the header array back into a map for
the backend (drops empty rows).
- i18n: add custom-header labels / descriptions / placeholders to
zh-CN, en-US, ru-RU, ko-KR.
Tests:
- `internal/utils/extraheaders_test.go` covers reserved-header filtering
and round-tripper wrapping (including nil-client fallback).
- New `config_from_model_test.go` in chat / embedding / rerank / vlm /
asr verifies all fields (CustomHeaders, ExtraConfig, InterfaceType
defaulting, AppID/AppSecret) are propagated end-to-end.
- Introduced a unified capability requirements system for agent tools, ensuring consistent filtering of knowledge bases based on their capabilities.
- Implemented derived `kb_filter` logic to streamline compatibility checks between agent tools and knowledge bases, reducing redundancy in configuration.
- Updated frontend components to reflect new filtering logic, including improved handling of empty states and user feedback when no compatible knowledge bases are available.
- Enhanced internationalization support by adding relevant messages for tool compatibility issues across multiple languages.
- Introduced UpdateAutoLinkedContent method to persist changes from machine-only link decorators without incrementing the version number, ensuring user-facing revisions remain consistent.
- Updated existing methods to utilize the new auto-linked content update logic, enhancing the handling of cross-link injection and dead-link cleanup.
- Enhanced the KnowledgeBase and WikiBrowser components to improve user experience during wiki operations, including status tracking and indexing indicators.
- Added scheduling for wiki status probes to provide timely feedback on indexing processes after user-triggered actions.
- Updated the scrubWikiPendingIngest method to accept a reason parameter, improving logging clarity during wiki cleanup and reparse operations.
- Introduced prepareWikiForReparse to maintain the integrity of the pending ingest queue during reparse events, preventing stale operations from executing.
- Enhanced the mapping logic in the wiki ingest process to differentiate between stale and reparse updates, ensuring accurate handling of knowledge contributions.
- Improved logging to provide better insights into the number of reparse and stale slugs processed during wiki ingestion.
- Enhanced the UpdatePage and UpdateMeta methods to differentiate between user-visible content changes and bookkeeping updates, ensuring the version number is only incremented for actual content modifications.
- Updated the WikiPage struct and associated interfaces to reflect the new versioning policy, improving clarity on when the version is modified.
- Improved documentation for methods to clarify their intended use and behavior regarding versioning and metadata updates.
- Changed `kb_selection_mode` in agent type presets from "selected" to "all" to allow agents to access all knowledge bases by default.
- Revised prompt templates to clarify the handling of bound knowledge bases, replacing the `{{knowledge_bases}}` placeholder with a reference to the `<bound_knowledge_bases>` block in the user message's `<runtime_context>`.
- Enhanced internal logic to ensure that only searchable knowledge bases are considered during retrieval, improving the efficiency of knowledge searches.
- Added new functions for better formatting and handling of knowledge base metadata in the runtime context.
- Updated documentation and comments for clarity on the changes made to knowledge base interactions.
- Introduced extraction granularity settings in the wiki configuration, allowing users to control the level of detail in entity and concept extraction (focused, standard, exhaustive).
- Updated the UI to reflect these new options, including tooltips for better user guidance.
- Refactored the knowledge base editor to support the new extraction granularity feature, ensuring a seamless user experience.
- Improved the backend logic for wiki ingest tasks to utilize the new granularity settings, enhancing the accuracy of extracted content.
- Added tests to validate the new extraction granularity functionality and ensure robust performance across different configurations.
- Updated the knowledge base service to ensure WikiConfig is created when wiki indexing is enabled, enhancing backward compatibility.
- Removed legacy syncing of WikiConfig.Enabled to IndexingStrategy.WikiEnabled, streamlining the logic for wiki enablement.
- Introduced a custom retry delay function for asynq tasks to handle wiki ingest lock conflicts more effectively, improving task management.
- Cleaned up the container file by removing unnecessary whitespace, contributing to better code readability.
- Deleted the RebuildKnowledgeBaseIndex function from the knowledge service, which was responsible for reprocessing all documents in a knowledge base.
- Removed the RebuildIndex handler from the KnowledgeBaseHandler, eliminating the endpoint that triggered the rebuild process.
- Updated the router to remove the route for rebuilding the index, streamlining the knowledge base management functionality.
- Updated the DeleteKnowledge and DeleteKnowledgeList functions to skip vector store cleanup for knowledge entries without an embedding model, preventing unnecessary errors.
- Enhanced logging to provide clear information when skipping cleanup for knowledge without embeddings, improving overall robustness and clarity in the deletion process.
These changes ensure that the deletion process is more efficient and user-friendly, particularly for knowledge bases that do not utilize embeddings.
- Added a check to skip vector and keyword indexing for knowledge bases that do not require an embedding model, preventing errors during the indexing process.
- Ensured that chunks are marked as indexed even when skipping the indexing step, maintaining a consistent state for downstream processes.
- Enhanced logging to provide clear information on the indexing status and any issues encountered while updating chunk statuses.
These changes improve the robustness of the indexing process in the ImageMultimodalService, ensuring better handling of knowledge bases without embedding capabilities.
- Updated the agent system prompt to emphasize the use of regex for chunk searches, improving retrieval accuracy and efficiency.
- Introduced a new `Capabilities` field in the `KnowledgeBaseInfo` struct to define the retrieval surfaces available for each knowledge base, guiding the agent's strategy selection.
- Enhanced the `grep_chunks` tool to support regex queries, allowing for more flexible and powerful text pattern matching in knowledge base chunks.
- Improved JSON handling in the `RepairJSON` function to address invalid escape sequences, particularly for regex patterns, ensuring robust parsing and error handling.
- Updated documentation and comments across various files to clarify usage and expectations for regex and knowledge base capabilities.
These changes significantly enhance the agent's ability to retrieve relevant information while ensuring the integrity of input data handling.
- Added a new configuration file `agent_type_presets.yaml` to define various agent type presets, including RAG Q&A, Wiki Q&A, Hybrid (Wiki + RAG), and Data Analysis.
- Updated the `builtin_agents.yaml` to associate built-in agents with the new agent type presets, enhancing their configuration options.
- Enhanced the agent system prompt templates to reflect the new agent types and their functionalities.
- Implemented API endpoints to retrieve agent type presets, allowing for dynamic loading in the frontend.
- Updated the frontend to support agent type selection in the agent editor modal, improving user experience by auto-filling relevant configurations based on selected presets.
These changes significantly enhance the flexibility and usability of the agent system, allowing users to easily configure agents based on predefined types.
- Updated the SuggestedQuestion interface to include 'wiki' as a new source type, allowing for more diverse question origins.
- Implemented ListRecentForSuggestions method in the wikiPageRepository to retrieve recent wiki pages for fallback suggestions, improving the agent's ability to generate relevant questions when other sources are unavailable.
- Enhanced the customAgentService to utilize the new wiki page suggestions, ensuring a broader range of questions can be generated from wiki content.
These changes significantly improve the agent's suggestion capabilities by incorporating wiki content, enhancing user interaction and information retrieval.
- Enhanced the `cleanupWikiOnKnowledgeDelete` function to pass the full knowledge object, allowing for more accurate title and summary retrieval during wiki cleanup.
- Introduced a tombstone mechanism in Redis to mark recently deleted knowledge, enabling in-flight wiki ingest tasks to skip processing for deleted documents.
- Updated the `isKnowledgeGone` function to check for deleted knowledge using the tombstone, improving efficiency in handling concurrent delete and ingest operations.
- Added logic in the `wikiIngestService` to filter out updates for deleted knowledge, ensuring that no ghost references are created during the ingest process.
- Implemented lint checks for stale source references in the `WikiLintService`, allowing for automatic cleanup of pages referencing deleted knowledge.
These changes significantly enhance the robustness of the wiki system, ensuring consistency and accuracy in handling knowledge deletions and related wiki pages.
- Updated the agent system prompt to include new auxiliary tools and improved synthesis and issue flagging steps, enhancing the clarity of the workflow.
- Revised constraints to emphasize the importance of retrieving information from the wiki first and using skills when applicable.
- Expanded localization files for English, Korean, Russian, and Chinese to include new wiki tool descriptions and statuses, ensuring consistency across languages.
- Improved the agent editor modal to provide a clearer overview of tool statuses and retrieval preferences, enhancing user experience.
These changes significantly improve the functionality and usability of the agent system, particularly in relation to wiki interactions and tool management.
- Added the WikiScope struct to define the retrieval scope for wiki knowledge bases, allowing for optional filtering by specific document IDs.
- Implemented NewWikiScopesFromKBIDs constructor for easier creation of WikiScope instances.
- Enhanced the wikiReadPageTool and wikiSearchTool to utilize WikiScope, improving the handling of knowledge base IDs and document filtering during page retrieval.
- Updated the agent service to carry document whitelists into the wiki scopes, ensuring that only relevant pages are surfaced based on user mentions.
These changes significantly improve the flexibility and accuracy of wiki page retrieval, enhancing the overall user experience when interacting with wiki content.
- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings.
- The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities.
- Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system.
These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
- Introduced `enrichChunkImageInfo` function to populate image information for text chunks by retrieving data from child chunks, ensuring that image-related content is not lost during processing.
- Updated `reconstructContent` to `reconstructEnrichedContent`, allowing for the inclusion of OCR text and captions from images, improving the quality of content sent to the LLM.
- Added `writeDedupItemXML` and `xmlEscape` functions to format entity and concept entries as structured XML, enhancing clarity and preventing nonsensical merges in deduplication prompts.
- Refined the `deduplicateExtractedBatch` method to utilize the new XML formatting for better output consistency.
These changes significantly improve the handling of image data and the overall clarity of deduplication prompts, contributing to a more effective wiki content management system.
- Simplified the `InjectCrossLinks` and `injectCrossLinks` methods by utilizing a shared `linkifyContent` function, ensuring consistent handling of code blocks, existing links, and word boundaries.
- Replaced manual reference collection and sorting with a dedicated `collectLinkRefs` function to streamline the process of gathering link references from wiki pages.
- Enhanced the overall readability and maintainability of the code by removing redundant structures and logic.
These changes improve the efficiency and clarity of the cross-linking functionality within the wiki service, contributing to a more robust content management system.
- Modified the chat creation component to improve layout and responsiveness by adjusting the dialogue answers section.
- Refined the logic in the KnowledgePostProcessService to conditionally spawn wiki ingest tasks based on wiki configuration settings, enhancing task management.
- Added migration scripts for creating and dropping the `wiki_page_issues` table, facilitating better issue tracking within the wiki system.
These changes enhance the user interface for chat creation and improve the backend processing of knowledge base tasks, contributing to a more efficient and user-friendly experience.
- Introduced `encodeSlugPath` function to safely encode wiki slugs for API requests, preserving hierarchical routing.
- Updated API functions to utilize the new slug encoding for `getWikiPage`, `updateWikiPage`, and `deleteWikiPage`.
- Enhanced `ListBySourceRef` method in the repository to safely handle source knowledge IDs with JSON marshalling, improving query security.
- Added `escapeLikePattern` function to ensure safe concatenation of LIKE patterns in SQL queries, preventing unintended matches.
- Updated `KnowledgePostProcessService` to include Redis client for task management, improving performance in handling wiki ingest tasks.
- Enhanced `WikiLintIssue` structure to include `TargetSlug` for better issue tracking and auto-fixing capabilities.
These changes significantly improve the robustness and security of the wiki management system, enhancing user experience and data integrity.
- Updated the agent system prompt to refine the workflow for fixing issues, emphasizing the need to verify if issues still exist before making edits.
- Introduced a new `embeddedMode` prop in the frontend components to manage UI behavior based on the context of use.
- Simplified issue fix prompts in multiple languages for clarity, ensuring users receive concise instructions for resolving issues.
- Enhanced the WikiBrowser component to improve the display of issues and actions, including updated icons and streamlined interaction elements.
These changes significantly improve the user experience and functionality of the Wiki Fixer agent, fostering more efficient issue resolution and content management.
- Added a new built-in agent, "Wiki Fixer," designed to repair and optimize Wiki pages based on linter issues, with multilingual support for enhanced accessibility.
- Implemented a corresponding system prompt detailing the agent's role, mission, and workflow for effective issue resolution.
- Introduced new API functions for listing and updating wiki issues, allowing for better management of content conflicts and errors.
- Enhanced the frontend with new UI elements to display pending issues and facilitate auto-fixing, improving user interaction with the Wiki content.
These changes significantly enhance the agent's capabilities for maintaining the accuracy and quality of Wiki pages, fostering a more reliable knowledge base.
- Added a new tool, `wiki_flag_issue`, enabling users to report factual errors, mixed entities, or outdated information on wiki pages.
- Updated the agent system prompt and tool definitions to include the new tool, enhancing the agent's capabilities for maintaining wiki accuracy.
- Implemented backend functionality for creating, listing, and updating the status of flagged issues, ensuring effective tracking and resolution.
These changes significantly improve the agent's ability to manage and report issues within the wiki, fostering a more reliable knowledge base.
- Added a new tool, `wiki_read_source_doc`, allowing agents to access specific source documents for detailed information retrieval when wiki page content is insufficient.
- Updated the agent system prompt and tool definitions to incorporate the new tool, enhancing the agent's capabilities for in-depth knowledge extraction.
- Modified existing tools to support the new functionality, ensuring seamless integration within the agent's workflow.
These changes significantly improve the agent's ability to provide accurate and detailed responses by leveraging source documents alongside wiki content.
- Updated the agent system prompt to clarify the workflow for knowledge retrieval, emphasizing the use of `wiki_search` and `wiki_read_page` tools for specific queries and general overviews.
- Modified the `wikiReadPageTool` to support reading multiple wiki pages simultaneously, improving efficiency in fetching content.
- Enhanced the `wikiSearchTool` to utilize PostgreSQL POSIX regular expressions for more effective search queries, allowing for complex pattern matching.
- Adjusted the `wikiPageRepository` to replace `ILIKE` with regex matching for search queries, increasing the precision of search results.
These changes significantly improve the agent's ability to retrieve and synthesize information from the wiki, enhancing user interactions and response accuracy.
- Introduced a new built-in agent, "Wiki Researcher," designed for navigating and answering questions based on Wiki knowledge bases, complete with multilingual support.
- Added a corresponding system prompt that outlines the agent's role, mission, and workflow for effective knowledge graph traversal.
- Updated the agent configuration to include specific tools and parameters tailored for the Wiki Researcher, enhancing its functionality and user interaction.
- Removed deprecated wiki tools from the agent service to streamline the toolset and improve performance.
These changes significantly enhance the capabilities of the agent system, providing users with a specialized tool for in-depth research and information retrieval from Wiki sources.
- Updated the `WikiDeduplicationPrompt` to clarify the criteria for identifying exact duplicates, emphasizing strict matching of real-world entities and concepts.
- Improved the `appendLogEntry` method to include a summary parameter for better context in log entries, enhancing traceability of operations.
- Consolidated log entry creation for both retraction and ingestion operations, ensuring chronological clarity in the log.
- Implemented validation for slug merges to prevent mismatches based on entity types, improving the accuracy of deduplication.
These changes enhance the clarity and reliability of the wiki ingestion process, leading to more accurate content management.
- Introduced a new `WikiBatchContext` to share data across Map and Reduce phases, reducing redundant database queries.
- Replaced separate deduplication calls with a single `deduplicateExtractedBatch` method, improving efficiency by leveraging pre-loaded page data.
- Enhanced JSON handling by implementing a `cleanLLMJSON` function to sanitize LLM-generated output, ensuring safe parsing.
- Updated relevant methods to utilize the new context and deduplication logic, streamlining the overall ingestion workflow.
These changes enhance the performance and clarity of the wiki ingestion service, leading to more efficient content processing.
- Renamed `WikiPageUpdatePrompt` to `WikiPageModifyPrompt` to better reflect its functionality of adding new information and removing outdated content in a single operation.
- Enhanced the instructions for both `WikiPageModifyPrompt` and `WikiPageRetractPrompt` to clarify the handling of additions and retractions, ensuring accurate content updates.
- Consolidated the logic for enqueuing retraction tasks in the `cleanupWikiOnKnowledgeDelete` method to streamline the process of managing affected pages.
- Removed the `rebuildWikiIndexSimple` function to simplify the codebase, as its functionality is no longer needed with the new ingestion approach.
These changes improve the clarity and efficiency of the wiki content management process, enhancing overall system performance.
- Introduced `pending_tasks` and `is_active` fields in the `WikiStats` interface to track the number of tasks waiting for processing and the current activity status of the wiki ingestion.
- Updated localization files for English, Korean, Russian, and Chinese to include a new message for displaying the number of pending tasks in the wiki queue.
- Enhanced the `WikiBrowser` component to display the queue status, improving user awareness of ongoing tasks.
- Implemented polling logic to refresh wiki stats periodically when there are pending tasks or active ingestion, ensuring up-to-date information is presented.
These changes enhance the functionality and user experience of the wiki management system by providing real-time insights into task status.
- Modified the `extractEntitiesAndConcepts` method to return a map of slugs to extracted items, including aliases for improved wiki-link generation.
- Updated the `processOneDocument` method to handle slug items with aliases, enhancing the clarity of generated summaries.
- Adjusted the `injectCrossLinks` method to utilize match text for cross-linking, ensuring accurate references in content.
These changes improve the accuracy and usability of wiki links and summaries by incorporating alias management.
- Introduced a new `knowledgeSvc` interface to the `wikiIngestService`, allowing for improved document title retrieval based on knowledge ID.
- Updated the document processing logic to prioritize titles from the knowledge service, enhancing accuracy.
- Refactored content reconstruction to sort chunks by `StartAt` and `ChunkIndex`, improving the organization of the final output.
- Added functionality to append image information at the end of the content, ensuring a cleaner text flow.
These changes improve the efficiency and clarity of the wiki ingest process, enhancing overall content management.
- Enhanced the `renderMarkdown` function to support the new `[[slug|display name]]` format for wiki links, improving link clarity and usability.
- Adjusted CSS styles for wiki links to provide a more intuitive hover effect and visual distinction.
- Updated prompts in the `prompts_wiki.go` file to reflect the new wiki link syntax, ensuring consistency across the application.
These changes improve the overall user experience by making wiki links more informative and visually appealing.
- Updated the WikiSummaryPrompt to clarify the wiki-link syntax, specifying the use of slugs for referenced names.
- Modified the extractEntitiesAndConcepts method to return a mapping of slugs to display names, improving the accuracy of wiki-link references in summaries.
- Adjusted the summary content generation to include display names alongside slugs, enhancing the readability and context of generated summaries.
These changes improve the clarity and usability of wiki summaries by ensuring accurate and consistent link references.
- Introduced a new method to list all wiki pages without pagination, facilitating index rebuilding and cross-linking.
- Updated the wiki ingest and retract processes to include document summaries, improving clarity in log entries and index updates.
- Enhanced the cleanup process for deleted knowledge, ensuring accurate summary retention and index regeneration.
- Streamlined the handling of page updates and deletions, optimizing the overall wiki management workflow.
These changes improve the efficiency and reliability of wiki content management and indexing.
- Modified the EnqueueWikiIngest function to accept a Redis client, enabling the use of a Redis list for managing pending wiki ingest documents.
- Introduced a debouncing mechanism to optimize task processing, ensuring that multiple rapid uploads are handled efficiently without unnecessary locks.
- Enhanced the overall architecture for better performance and reliability in handling wiki ingest tasks.
These changes streamline the ingestion process and improve the handling of concurrent document uploads.
- Eliminated the detectSynthesisOpportunities function, which was responsible for suggesting synthesis opportunities based on page counts and types. This function was deemed unnecessary as it relied on heuristic methods without LLM integration.
- The removal streamlines the codebase and focuses on essential functionalities within the wiki_ingest service.
This change simplifies the service and prepares it for future enhancements that may involve more sophisticated synthesis logic.
- Modified the `rebuildWikiIndexSimple` function to include `knowledgeID` in the parameters, ensuring accurate logging of deleted pages.
- Enhanced the log entry format in `appendLogEntry` to include the action type and improved timestamp formatting, providing clearer context for actions taken on wiki pages.
- Adjusted the `ProcessWikiRetract` and `ProcessWikiIngest` methods to align with the new log entry structure, ensuring consistency across logging actions.
These changes improve the clarity and accuracy of wiki activity logs, facilitating better tracking of content changes.
- Updated the search logic in the WikiPage repository to use ILIKE for case-insensitive matching on title, content, summary, and slug fields, improving search accuracy.
- Added a watcher in the WikiBrowser component to reload all pages when the search query is cleared, enhancing user experience by ensuring relevant content is displayed immediately.
These changes collectively improve the search capabilities and responsiveness of the wiki interface.
Implemented new features in the WikiBrowser component to allow users to toggle the visibility of arrows on graph edges. This includes:
1. **Localization Updates**: Added translations for "Show arrows" and "Hide arrows" in English, Korean, Russian, and Chinese.
2. **Graph Rendering Enhancements**: Introduced logic to dynamically update SVG line attributes based on the toggle state, improving visual clarity in graph representations.
3. **State Management**: Added reactive state management for arrow visibility, ensuring seamless user interaction.
These changes enhance the user experience by providing more control over graph visualizations.
Added functionality to enable updates for the `wiki_config` field in the `KnowledgeBaseConfig` struct. This includes:
1. **Type Definition Update**: Added `WikiConfig` field to `KnowledgeBaseConfig` to allow JSON unmarshaling during update requests.
2. **Service Layer Enhancement**: Updated the `UpdateKnowledgeBase` method to persist changes to `wiki_config` when provided in the request.
3. **Frontend TypeScript Adjustments**: Enhanced type definitions in the frontend to support `wiki_config`, improving IDE autocomplete and type checking.
This fix addresses the critical issue where `wiki_config` was not being updated due to its absence in the `KnowledgeBaseConfig` struct, ensuring proper handling of wiki configurations during knowledge base updates.
Added a new utility function, WithDeadlockRetry, to handle MySQL deadlock errors during chunk creation. This function retries the operation up to three times with exponential backoff. Updated the CreateChunks method to utilize this new function, ensuring robust handling of concurrent write scenarios and preventing potential deadlocks. Additionally, improved documentation for CreateChunks to clarify the use of Omit for avoiding auto-increment column issues.
Implemented three critical fixes to enable wiki_config persistence during
KB creation and updates:
1. Add WikiConfig field to KnowledgeBaseConfig struct
- File: internal/types/knowledgebase.go
- Enables JSON unmarshaling of wiki_config from update requests
- Previously the field was silently dropped during deserialization
2. Update UpdateKnowledgeBase service to persist wiki_config
- File: internal/application/service/knowledgebase.go
- Added logic to apply wiki_config changes when config parameter is provided
- Mirrors existing logic for FAQConfig persistence
3. Update frontend TypeScript types for IDE completion
- File: frontend/src/api/knowledge-base/index.ts
- Added WikiConfig type to createKnowledgeBase function signature
- Added structured WikiConfig type to updateKnowledgeBase config parameter
- Enables TypeScript IDE autocomplete and type checking
Flow:
- Frontend: updateKnowledgeBase({name, description, config: {wiki_config}})
- Handler: Parses into UpdateKnowledgeBaseRequest via KnowledgeBaseConfig
- Service: Applies wiki_config to KB entity via UpdateKnowledgeBase
- Database: Persists wiki_config JSONB field to postgres
Root cause analysis:
Previously, the KnowledgeBaseConfig struct was missing the WikiConfig field,
causing JSON unmarshaling to silently drop any wiki_config data sent in
PUT requests. KB creation worked because it used the full KnowledgeBase struct.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
wizardchen