Closes#620#497. Add opt-in Langfuse observability covering all five
model types (chat, embedding, rerank, VLM, ASR) with HTTP-request-scoped
traces and Docker Compose support (both cloud and self-hosted).
Core package internal/tracing/langfuse:
- HTTP client with batched async ingestion (non-blocking in request path)
- Sampling, environment / release tagging, and graceful fallback when
LANGFUSE_* env vars are absent (wrappers become no-ops)
- Gin middleware opens one trace per traced request and finishes it after
the handler chain returns, attaching method / path / user / session
- Trace context is stored under a typed key exported from internal/types
so logger.CloneContext can preserve it across handler / goroutine
boundaries (otherwise each LLM call auto-created an orphan trace,
fragmenting one request into many)
Per-model generation wrappers (opt-in via NewChat/NewEmbedder/...):
- chat: captures prompt, streaming output, token usage + TTFT
- embedding: approximates tokens when the provider omits usage
- rerank: previews query/docs, summarizes results to keep payload small
- vlm: records image count and total bytes, never uploads raw pixels
- asr: records file size and audio duration, never uploads audio bytes
Async title generation (GenerateTitleAsync) now forwards the trace key
into the goroutine so title calls appear under the parent chat trace.
Docker Compose:
- LANGFUSE_* env passthrough on the `app` service for cloud deployments
- Optional `langfuse` profile spins up a self-hosted Langfuse stack that
reuses WeKnora's existing PostgreSQL (separate database via an idempotent
init container that fixes ICU collation drift) and Redis (separate DB
number), adding only ClickHouse, MinIO, web and worker containers
- web/worker entrypoints URL-encode DB_PASSWORD / REDIS_PASSWORD at start
to avoid Prisma P1013 when passwords contain @ / # / etc.
Docs: docs/Langfuse集成.md covers cloud vs self-hosted, per-model usage
strategy, code map, and resource footprint.
The share-time validation forced every agent bound to a knowledge base to
have a rerank model configured, which blocked Wiki-type agents
(`wiki_search` / `wiki_read_page` / …) from being shared even though they
never invoke the reranker at runtime.
Align the check with the runtime logic in `session_agent_qa.go`: only
require a rerank model when the agent's allowed_tools include
`knowledge_search` (or rely on the default tool set, which contains it).
Wiki-only agents can now be shared without a rerank model; hybrid / RAG
agents still enforce the requirement.
Also update the handler error message to reflect the new rule.
Introduce per-model `custom_headers` config (similar to OpenAI Python SDK's
`extra_headers`) so users can inject gateway auth tokens, trace IDs, etc.
into every outbound API request for chat / embedding / rerank / VLLM / ASR
models. Reserved headers like Authorization / Content-Type are always
preserved to avoid breaking auth or signing flows.
Along the way, unify production and "test connection" paths onto a single
`ConfigFromModel(*types.Model, appID, appSecret)` constructor in each
model package. Both `service.modelService.GetXxxModel` and the four
`handler.initialization.Check*Model` / `TestEmbeddingModel` endpoints now
go through the same field mapping, so new parameters only need to be
added in one place.
Backend:
- Add `CustomHeaders map[string]string` to `types.ModelParameters` with
JSON/YAML `omitempty` tags for forward compatibility.
- New `internal/utils/extraheaders.go` providing `ApplyCustomHeaders`
(for hand-rolled HTTP paths) and `WrapHTTPClientWithHeaders` /
`CustomHeadersRoundTripper` (for SDK paths like go-openai). Reserved
headers (Authorization, api-key, Content-Type, Accept, Host,
Content-Length, User-Agent) are filtered out.
- Inject headers in every remote model path: chat (SDK + raw HTTP),
embedding (openai/aliyun/jina/volcengine/nvidia/azure_openai), rerank
(remote_api/aliyun/jina/nvidia/zhipu — via shared `customHeaderSetter`
interface), VLM and ASR (via http.Client wrapping).
- Add `ConfigFromModel` to each of chat/embedding/rerank/vlm/asr,
consolidating field mapping (ExtraConfig, CustomHeaders, InterfaceType
defaulting, WeKnoraCloud credentials, etc.) and cover it with
dedicated unit tests per package.
- Refactor `service/model.go`: replace ~100 lines of hand-written Config
literals with one-line `ConfigFromModel` calls; drop the now-unused
`stringMapToAnyMap` helper.
- Refactor `handler/initialization.go` test-connection endpoints: merge
four ad-hoc request structs into one `ModelTestRequest`, extract
`buildTestModel` and `resolveTenantWeKnoraCloudCreds` helpers, and
route all four endpoints through `ConfigFromModel` + `NewXxx` so the
test path is now behaviorally identical to production.
Frontend:
- Add `custom_headers` to the `ModelConfig` API type.
- `ModelEditorDialog.vue`: new Key-Value header editor (add/remove rows),
auto-converts map <-> array when loading/saving, and passes the
current header set to the `Test Connection` button so the preview
reflects exactly what production will send.
- `ModelSettings.vue`: serialize the header array back into a map for
the backend (drops empty rows).
- i18n: add custom-header labels / descriptions / placeholders to
zh-CN, en-US, ru-RU, ko-KR.
Tests:
- `internal/utils/extraheaders_test.go` covers reserved-header filtering
and round-tripper wrapping (including nil-client fallback).
- New `config_from_model_test.go` in chat / embedding / rerank / vlm /
asr verifies all fields (CustomHeaders, ExtraConfig, InterfaceType
defaulting, AppID/AppSecret) are propagated end-to-end.
The agent loop previously kept running when the LLM emitted final_answer with
malformed JSON arguments: observe.analyzeResponse strictly json.Unmarshal'd the
raw args and returned isDone=false on failure, while act.runToolCall separately
repaired the JSON and executed the tool. The mismatch made final_answer
non-terminal, so the LLM saw its own answer in the next round's tool result
and re-emitted final_answer, surfacing near-identical answers multiple times
to the user (issue #1008).
Fix by unifying parsing in a new tools.ParseFinalAnswerArgs helper with three
fallbacks (strict -> RepairJSON -> regex extraction) and making final_answer
always terminal in analyzeResponse. When no answer can be recovered, emit a
user-visible fallback message and a Done=true event so the UI still resolves.
Also route final_answer.Execute through the same helper so tool execution and
loop termination stay consistent.
Adds regression tests in observe_test.go and final_answer_test.go covering
valid args, RepairJSON recovery, unrecoverable args, and garbage input.
Refs: Tencent/WeKnora#1008
- Removed redundant struct definitions for WeKnoraCloud requests and messages, streamlining the customization process.
- Updated the weKnoraCloudRequestCustomizer function to directly modify the request object, enhancing clarity and reducing complexity.
- Improved the convertToWeKnoraCloudMessagesFromOpenAI function to maintain the original message structure while ensuring compatibility with WeKnoraCloud requirements.
- Introduced a unified capability requirements system for agent tools, ensuring consistent filtering of knowledge bases based on their capabilities.
- Implemented derived `kb_filter` logic to streamline compatibility checks between agent tools and knowledge bases, reducing redundancy in configuration.
- Updated frontend components to reflect new filtering logic, including improved handling of empty states and user feedback when no compatible knowledge bases are available.
- Enhanced internationalization support by adding relevant messages for tool compatibility issues across multiple languages.
- Removed header printing from the chatWithRawHTTP method to streamline logging.
- Updated log message to focus on endpoint and model name, enhancing clarity while reducing verbosity.
- Introduced UpdateAutoLinkedContent method to persist changes from machine-only link decorators without incrementing the version number, ensuring user-facing revisions remain consistent.
- Updated existing methods to utilize the new auto-linked content update logic, enhancing the handling of cross-link injection and dead-link cleanup.
- Enhanced the KnowledgeBase and WikiBrowser components to improve user experience during wiki operations, including status tracking and indexing indicators.
- Added scheduling for wiki status probes to provide timely feedback on indexing processes after user-triggered actions.
- Updated the scrubWikiPendingIngest method to accept a reason parameter, improving logging clarity during wiki cleanup and reparse operations.
- Introduced prepareWikiForReparse to maintain the integrity of the pending ingest queue during reparse events, preventing stale operations from executing.
- Enhanced the mapping logic in the wiki ingest process to differentiate between stale and reparse updates, ensuring accurate handling of knowledge contributions.
- Improved logging to provide better insights into the number of reparse and stale slugs processed during wiki ingestion.
- Enhanced the UpdatePage and UpdateMeta methods to differentiate between user-visible content changes and bookkeeping updates, ensuring the version number is only incremented for actual content modifications.
- Updated the WikiPage struct and associated interfaces to reflect the new versioning policy, improving clarity on when the version is modified.
- Improved documentation for methods to clarify their intended use and behavior regarding versioning and metadata updates.
- Changed `kb_selection_mode` in agent type presets from "selected" to "all" to allow agents to access all knowledge bases by default.
- Revised prompt templates to clarify the handling of bound knowledge bases, replacing the `{{knowledge_bases}}` placeholder with a reference to the `<bound_knowledge_bases>` block in the user message's `<runtime_context>`.
- Enhanced internal logic to ensure that only searchable knowledge bases are considered during retrieval, improving the efficiency of knowledge searches.
- Added new functions for better formatting and handling of knowledge base metadata in the runtime context.
- Updated documentation and comments for clarity on the changes made to knowledge base interactions.
- Introduced extraction granularity settings in the wiki configuration, allowing users to control the level of detail in entity and concept extraction (focused, standard, exhaustive).
- Updated the UI to reflect these new options, including tooltips for better user guidance.
- Refactored the knowledge base editor to support the new extraction granularity feature, ensuring a seamless user experience.
- Improved the backend logic for wiki ingest tasks to utilize the new granularity settings, enhancing the accuracy of extracted content.
- Added tests to validate the new extraction granularity functionality and ensure robust performance across different configurations.
- Updated the knowledge base service to ensure WikiConfig is created when wiki indexing is enabled, enhancing backward compatibility.
- Removed legacy syncing of WikiConfig.Enabled to IndexingStrategy.WikiEnabled, streamlining the logic for wiki enablement.
- Introduced a custom retry delay function for asynq tasks to handle wiki ingest lock conflicts more effectively, improving task management.
- Cleaned up the container file by removing unnecessary whitespace, contributing to better code readability.
- Deleted the RebuildKnowledgeBaseIndex function from the knowledge service, which was responsible for reprocessing all documents in a knowledge base.
- Removed the RebuildIndex handler from the KnowledgeBaseHandler, eliminating the endpoint that triggered the rebuild process.
- Updated the router to remove the route for rebuilding the index, streamlining the knowledge base management functionality.
- Added comments to clarify the purpose of the `builtinAgentIDsOrdered` list, specifically noting the exclusion of `BuiltinWikiFixerID` to prevent clutter in the user-facing agent list.
- This change enhances the understanding of the built-in agent configuration and maintains a cleaner user interface for agent selection.
- Updated the DeleteKnowledge and DeleteKnowledgeList functions to skip vector store cleanup for knowledge entries without an embedding model, preventing unnecessary errors.
- Enhanced logging to provide clear information when skipping cleanup for knowledge without embeddings, improving overall robustness and clarity in the deletion process.
These changes ensure that the deletion process is more efficient and user-friendly, particularly for knowledge bases that do not utilize embeddings.
- Added a check to skip vector and keyword indexing for knowledge bases that do not require an embedding model, preventing errors during the indexing process.
- Ensured that chunks are marked as indexed even when skipping the indexing step, maintaining a consistent state for downstream processes.
- Enhanced logging to provide clear information on the indexing status and any issues encountered while updating chunk statuses.
These changes improve the robustness of the indexing process in the ImageMultimodalService, ensuring better handling of knowledge bases without embedding capabilities.
- Updated the agent system prompt to emphasize the use of regex for chunk searches, improving retrieval accuracy and efficiency.
- Introduced a new `Capabilities` field in the `KnowledgeBaseInfo` struct to define the retrieval surfaces available for each knowledge base, guiding the agent's strategy selection.
- Enhanced the `grep_chunks` tool to support regex queries, allowing for more flexible and powerful text pattern matching in knowledge base chunks.
- Improved JSON handling in the `RepairJSON` function to address invalid escape sequences, particularly for regex patterns, ensuring robust parsing and error handling.
- Updated documentation and comments across various files to clarify usage and expectations for regex and knowledge base capabilities.
These changes significantly enhance the agent's ability to retrieve relevant information while ensuring the integrity of input data handling.
- Added a new configuration file `agent_type_presets.yaml` to define various agent type presets, including RAG Q&A, Wiki Q&A, Hybrid (Wiki + RAG), and Data Analysis.
- Updated the `builtin_agents.yaml` to associate built-in agents with the new agent type presets, enhancing their configuration options.
- Enhanced the agent system prompt templates to reflect the new agent types and their functionalities.
- Implemented API endpoints to retrieve agent type presets, allowing for dynamic loading in the frontend.
- Updated the frontend to support agent type selection in the agent editor modal, improving user experience by auto-filling relevant configurations based on selected presets.
These changes significantly enhance the flexibility and usability of the agent system, allowing users to easily configure agents based on predefined types.
- Updated the SuggestedQuestion interface to include 'wiki' as a new source type, allowing for more diverse question origins.
- Implemented ListRecentForSuggestions method in the wikiPageRepository to retrieve recent wiki pages for fallback suggestions, improving the agent's ability to generate relevant questions when other sources are unavailable.
- Enhanced the customAgentService to utilize the new wiki page suggestions, ensuring a broader range of questions can be generated from wiki content.
These changes significantly improve the agent's suggestion capabilities by incorporating wiki content, enhancing user interaction and information retrieval.
- Enhanced the `cleanupWikiOnKnowledgeDelete` function to pass the full knowledge object, allowing for more accurate title and summary retrieval during wiki cleanup.
- Introduced a tombstone mechanism in Redis to mark recently deleted knowledge, enabling in-flight wiki ingest tasks to skip processing for deleted documents.
- Updated the `isKnowledgeGone` function to check for deleted knowledge using the tombstone, improving efficiency in handling concurrent delete and ingest operations.
- Added logic in the `wikiIngestService` to filter out updates for deleted knowledge, ensuring that no ghost references are created during the ingest process.
- Implemented lint checks for stale source references in the `WikiLintService`, allowing for automatic cleanup of pages referencing deleted knowledge.
These changes significantly enhance the robustness of the wiki system, ensuring consistency and accuracy in handling knowledge deletions and related wiki pages.
- Updated the agent system prompt to include new auxiliary tools and improved synthesis and issue flagging steps, enhancing the clarity of the workflow.
- Revised constraints to emphasize the importance of retrieving information from the wiki first and using skills when applicable.
- Expanded localization files for English, Korean, Russian, and Chinese to include new wiki tool descriptions and statuses, ensuring consistency across languages.
- Improved the agent editor modal to provide a clearer overview of tool statuses and retrieval preferences, enhancing user experience.
These changes significantly improve the functionality and usability of the agent system, particularly in relation to wiki interactions and tool management.
- Improved the `getKbIdForWiki` function to better handle multiple knowledge bases associated with a single slug, ensuring the most recent tool call's mapping is prioritized.
- Updated the `buildRuntimeContextBlock` function to include active knowledge bases and pinned documents, providing a clearer context for multi-turn interactions and preventing prompt injection.
- Added `registerLinkedSlugs` function to track outbound and inbound links for wiki pages, facilitating better resolution of slugs in the frontend without additional requests.
These enhancements significantly improve the accuracy and safety of knowledge base retrieval and context management in the agent system.
- Added the `isStructuralPage` function to identify wiki-level structural/meta pages (index, log) that should not be filtered by knowledge IDs.
- Updated the `pageIntersectsKnowledgeIDs` function to incorporate logic for structural pages, ensuring they are always accessible regardless of filtering.
- Enhanced the handling of pages with no SourceRefs to allow them through the filter, improving the accuracy of content retrieval.
These changes enhance the filtering logic for wiki pages, ensuring that essential structural pages remain accessible while refining the overall content management process.
- Added the WikiScope struct to define the retrieval scope for wiki knowledge bases, allowing for optional filtering by specific document IDs.
- Implemented NewWikiScopesFromKBIDs constructor for easier creation of WikiScope instances.
- Enhanced the wikiReadPageTool and wikiSearchTool to utilize WikiScope, improving the handling of knowledge base IDs and document filtering during page retrieval.
- Updated the agent service to carry document whitelists into the wiki scopes, ensuring that only relevant pages are surfaced based on user mentions.
These changes significantly improve the flexibility and accuracy of wiki page retrieval, enhancing the overall user experience when interacting with wiki content.
- Modified the XML output structure in the wikiReadPageTool to replace title and slug elements with a unified link format, enhancing clarity and consistency in the representation of wiki pages.
- Adjusted the wikiSearchTool to align with the new output format, ensuring that search results maintain the same structure for better integration and usability.
These changes streamline the output format for wiki pages, contributing to a more coherent and user-friendly experience when accessing wiki content.
- Updated the chat component to include a new class for handling sidebar collapse, improving layout responsiveness.
- Enhanced the wiki page retrieval logic to support multiple knowledge bases for the same slug, allowing for more comprehensive results.
- Introduced a mechanism to track ambiguous slugs across knowledge bases, improving clarity in search results.
These changes significantly improve the user experience in the chat interface and enhance the functionality of the wiki page retrieval system.
- Updated the agent system prompt to require wiki-links in citations, improving user navigation to source pages.
- Implemented logic in AgentStreamDisplay to handle wiki link clicks and keyboard interactions, facilitating seamless navigation to knowledge base wiki pages.
- Enhanced WikiBrowser to auto-select pages based on query parameters, improving user experience when accessing specific wiki content.
- Added localization support for new error messages related to knowledge base identification, ensuring clarity in user interactions.
These changes significantly improve the usability and functionality of the wiki navigation system, providing users with a more intuitive experience when accessing and citing wiki content.
- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings.
- The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities.
- Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system.
These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
- Introduced `enrichChunkImageInfo` function to populate image information for text chunks by retrieving data from child chunks, ensuring that image-related content is not lost during processing.
- Updated `reconstructContent` to `reconstructEnrichedContent`, allowing for the inclusion of OCR text and captions from images, improving the quality of content sent to the LLM.
- Added `writeDedupItemXML` and `xmlEscape` functions to format entity and concept entries as structured XML, enhancing clarity and preventing nonsensical merges in deduplication prompts.
- Refined the `deduplicateExtractedBatch` method to utilize the new XML formatting for better output consistency.
These changes significantly improve the handling of image data and the overall clarity of deduplication prompts, contributing to a more effective wiki content management system.
- Simplified the `InjectCrossLinks` and `injectCrossLinks` methods by utilizing a shared `linkifyContent` function, ensuring consistent handling of code blocks, existing links, and word boundaries.
- Replaced manual reference collection and sorting with a dedicated `collectLinkRefs` function to streamline the process of gathering link references from wiki pages.
- Enhanced the overall readability and maintainability of the code by removing redundant structures and logic.
These changes improve the efficiency and clarity of the cross-linking functionality within the wiki service, contributing to a more robust content management system.
- Modified the chat creation component to improve layout and responsiveness by adjusting the dialogue answers section.
- Refined the logic in the KnowledgePostProcessService to conditionally spawn wiki ingest tasks based on wiki configuration settings, enhancing task management.
- Added migration scripts for creating and dropping the `wiki_page_issues` table, facilitating better issue tracking within the wiki system.
These changes enhance the user interface for chat creation and improve the backend processing of knowledge base tasks, contributing to a more efficient and user-friendly experience.
- Introduced `encodeSlugPath` function to safely encode wiki slugs for API requests, preserving hierarchical routing.
- Updated API functions to utilize the new slug encoding for `getWikiPage`, `updateWikiPage`, and `deleteWikiPage`.
- Enhanced `ListBySourceRef` method in the repository to safely handle source knowledge IDs with JSON marshalling, improving query security.
- Added `escapeLikePattern` function to ensure safe concatenation of LIKE patterns in SQL queries, preventing unintended matches.
- Updated `KnowledgePostProcessService` to include Redis client for task management, improving performance in handling wiki ingest tasks.
- Enhanced `WikiLintIssue` structure to include `TargetSlug` for better issue tracking and auto-fixing capabilities.
These changes significantly improve the robustness and security of the wiki management system, enhancing user experience and data integrity.
- Added new tools for wiki page editing, including writing, replacing text, renaming, and deleting pages, with corresponding data structures for each action.
- Enhanced the frontend with a new `WikiEditResult` component to display results of wiki edits, supporting multiple languages for improved accessibility.
- Updated localization files for English and Chinese to include new strings related to wiki actions.
- Modified existing tools to return structured data for wiki actions, improving the clarity and usability of the output messages.
These changes significantly enhance the wiki editing capabilities, providing users with a more robust and user-friendly experience when managing wiki content.
- Updated the agent system prompt to refine the workflow for fixing issues, emphasizing the need to verify if issues still exist before making edits.
- Introduced a new `embeddedMode` prop in the frontend components to manage UI behavior based on the context of use.
- Simplified issue fix prompts in multiple languages for clarity, ensuring users receive concise instructions for resolving issues.
- Enhanced the WikiBrowser component to improve the display of issues and actions, including updated icons and streamlined interaction elements.
These changes significantly improve the user experience and functionality of the Wiki Fixer agent, fostering more efficient issue resolution and content management.
- Added a new built-in agent, "Wiki Fixer," designed to repair and optimize Wiki pages based on linter issues, with multilingual support for enhanced accessibility.
- Implemented a corresponding system prompt detailing the agent's role, mission, and workflow for effective issue resolution.
- Introduced new API functions for listing and updating wiki issues, allowing for better management of content conflicts and errors.
- Enhanced the frontend with new UI elements to display pending issues and facilitate auto-fixing, improving user interaction with the Wiki content.
These changes significantly enhance the agent's capabilities for maintaining the accuracy and quality of Wiki pages, fostering a more reliable knowledge base.
- Added a new tool, `wiki_flag_issue`, enabling users to report factual errors, mixed entities, or outdated information on wiki pages.
- Updated the agent system prompt and tool definitions to include the new tool, enhancing the agent's capabilities for maintaining wiki accuracy.
- Implemented backend functionality for creating, listing, and updating the status of flagged issues, ensuring effective tracking and resolution.
These changes significantly improve the agent's ability to manage and report issues within the wiki, fostering a more reliable knowledge base.
- Updated the `WikiPageModifyPrompt` to include a critical conflict check, ensuring that new information pertains to the same core entity before merging.
- Enhanced the `WikiDeduplicationPrompt` to emphasize the importance of not merging different products or versions, reinforcing the principle that related does not equal the same.
- These changes improve the accuracy of content updates and deduplication processes, ensuring better management of wiki information.
- Added a new tool, `wiki_read_source_doc`, allowing agents to access specific source documents for detailed information retrieval when wiki page content is insufficient.
- Updated the agent system prompt and tool definitions to incorporate the new tool, enhancing the agent's capabilities for in-depth knowledge extraction.
- Modified existing tools to support the new functionality, ensuring seamless integration within the agent's workflow.
These changes significantly improve the agent's ability to provide accurate and detailed responses by leveraging source documents alongside wiki content.
- Updated the agent system prompt to clarify the workflow for knowledge retrieval, emphasizing the use of `wiki_search` and `wiki_read_page` tools for specific queries and general overviews.
- Modified the `wikiReadPageTool` to support reading multiple wiki pages simultaneously, improving efficiency in fetching content.
- Enhanced the `wikiSearchTool` to utilize PostgreSQL POSIX regular expressions for more effective search queries, allowing for complex pattern matching.
- Adjusted the `wikiPageRepository` to replace `ILIKE` with regex matching for search queries, increasing the precision of search results.
These changes significantly improve the agent's ability to retrieve and synthesize information from the wiki, enhancing user interactions and response accuracy.
- Added a new constraint to the agent system prompt emphasizing the need for fresh searches and reads for each question, ensuring up-to-date information retrieval.
- Updated the agent tool configuration to include `ToolWikiSearch` and `ToolWikiReadPage`, enhancing the agent's capabilities for accessing and utilizing wiki content effectively.
These changes improve the agent's performance in providing accurate and current responses based on wiki knowledge.
- Introduced a new test file for JSON handling to ensure that unmarshalling correctly copies input data.
- Validated that the stored JSON remains intact after input mutation, and confirmed successful marshalling of the output.
- This test enhances the reliability of JSON operations within the types package.
- Introduced a new built-in agent, "Wiki Researcher," designed for navigating and answering questions based on Wiki knowledge bases, complete with multilingual support.
- Added a corresponding system prompt that outlines the agent's role, mission, and workflow for effective knowledge graph traversal.
- Updated the agent configuration to include specific tools and parameters tailored for the Wiki Researcher, enhancing its functionality and user interaction.
- Removed deprecated wiki tools from the agent service to streamline the toolset and improve performance.
These changes significantly enhance the capabilities of the agent framework, providing users with a specialized tool for in-depth Wiki-based research.
- Introduced a new built-in agent, "Wiki Researcher," designed for navigating and answering questions based on Wiki knowledge bases, complete with multilingual support.
- Added a corresponding system prompt that outlines the agent's role, mission, and workflow for effective knowledge graph traversal.
- Updated the agent configuration to include specific tools and parameters tailored for the Wiki Researcher, enhancing its functionality and user interaction.
- Removed deprecated wiki tools from the agent service to streamline the toolset and improve performance.
These changes significantly enhance the capabilities of the agent system, providing users with a specialized tool for in-depth research and information retrieval from Wiki sources.
- Updated the `WikiDeduplicationPrompt` to clarify the criteria for identifying exact duplicates, emphasizing strict matching of real-world entities and concepts.
- Improved the `appendLogEntry` method to include a summary parameter for better context in log entries, enhancing traceability of operations.
- Consolidated log entry creation for both retraction and ingestion operations, ensuring chronological clarity in the log.
- Implemented validation for slug merges to prevent mismatches based on entity types, improving the accuracy of deduplication.
These changes enhance the clarity and reliability of the wiki ingestion process, leading to more accurate content management.
- Introduced a new `WikiBatchContext` to share data across Map and Reduce phases, reducing redundant database queries.
- Replaced separate deduplication calls with a single `deduplicateExtractedBatch` method, improving efficiency by leveraging pre-loaded page data.
- Enhanced JSON handling by implementing a `cleanLLMJSON` function to sanitize LLM-generated output, ensuring safe parsing.
- Updated relevant methods to utilize the new context and deduplication logic, streamlining the overall ingestion workflow.
These changes enhance the performance and clarity of the wiki ingestion service, leading to more efficient content processing.
- Renamed `WikiPageUpdatePrompt` to `WikiPageModifyPrompt` to better reflect its functionality of adding new information and removing outdated content in a single operation.
- Enhanced the instructions for both `WikiPageModifyPrompt` and `WikiPageRetractPrompt` to clarify the handling of additions and retractions, ensuring accurate content updates.
- Consolidated the logic for enqueuing retraction tasks in the `cleanupWikiOnKnowledgeDelete` method to streamline the process of managing affected pages.
- Removed the `rebuildWikiIndexSimple` function to simplify the codebase, as its functionality is no longer needed with the new ingestion approach.
These changes improve the clarity and efficiency of the wiki content management process, enhancing overall system performance.
- Introduced `pending_tasks` and `is_active` fields in the `WikiStats` interface to track the number of tasks waiting for processing and the current activity status of the wiki ingestion.
- Updated localization files for English, Korean, Russian, and Chinese to include a new message for displaying the number of pending tasks in the wiki queue.
- Enhanced the `WikiBrowser` component to display the queue status, improving user awareness of ongoing tasks.
- Implemented polling logic to refresh wiki stats periodically when there are pending tasks or active ingestion, ensuring up-to-date information is presented.
These changes enhance the functionality and user experience of the wiki management system by providing real-time insights into task status.
- Enhanced the logging of the tenant access token in the Feishu client to handle variable lengths for prefix and suffix, ensuring accurate display even for shorter tokens.
- Updated the test cases in `connector_test.go` to reflect changes in the endpoint path for file downloads, improving test reliability.
- Adjusted example outputs in `example_test.go` for clarity and consistency.
- Modified image resolution tests to correct expected outcomes based on updated criteria for icon images.
- Added environment variable settings in image resolver tests to ensure localhost accessibility during testing.
These changes enhance the robustness of logging and testing mechanisms across the application.
wizardchen