- Introduced a unified capability requirements system for agent tools, ensuring consistent filtering of knowledge bases based on their capabilities.
- Implemented derived `kb_filter` logic to streamline compatibility checks between agent tools and knowledge bases, reducing redundancy in configuration.
- Updated frontend components to reflect new filtering logic, including improved handling of empty states and user feedback when no compatible knowledge bases are available.
- Enhanced internationalization support by adding relevant messages for tool compatibility issues across multiple languages.
- Removed header printing from the chatWithRawHTTP method to streamline logging.
- Updated log message to focus on endpoint and model name, enhancing clarity while reducing verbosity.
- Introduced chunkingDirty state to track manual adjustments to chunking settings, preventing automatic overrides during strategy changes.
- Added WIKI_ONLY_CHUNKING_PRESET for specific configurations when using the wiki-only indexing strategy, ensuring optimal settings in creation mode.
- Implemented logic to apply or revert chunking presets based on the current indexing strategy, maintaining user-defined settings in edit mode.
- Updated resetState function to clear chunkingDirty state upon modal reset.
- Introduced UpdateAutoLinkedContent method to persist changes from machine-only link decorators without incrementing the version number, ensuring user-facing revisions remain consistent.
- Updated existing methods to utilize the new auto-linked content update logic, enhancing the handling of cross-link injection and dead-link cleanup.
- Enhanced the KnowledgeBase and WikiBrowser components to improve user experience during wiki operations, including status tracking and indexing indicators.
- Added scheduling for wiki status probes to provide timely feedback on indexing processes after user-triggered actions.
- Added functionality to track and display the wiki indexing status, including pending tasks and active state.
- Introduced a polling mechanism to update the wiki status at regular intervals, enhancing user awareness of background processes.
- Updated the breadcrumb navigation to show indexing indicators, improving the user interface and experience during wiki operations.
- Enhanced the WikiBrowser component to emit status changes to the parent component for better state management.
- Updated the scrubWikiPendingIngest method to accept a reason parameter, improving logging clarity during wiki cleanup and reparse operations.
- Introduced prepareWikiForReparse to maintain the integrity of the pending ingest queue during reparse events, preventing stale operations from executing.
- Enhanced the mapping logic in the wiki ingest process to differentiate between stale and reparse updates, ensuring accurate handling of knowledge contributions.
- Improved logging to provide better insights into the number of reparse and stale slugs processed during wiki ingestion.
- Added line-height to text elements for better readability.
- Enhanced the legend-action-icon class with display properties for improved alignment and consistency in icon presentation.
- Adjusted font sizes and line heights for better visual hierarchy and user experience.
- Upgraded the `mermaid` package from version 11.4.1 to 11.14.0 for improved features and bug fixes.
- Changed the resolved URLs for several `@chevrotain` packages to use Tencent's mirror, enhancing download reliability.
- Added the `@upsetjs/venn.js` package version 2.0.0 to the project, expanding visualization capabilities.
- Updated the `acorn` package from version 8.15.0 to 8.16.0 to incorporate the latest improvements.
- Removed deprecated CGO flags from the build command in the .air.toml file, streamlining the build process for the application.
- This change enhances compatibility and reduces potential build warnings, improving the overall development experience.
- Enhanced the UpdatePage and UpdateMeta methods to differentiate between user-visible content changes and bookkeeping updates, ensuring the version number is only incremented for actual content modifications.
- Updated the WikiPage struct and associated interfaces to reflect the new versioning policy, improving clarity on when the version is modified.
- Improved documentation for methods to clarify their intended use and behavior regarding versioning and metadata updates.
- Deleted migration files for the wiki_pages and wiki_page_issues tables, as well as the indexing_strategy column from knowledge_bases, to clean up the schema.
- This removal streamlines the migration process and ensures that only relevant migrations are retained in the project.
- Changed `kb_selection_mode` in agent type presets from "selected" to "all" to allow agents to access all knowledge bases by default.
- Revised prompt templates to clarify the handling of bound knowledge bases, replacing the `{{knowledge_bases}}` placeholder with a reference to the `<bound_knowledge_bases>` block in the user message's `<runtime_context>`.
- Enhanced internal logic to ensure that only searchable knowledge bases are considered during retrieval, improving the efficiency of knowledge searches.
- Added new functions for better formatting and handling of knowledge base metadata in the runtime context.
- Updated documentation and comments for clarity on the changes made to knowledge base interactions.
- Deleted `AGENT_WIKI_ANALYSIS.md`, `ANALYSIS_SUMMARY.md`, `COMPLETION_REPORT.md`, `DEPLOYMENT_CHECKLIST.md`, `DOCUMENTATION_INDEX.md`, `EXECUTIVE_SUMMARY_WIKI_CONFIG_FIX.md`, and `EXECUTIVE_SUMMARY.md` as they are no longer relevant to the current project scope.
- This cleanup helps streamline the repository and reduces clutter, ensuring that only up-to-date and necessary documentation remains accessible.
- Introduced extraction granularity settings in the wiki configuration, allowing users to control the level of detail in entity and concept extraction (focused, standard, exhaustive).
- Updated the UI to reflect these new options, including tooltips for better user guidance.
- Refactored the knowledge base editor to support the new extraction granularity feature, ensuring a seamless user experience.
- Improved the backend logic for wiki ingest tasks to utilize the new granularity settings, enhancing the accuracy of extracted content.
- Added tests to validate the new extraction granularity functionality and ensure robust performance across different configurations.
- Enhanced the `sanitizeForDisplay` function to correctly remove empty inline code snippets (`` or ` `) while preserving triple-backtick fenced code blocks.
- This change ensures better rendering of code blocks in the chat interface, improving overall display accuracy.
- Updated the knowledge base service to ensure WikiConfig is created when wiki indexing is enabled, enhancing backward compatibility.
- Removed legacy syncing of WikiConfig.Enabled to IndexingStrategy.WikiEnabled, streamlining the logic for wiki enablement.
- Introduced a custom retry delay function for asynq tasks to handle wiki ingest lock conflicts more effectively, improving task management.
- Cleaned up the container file by removing unnecessary whitespace, contributing to better code readability.
- Deleted the RebuildKnowledgeBaseIndex function from the knowledge service, which was responsible for reprocessing all documents in a knowledge base.
- Removed the RebuildIndex handler from the KnowledgeBaseHandler, eliminating the endpoint that triggered the rebuild process.
- Updated the router to remove the route for rebuilding the index, streamlining the knowledge base management functionality.
- Introduced a check for Wails desktop environment to conditionally intercept Cmd/Ctrl+R key presses, preventing full page reloads that could lead to a white screen.
- Updated event listener management to ensure key handling is only active in the Wails desktop context, improving user experience by maintaining application state during refresh actions.
- Added comments for clarity on the purpose of these changes and their impact on different deployment environments.
- Modified the CSS for markdown tables to use `width: fit-content`, allowing tables to shrink to their content while still respecting `max-width: 100%`.
- This change enhances the presentation of tables, ensuring better usability and visual consistency within the WikiBrowser interface.
- Implemented a new `refreshSelectedPage` function to update the content of the currently selected wiki page without altering navigation history, improving user experience by ensuring up-to-date information is displayed.
- Enhanced the graph view to reload when the user is currently viewing it, ensuring new nodes and edges are visible.
- Added comprehensive CSS styles for markdown tables, improving their presentation and usability within the WikiBrowser interface.
These changes enhance the interactivity and visual consistency of the WikiBrowser, providing users with a more dynamic and engaging experience.
- Eliminated the "thinking" and "todo_write" tools from the allowed tools list in multiple sections of the `builtin_agents.yaml` file, streamlining the agent capabilities.
- This change enhances the clarity and relevance of the tools available to built-in agents, ensuring a more focused and efficient configuration.
- Added comments to clarify the purpose of the `builtinAgentIDsOrdered` list, specifically noting the exclusion of `BuiltinWikiFixerID` to prevent clutter in the user-facing agent list.
- This change enhances the understanding of the built-in agent configuration and maintains a cleaner user interface for agent selection.
- Introduced a shared timer for debouncing mouseleave to mouseenter transitions, preventing flickering when quickly moving between adjacent nodes.
- Updated event listeners to clear and set the hover timer, ensuring smoother highlight transitions without abrupt visual changes.
- Enhanced the rendering logic to maintain consistent highlighting behavior based on node selection state.
These changes enhance the user experience by providing a more fluid interaction with the graph nodes in the WikiBrowser.
- Simplified the rendering of wiki citations in chat bubbles by removing truncation and inline pill styles, replacing them with a standard link format.
- Updated CSS styles to reflect the new inline link design, featuring a brand-blue dashed underline for better visual integration with surrounding text.
- Enhanced accessibility by maintaining the use of the `data-slug` attribute for interaction handling.
These changes streamline the presentation of wiki citations, improving user experience and visual consistency across the chat interface.
- Expanded the `exclude_dir` list in the `.air.toml` configuration to include additional directories such as "data", "WeKnora-Chrome-Extension", "Formula", and others, ensuring a more comprehensive exclusion during the build process.
- This change aims to streamline the build environment by preventing unnecessary file monitoring and potential conflicts, enhancing overall efficiency.
- Updated the DeleteKnowledge and DeleteKnowledgeList functions to skip vector store cleanup for knowledge entries without an embedding model, preventing unnecessary errors.
- Enhanced logging to provide clear information when skipping cleanup for knowledge without embeddings, improving overall robustness and clarity in the deletion process.
These changes ensure that the deletion process is more efficient and user-friendly, particularly for knowledge bases that do not utilize embeddings.
- Added a check to skip vector and keyword indexing for knowledge bases that do not require an embedding model, preventing errors during the indexing process.
- Ensured that chunks are marked as indexed even when skipping the indexing step, maintaining a consistent state for downstream processes.
- Enhanced logging to provide clear information on the indexing status and any issues encountered while updating chunk statuses.
These changes improve the robustness of the indexing process in the ImageMultimodalService, ensuring better handling of knowledge bases without embedding capabilities.
- Changed the agent type from "custom" to "rag-qa" in the `builtin_agents.yaml` file, aligning with the new agent type presets introduced previously.
- This update enhances the configuration of built-in agents, ensuring they utilize the appropriate agent type for improved functionality and performance.
These changes contribute to a more coherent and effective agent system by standardizing agent types across configurations.
- Enhanced the agent system prompt to clarify citation formats for wiki and chunk sources, ensuring accurate inline citations.
- Introduced a new citation format section in the prompt, detailing the required syntax for both wiki pages and knowledge chunks.
- Updated the frontend to visually align wiki citations with knowledge base citations, improving user experience and consistency in citation presentation.
- Implemented truncation for long wiki labels in citations to maintain visual consistency in chat bubbles.
These changes improve the clarity and usability of citations in the agent system, enhancing the overall information retrieval process.
- Updated the agent system prompt to emphasize the use of regex for chunk searches, improving retrieval accuracy and efficiency.
- Introduced a new `Capabilities` field in the `KnowledgeBaseInfo` struct to define the retrieval surfaces available for each knowledge base, guiding the agent's strategy selection.
- Enhanced the `grep_chunks` tool to support regex queries, allowing for more flexible and powerful text pattern matching in knowledge base chunks.
- Improved JSON handling in the `RepairJSON` function to address invalid escape sequences, particularly for regex patterns, ensuring robust parsing and error handling.
- Updated documentation and comments across various files to clarify usage and expectations for regex and knowledge base capabilities.
These changes significantly enhance the agent's ability to retrieve relevant information while ensuring the integrity of input data handling.
- Added a new configuration file `agent_type_presets.yaml` to define various agent type presets, including RAG Q&A, Wiki Q&A, Hybrid (Wiki + RAG), and Data Analysis.
- Updated the `builtin_agents.yaml` to associate built-in agents with the new agent type presets, enhancing their configuration options.
- Enhanced the agent system prompt templates to reflect the new agent types and their functionalities.
- Implemented API endpoints to retrieve agent type presets, allowing for dynamic loading in the frontend.
- Updated the frontend to support agent type selection in the agent editor modal, improving user experience by auto-filling relevant configurations based on selected presets.
These changes significantly enhance the flexibility and usability of the agent system, allowing users to easily configure agents based on predefined types.
- Updated the SuggestedQuestion interface to include 'wiki' as a new source type, allowing for more diverse question origins.
- Implemented ListRecentForSuggestions method in the wikiPageRepository to retrieve recent wiki pages for fallback suggestions, improving the agent's ability to generate relevant questions when other sources are unavailable.
- Enhanced the customAgentService to utilize the new wiki page suggestions, ensuring a broader range of questions can be generated from wiki content.
These changes significantly improve the agent's suggestion capabilities by incorporating wiki content, enhancing user interaction and information retrieval.
- Introduced a new warning message in English and Chinese to inform users that the indexing strategy cannot be changed once the knowledge base contains content.
- Updated the UI to visually indicate when indexing options are locked, preventing user interaction with disabled checkboxes.
- Enhanced the styling of the indexing strategy section to improve user experience and clarity.
These changes ensure users are aware of the restrictions on indexing strategy modifications, enhancing the overall usability of the knowledge base editor.
- Enhanced the `cleanupWikiOnKnowledgeDelete` function to pass the full knowledge object, allowing for more accurate title and summary retrieval during wiki cleanup.
- Introduced a tombstone mechanism in Redis to mark recently deleted knowledge, enabling in-flight wiki ingest tasks to skip processing for deleted documents.
- Updated the `isKnowledgeGone` function to check for deleted knowledge using the tombstone, improving efficiency in handling concurrent delete and ingest operations.
- Added logic in the `wikiIngestService` to filter out updates for deleted knowledge, ensuring that no ghost references are created during the ingest process.
- Implemented lint checks for stale source references in the `WikiLintService`, allowing for automatic cleanup of pages referencing deleted knowledge.
These changes significantly enhance the robustness of the wiki system, ensuring consistency and accuracy in handling knowledge deletions and related wiki pages.
- Updated the agent system prompt to include new auxiliary tools and improved synthesis and issue flagging steps, enhancing the clarity of the workflow.
- Revised constraints to emphasize the importance of retrieving information from the wiki first and using skills when applicable.
- Expanded localization files for English, Korean, Russian, and Chinese to include new wiki tool descriptions and statuses, ensuring consistency across languages.
- Improved the agent editor modal to provide a clearer overview of tool statuses and retrieval preferences, enhancing user experience.
These changes significantly improve the functionality and usability of the agent system, particularly in relation to wiki interactions and tool management.
- Renamed the "Wiki Researcher" to "Wiki Questioner" and adjusted its description for better alignment with its functionality.
- Updated the "Wiki Fixer" to "Wiki Revising" with a revised description to reflect its focus on revising Wiki pages based on inspection issues.
- Made corresponding changes to localized names and descriptions in Chinese, Japanese, and Korean to ensure consistency across languages.
These updates enhance the clarity and usability of the built-in agents in the system.
- Enhanced the indexing strategy section by adding descriptions and a new layout for better user interaction.
- Introduced toggle functionality for vector indexing and improved the visual presentation of indexing options with new styles and badges.
- Updated the component's CSS to create a more responsive and user-friendly design for indexing check items.
These changes significantly improve the usability and clarity of the indexing strategy settings in the knowledge base editor.
- Improved the `getKbIdForWiki` function to better handle multiple knowledge bases associated with a single slug, ensuring the most recent tool call's mapping is prioritized.
- Updated the `buildRuntimeContextBlock` function to include active knowledge bases and pinned documents, providing a clearer context for multi-turn interactions and preventing prompt injection.
- Added `registerLinkedSlugs` function to track outbound and inbound links for wiki pages, facilitating better resolution of slugs in the frontend without additional requests.
These enhancements significantly improve the accuracy and safety of knowledge base retrieval and context management in the agent system.
- Added the `isStructuralPage` function to identify wiki-level structural/meta pages (index, log) that should not be filtered by knowledge IDs.
- Updated the `pageIntersectsKnowledgeIDs` function to incorporate logic for structural pages, ensuring they are always accessible regardless of filtering.
- Enhanced the handling of pages with no SourceRefs to allow them through the filter, improving the accuracy of content retrieval.
These changes enhance the filtering logic for wiki pages, ensuring that essential structural pages remain accessible while refining the overall content management process.
- Added the WikiScope struct to define the retrieval scope for wiki knowledge bases, allowing for optional filtering by specific document IDs.
- Implemented NewWikiScopesFromKBIDs constructor for easier creation of WikiScope instances.
- Enhanced the wikiReadPageTool and wikiSearchTool to utilize WikiScope, improving the handling of knowledge base IDs and document filtering during page retrieval.
- Updated the agent service to carry document whitelists into the wiki scopes, ensuring that only relevant pages are surfaced based on user mentions.
These changes significantly improve the flexibility and accuracy of wiki page retrieval, enhancing the overall user experience when interacting with wiki content.
- Modified the XML output structure in the wikiReadPageTool to replace title and slug elements with a unified link format, enhancing clarity and consistency in the representation of wiki pages.
- Adjusted the wikiSearchTool to align with the new output format, ensuring that search results maintain the same structure for better integration and usability.
These changes streamline the output format for wiki pages, contributing to a more coherent and user-friendly experience when accessing wiki content.
- Updated the chat component to include a new class for handling sidebar collapse, improving layout responsiveness.
- Enhanced the wiki page retrieval logic to support multiple knowledge bases for the same slug, allowing for more comprehensive results.
- Introduced a mechanism to track ambiguous slugs across knowledge bases, improving clarity in search results.
These changes significantly improve the user experience in the chat interface and enhance the functionality of the wiki page retrieval system.
- Introduced a new wiki drawer component in AgentStreamDisplay for enhanced navigation to wiki pages, allowing users to view detailed content and related metadata.
- Implemented localization for the new "View in Graph" feature in both English and Chinese, improving accessibility for diverse user bases.
- Updated the WikiBrowser component to utilize the new drawer for displaying wiki content, streamlining the user experience when interacting with wiki links.
These enhancements significantly improve the usability and functionality of the wiki navigation system, providing users with a more intuitive experience when accessing and citing wiki content.
- Updated the agent system prompt to require wiki-links in citations, improving user navigation to source pages.
- Implemented logic in AgentStreamDisplay to handle wiki link clicks and keyboard interactions, facilitating seamless navigation to knowledge base wiki pages.
- Enhanced WikiBrowser to auto-select pages based on query parameters, improving user experience when accessing specific wiki content.
- Added localization support for new error messages related to knowledge base identification, ensuring clarity in user interactions.
These changes significantly improve the usability and functionality of the wiki navigation system, providing users with a more intuitive experience when accessing and citing wiki content.
- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings.
- The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities.
- Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system.
These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
- Introduced `enrichChunkImageInfo` function to populate image information for text chunks by retrieving data from child chunks, ensuring that image-related content is not lost during processing.
- Updated `reconstructContent` to `reconstructEnrichedContent`, allowing for the inclusion of OCR text and captions from images, improving the quality of content sent to the LLM.
- Added `writeDedupItemXML` and `xmlEscape` functions to format entity and concept entries as structured XML, enhancing clarity and preventing nonsensical merges in deduplication prompts.
- Refined the `deduplicateExtractedBatch` method to utilize the new XML formatting for better output consistency.
These changes significantly improve the handling of image data and the overall clarity of deduplication prompts, contributing to a more effective wiki content management system.