Move batch reparse work to an asynq task to avoid HTTP timeouts, add i18n
for all locales, guard in-flight documents, and prevent concurrent delete/reparse.
This commit introduces the Batch Reparse feature for the Knowledge Base.
Users can now select multiple documents and trigger a re-parsing pipeline in batches, streamlining the recovery from parse failures.
Add recent=true to knowledge search so the mention picker can list recent
files without a keyword, and consolidate KB/FAQ/file chip styling across
chat input and message history.
- Enhanced the `reparseKnowledge` function to accept optional process configuration overrides, allowing users to adjust parsing settings during reparse operations.
- Updated the frontend components to integrate reparse capabilities, including the `UploadConfirmDialog` and `KnowledgeBase` views, providing a user-friendly interface for confirming reparse actions.
- Added new localization strings for reparse confirmation and process configuration details in multiple languages.
- Modified the backend API to handle reparse requests with optional configuration, ensuring proper validation and persistence of overrides.
- Introduced new types and interfaces to manage reparse-specific data structures in the store, improving type safety and clarity in the codebase.
- Updated the `CreateKnowledgeFromFile` function to accept an optional `processConfig` parameter for per-upload parse configuration overrides.
- Introduced the `KnowledgeProcessOverrides` type to define various parsing settings, including parser engine rules and chunking configurations.
- Modified the API client and related components to handle the new process configuration, improving flexibility in knowledge file uploads.
- Updated documentation and API specifications to reflect the new `process_config` parameter in the upload process.
- Added a new `UploadConfirmHost` component to facilitate user confirmation for uploads, integrating with the updated upload logic.
- Enhanced user experience by providing detailed configuration options during the upload process.
Lets users stop an in-flight document parse to free up LLM / worker
resources without losing the chunks and index already written. The
core insight is that the previous parse_status=completed flipped as
soon as primary chunks landed, while the most expensive subtasks
(graph extract = N LLM calls per chunk, plus summary, question
generation) were still running in the background — so "completed"
wasn't actually terminal from a resource standpoint.
State machine
pending -> processing -> finalizing -> completed
|
+-> cancelled (any of the three
in-flight states)
+-> failed
+-> deleting
`finalizing` is the new post-process fan-out window. parse_status
only promotes to `completed` once pending_subtasks_count (a new
column tracking summary + question + per-chunk graph extract)
drains to zero via atomic FinalizeSubtask. Wiki ingest is
intentionally excluded from the counter — it's a KB-scoped
debounced batch and would otherwise pin parse_status in
`finalizing` for the wiki batch window.
Backend
- New ParseStatusFinalizing + pending_subtasks_count column with
migration 000056.
- knowledgeRepository.SetFinalizing transitions processing -> finalizing
conditionally so a racing cancel cannot be clobbered.
- knowledgeRepository.FinalizeSubtask atomically decrements the
counter and self-promotes the row to completed when it hits zero.
- KnowledgePostProcess restructured to compute expected subtask
count up front, flip to finalizing (or completed when no
enrichment is enabled), and only then fan out subtasks. Subtask
handlers (summary, question, graph extract) defer-decrement on
terminal exit using the existing isFinalAsynqAttempt convention.
- New POST /api/v1/knowledge/{id}/cancel-parse handler accepting
pending / processing / finalizing. Marks the row cancelled,
zeroes the counter, best-effort dequeues asynq tasks via a new
TaskInspector abstraction (asynq-mode walks pending/scheduled/
retry queues; Lite-mode noop), and scrubs wiki ingest pending op.
- SpanTracker.AbortAttempt flat-sweeps every still-running span
for the attempt via a new repo.CancelAllOpenSpans helper so the
trace viewer's striped bars all flip to cancelled, even leaf
generations whose parent stage already EndSpan'd (multimodal
fan-out pattern). knowledge_post_process closes its postSpan
via SkipSpan on the cancel/deleting entry guard so a worker
that opens a span AFTER the cancel sweep doesn't leak it.
- Housekeeping and resetPendingTasks sweep finalizing rows
identically to processing so a crash/restart can't strand them.
- DeleteKnowledge/DeleteKnowledgeList proactively dequeue
downstream tasks via the same TaskInspector path.
- ChunkExtractService gets a cancel entry guard so the most
expensive enrichment (graph extract) bails immediately when the
parent knowledge is aborted.
Frontend
- New cancelKnowledgeParse API client + "Stop parsing" entry in
both list view and card view more menus, gated on
pending/processing/finalizing.
- Polling predicate refactored to a shared isParseInFlight helper
that recognises `finalizing` (previously the doc list silently
stopped polling once parse_status flipped from processing).
- Knowledge processing timeline: isPolling includes finalizing,
new isHardTerminal short-circuits LIVE for cancelled/failed/
completed so stranded child spans cannot pin LIVE on.
- DocumentListView.computeStatus distinguishes finalizing
("增强中") from completed and shows the previous "生成摘要中"
copy when summary_status is still pending under finalizing.
Added cancelled badge as well.
- i18n: statusFinalizing / statusCancelled / cancelParse* keys
across zh-CN, en-US, ko-KR, ru-RU.
Docs / SDK
- docs/api/knowledge.md: documents the new finalizing state,
cancel-parse semantics, and which statuses accept cancel.
- client (Go SDK): CancelKnowledgeParse with docstring listing
the cancellable statuses.
Renders a five-segment timeline (DocReader → Chunking → Embedding →
Multimodal → PostProcess) above each document so users can see exactly
which stage a document is in and which one failed, instead of a flat
"Parsing..." spinner.
- New KnowledgeProcessingTimeline component (full + compact mode):
full mode lives at the top of doc-content for any document not yet
in `completed`; compact mode replaces the parsing/failed lines in
the card hover popover with a five-dot mini-progress.
- Polls GET /api/v1/knowledge/:id/spans every 2s while
parse_status in {pending, processing}; stops on terminal status or
unmount. Hover popover instances opt out of polling to avoid
amplifying requests on grid hover.
- Failed / cancelled steps expand into an error card with the raw
error_message, an error_code -> localized title + suggestion mapping
for the twelve canonical codes (DOCREADER_TIMEOUT, EMBEDDING_RATE_LIMIT,
...), and a "Copy details" button that pastes the failed span JSON
for support tickets.
- Attempt selector renders only when latest_attempt > 1 so reparse
history is browsable; the latest attempt is selected by default.
- Last-error block under the timeline includes a Retry button that
calls reparseKnowledge() and immediately re-fetches.
- i18n strings added under knowledgeStages for en-US, zh-CN, ko-KR,
ru-RU. ja-JP locale file does not exist in this repository.
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.
Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.
Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.
KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.
The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.
The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).
Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
Introduced a new API client for managing user favorites, allowing users to list, add, and remove starred resources. Enhanced the sidebar to display favorites and recents, improving user navigation. Updated internationalization files to include relevant labels and tooltips for the new features across multiple languages.
Refs: #1303
Add three new optional filters to the document list under a knowledge base
detail page — parse status, source/channel, and updated time range — and
rework multi-select to no longer cause the card title to jitter on hover.
Backend
- Introduce types.KnowledgeListFilter to aggregate optional filter dimensions
(tag, keyword, file_type, parse_status, source, updated_from/to) and switch
ListPagedKnowledgeByKnowledgeBaseID (repository/service/interface) to accept
it instead of a growing positional parameter list.
- The ListKnowledge HTTP handler accepts new parse_status, source, start_time
and end_time query params; time params accept RFC3339, "YYYY-MM-DD HH:MM:SS"
and "YYYY-MM-DD". The repository routes source="manual"/"url" onto the type
column to stay consistent with file_type semantics; other source values match
the channel column.
- Update the four other callers (agent_service, initialization) to pass an
empty filter struct, preserving prior behavior.
Frontend
- Add three controls in the doc-filter-bar (status select, source select,
date-range picker with future-date disabled) wired through getKnowled /
listKnowledgeFiles into the new backend params.
- Replace the hover-triggered card checkbox with an explicit "批量管理" mode
(mirrors the session list UX): in card view the checkbox only renders while
batch mode is on, entered via the per-card "..." menu; the list view keeps
its leading checkbox column. Switching from list to grid auto-enables batch
mode when something is already selected, so the selection stays visible.
- DocumentBatchBar now stays open whenever batch mode or selection > 0, and
its "取消选择" button both clears the selection and exits batch mode.
API surface sync
- Regenerate Swagger artifacts (docs/docs.go / swagger.json / swagger.yaml).
- Update docs/api/knowledge.md with the new query parameters.
- Add backward-compatible ListKnowledgeWithFilter + KnowledgeListFilter to the
Go SDK; the existing ListKnowledge keeps its signature.
i18n
- New filter labels in zh-CN / en-US / ko-KR / ru-RU; reuse existing
menu.batchManage / batchManage.cancel for the multi-select strings.
Add a list-view alongside the existing card grid (issue #957) and
multi-select batch delete (issue #1045) for documents inside a knowledge
base. The two views are toggled from the toolbar and the preference is
persisted in localStorage. Selection state is shared between views with
shift-range support and a sticky batch-action bar.
Backend
- POST /api/v1/knowledge/batch-delete: validates KB scope and editor
permission, then delegates to the existing DeleteKnowledgeList service
which already cascades vector/file/graph cleanup. Caps batch size at
200 and uses a single GetKnowledgeBatch call for membership validation.
Frontend
- DocumentListView: table layout with sticky header, checkbox column
(with indeterminate select-all), file-type icons, status badges,
per-row action menu.
- DocumentBatchBar: floating pill bar with selected count and delete
action; appears when items are selected.
- KnowledgeBase.vue: view-mode toggle, selection state, batch-delete
confirmation dialog, hover/active checkbox overlay on existing cards.
- Extracted formatFileSize and getFileIcon to utils/files.ts to share
with the new list view and avoid further drift from the existing
inline copies.
- i18n keys added for zh-CN, en-US, ko-KR, ru-RU.
Closes#1045Closes#957
- Introduced extraction granularity settings in the wiki configuration, allowing users to control the level of detail in entity and concept extraction (focused, standard, exhaustive).
- Updated the UI to reflect these new options, including tooltips for better user guidance.
- Refactored the knowledge base editor to support the new extraction granularity feature, ensuring a seamless user experience.
- Improved the backend logic for wiki ingest tasks to utilize the new granularity settings, enhancing the accuracy of extracted content.
- Added tests to validate the new extraction granularity functionality and ensure robust performance across different configurations.
- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings.
- The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities.
- Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system.
These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
Implemented three critical fixes to enable wiki_config persistence during
KB creation and updates:
1. Add WikiConfig field to KnowledgeBaseConfig struct
- File: internal/types/knowledgebase.go
- Enables JSON unmarshaling of wiki_config from update requests
- Previously the field was silently dropped during deserialization
2. Update UpdateKnowledgeBase service to persist wiki_config
- File: internal/application/service/knowledgebase.go
- Added logic to apply wiki_config changes when config parameter is provided
- Mirrors existing logic for FAQConfig persistence
3. Update frontend TypeScript types for IDE completion
- File: frontend/src/api/knowledge-base/index.ts
- Added WikiConfig type to createKnowledgeBase function signature
- Added structured WikiConfig type to updateKnowledgeBase config parameter
- Enables TypeScript IDE autocomplete and type checking
Flow:
- Frontend: updateKnowledgeBase({name, description, config: {wiki_config}})
- Handler: Parses into UpdateKnowledgeBaseRequest via KnowledgeBaseConfig
- Service: Applies wiki_config to KB entity via UpdateKnowledgeBase
- Database: Persists wiki_config JSONB field to postgres
Root cause analysis:
Previously, the KnowledgeBaseConfig struct was missing the WikiConfig field,
causing JSON unmarshaling to silently drop any wiki_config data sent in
PUT requests. KB creation worked because it used the full KnowledgeBase struct.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit introduces the complete wiki feature for WeKnora, enabling AI-powered wiki page generation and management. The implementation includes:
**Backend Changes:**
- Wiki data model: WikiPage type with support for multiple page types (Summary, Entity, Concept, Index, Log)
- Database schema: wiki_pages table with full migration support
- WikiPageService: CRUD operations and page management
- WikiPageRepository: GORM-based persistence layer
- Wiki ingest pipeline: Automated generation of wiki pages from knowledge documents
* Summary page generation using LLM
* Entity and concept extraction in a single LLM call
* Synthesis opportunity detection
* Index page rebuilding
* Log page maintenance
- Wiki boost feature: Enhance chat retrieval with wiki context
- Wiki linting: Maintenance and validation utilities
- Agent wiki tools: Enable agents to query and interact with wiki pages
- Wiki prompts: Comprehensive LLM prompt templates for all wiki generation tasks
- Language support: Reuse existing middleware language infrastructure for LLM prompts
**Frontend Changes:**
- Wiki browser UI: View all wiki pages with filtering and search
- Wiki API client: Knowledge base wiki management endpoints
- Knowledge base editor: Configure wiki settings (language, auto-ingest, synthesis model)
- i18n updates: Support for English, Korean, Russian, and Chinese interfaces
**Configuration:**
- Container DI: Wire up all wiki services
- Router: Register wiki API endpoints
- Task handling: Support async wiki ingest tasks
**Testing:**
- Unit tests for wiki page types
- Service layer tests
- Endpoint tests for wiki operations
- Integration tests with LLM mocking
**Documentation:**
- Language refactoring analysis and guides
- Implementation completion reports
- Quick reference guides for developers
**Key Features:**
✅ LLM-powered wiki page generation from documents
✅ Multi-language support (9+ languages)
✅ Automatic extraction of entities and concepts
✅ Synthesis opportunity detection
✅ Index and log page maintenance
✅ Progressive wiki building across multiple documents
✅ Agent-based wiki interaction
✅ Chat retrieval enhancement with wiki context
✅ Full frontend UI for wiki browsing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Added ASR configuration options to the knowledge base and model settings.
- Implemented ASR model selection and transcription capabilities in the knowledge processing pipeline.
- Enhanced file upload support to include audio formats for ASR processing.
- Updated localization files to include ASR-related labels and descriptions.
- Refactored model handling to accommodate ASR as a new model type.
The backend introduced storage_provider_config as the authoritative
field in commit dbab7b0, but the frontend was never updated to use it.
This caused the UI to always display "local" regardless of the actual
storage provider, and saving KB settings could overwrite the correct
storage_provider_config with an incorrect value.
Changes:
- KnowledgeBaseEditorModal.vue: read from storage_provider_config with
storage_config fallback, dual-write both fields on save
- KnowledgeBaseList.vue: update feature badge logic and add type def
for storage_provider_config, exclude "local" from badge display
- api/knowledge-base/index.ts: add storage_provider_config to API type
- ko-KR.ts, ru-RU.ts: add missing i18n keys — multimodal section
(sidebar.multimodal, multimodal.title/description), channel labels
(channelLabel, channelWeb, channelApi, etc.), and agent.think
- Added API endpoints for moving knowledge items between knowledge bases, including validation for source and target compatibility.
- Implemented asynchronous task handling for knowledge moves, tracking progress and status updates.
- Introduced new localization strings for move-related UI elements in English, Korean, Russian, and Chinese.
- Enhanced the frontend to support knowledge move operations, including target selection and progress monitoring.
These changes significantly improve the application's capabilities for managing knowledge across different bases, providing users with a seamless experience in organizing their knowledge assets.
- Introduced a new API endpoint to toggle the pin status of knowledge bases, allowing users to pin important entries to the top of the list.
- Updated the frontend to include UI elements for pinning and unpinning knowledge bases, with corresponding success and error messages.
- Enhanced localization support by adding translations for pinning actions in English, Korean, Russian, and Chinese.
- Modified the database schema to include `is_pinned` and `pinned_at` fields for knowledge bases, enabling persistent pinning status.
These changes improve the organization and accessibility of knowledge bases, enhancing user experience by allowing quick access to prioritized entries.
- Introduced a new API function `reparseKnowledge` to trigger the re-parsing of knowledge documents.
- Updated the KnowledgeBase component to include a user interface option for document re-parsing, with confirmation prompts and status messages.
- Added corresponding translations for the reparse feature in both English and Chinese locales.
These changes enhance the document management capabilities by allowing users to rebuild knowledge documents, improving the overall user experience.
- Updated `config.yaml` to adjust conversation parameters for improved search performance.
- Added a new `knowledgeSemanticSearch` API function for semantic search capabilities.
- Introduced new SVG icons for search functionality.
- Created `KnowledgeSearch.vue` component for the search interface, including query input and result display.
- Enhanced menu and routing to support navigation to the new knowledge search page.
- Updated internationalization files to include translations for the knowledge search feature.
This update provides users with a powerful tool to perform semantic searches across knowledge bases, improving content discoverability.
- Updated the `KnowledgeBase` struct to rename `cos_config` to `storage_config` for consistency across the application.
- Implemented a custom `UnmarshalJSON` method to maintain backward compatibility with legacy responses that still use `cos_config`.
- Modified various frontend components to reflect the new `storage_config` naming, ensuring seamless integration with the updated backend structure.
- Enhanced Dockerfiles and service configurations to support the new storage engine setup.
This update improves the clarity of storage configurations and ensures compatibility with existing data structures.
- Added a new `.env.lite.example` file for the Lite version, providing a minimal configuration template.
- Updated `.env.example` to remove deprecated variables and include new Docreader settings.
- Enhanced Docker configurations to support the Lite version, including a new Dockerfile for the Docreader service.
- Introduced a Makefile target for building and running the Lite version, along with packaging capabilities.
- Created GitHub workflows for building and releasing Lite binaries, including Homebrew formula support.
- Implemented a new service file for managing the Lite version as a system service.
This update enables a streamlined, single-binary deployment of WeKnora, reducing external dependencies and simplifying setup.
- Updated the AgentSelector component to improve agent selection logic, ensuring accurate representation of selected agents.
- Introduced new AgentShareSettings component for managing agent sharing with organizations, including UI for selecting organizations and displaying shared agents.
- Enhanced API interactions for sharing agents, including new methods for listing and managing agent shares.
- Improved localization support for agent sharing features across multiple languages.
- Updated various components to reflect changes in agent capabilities and sharing status, enhancing user experience and clarity.
This update strengthens the agent management features, providing users with better tools for sharing and selecting agents within organizations.
- Introduced a new API for organization management, including creation, updating, and deletion of organizations.
- Implemented organization sharing functionality for knowledge bases, allowing users to share knowledge bases with specific organizations and set permissions.
- Added UI components for organization selection and sharing dialogs.
- Enhanced existing components to support organization-related features, including shared knowledge base handling and organization role management.
This update enhances collaboration capabilities within the application by allowing users to manage and share knowledge bases across different organizations.
- Reformatted the API functions in the knowledge base module for consistent indentation and spacing.
- Enhanced overall code readability without altering functionality.
- Introduced tag_id parameter in various knowledge-related API functions to allow categorization of knowledge entries.
- Updated frontend components to handle tag_id during knowledge creation and file uploads.
- Enhanced UI store to manage the selected tag ID for better user experience in categorizing knowledge.
- Updated the `uploadKnowledgeFile` function to accept an optional progress callback, enabling real-time upload progress tracking.
- Refactored the document upload handling in the menu component to support multiple file uploads with individual progress events.
- Removed deprecated document processing state management from the KnowledgeBase view, streamlining the upload process.
- Improved user feedback during file uploads by dispatching events for upload start, progress, and completion.
- Replaced `vlm_model_id` with a structured `vlm_config` object in knowledge base configurations, including `enabled` flag and `model_id`.
- Updated API documentation to reflect changes in VLM configuration structure.
- Enhanced frontend interfaces to accommodate new VLM configuration format.
- Improved localization for new features related to knowledge graph and multimodal capabilities.
- Added API functions for listing, creating, updating, and deleting knowledge base tags.
- Introduced batch update functionality for enabling/disabling FAQ entries and updating their tags.
- Enhanced the KnowledgeBase.vue and FAQEntryManager.vue components to support tag management and batch operations.
- Updated localization files to include new strings related to tag management and batch operations.