- Updated logic to manage <think> and </think> tags in chat message content, ensuring accurate representation of thinking states.
- Introduced a new function to handle the closing of thinking events, enhancing clarity and reducing redundancy in event emissions.
- Adjusted event handling for reasoning content to support historical message reconstruction, improving user experience during chat interactions.
Reconstructing full document content from chunks relied on position math
(offset = content length - (EndAt - lastEndAt)) that assumes
len([]rune(Content)) == EndAt - StartAt. Two upstream behaviors break that
invariant and caused the merged "全文" view to drop or duplicate text:
- The parent-child chunker prepends a synthetic table header to split
tables; that header is zero-width in position space (start == end), so
Content is longer than EndAt - StartAt.
- Chunk Content can retain HTML entities (", > ...), inflating the
rune count relative to the source span.
Replace position-based trimming with text-overlap matching: find where the
accumulated text's suffix reappears at the head of the next chunk and join
there. Positions are only used to size the search window. This naturally
skips the prepended header and is immune to entity length drift.
Extract the logic into searchutil.AppendWithOverlap / MergeTextChunks and
reuse it across the three backend stitching sites (wiki_ingest
reconstructContent, graph mergeChunkContents, chat_pipeline
mergeOverlappingChunks). Mirror the same algorithm in the frontend
doc-content merge. Also fix the markdown hr/heading styles (thin solid
divider, primary heading color).
The intent prompt override logic in query_understand applied
strings.TrimSpace to the value before assigning it as the system prompt
override, which silently stripped trailing newlines and intentional
formatting from agent-supplied prompts. Use TrimSpace only to detect
emptiness (so whitespace-only strings still fall back to the global
default) while passing the raw string through verbatim.
Extract the resolution into applyIntentPromptOverride and add unit tests
covering agent-wins, whitespace preservation, blank fallback, no-config,
and global-only paths.
When the rerank model returns an error (e.g. 401 Unauthorized), the
pipeline previously discarded all retrieved candidates and returned
empty results to the caller.
Now p.rerank returns ([]RankResult, error) to distinguish API failure
from threshold-filtered empty results. On API error, the pipeline
falls back to the original retrieval candidates (directLoad +
candidatesToRerank) and continues to CHUNK_MERGE/FILTER_TOP_K,
so users still get results even when the rerank model is misconfigured.
When a custom agent has `MultiTurnEnabled=false`, `applyAgentOverridesToChatManage`
sets `chatManage.MaxRounds = 0` to signal "no history". Two pipeline plugins
mistreated this zero value as "use the global default" and silently re-loaded
session history into the LLM context:
- `PluginLoadHistory` fell back to `Conversation.MaxRounds` when
`chatManage.MaxRounds == 0`.
- `PluginQueryUnderstand.loadHistory` had the same fallback, and even when
`LOAD_HISTORY` was skipped it would re-populate `chatManage.History`,
leaking previous turns into rewrite, image analysis, and the final answer.
The RAG branch in `session_knowledge_qa` also added `LOAD_HISTORY`
unconditionally, unlike the pure-chat branch which guarded it with `hasHistory`.
This change:
- Treats `chatManage.MaxRounds <= 0` as an explicit disable in both plugins;
no fallback to global config.
- Makes the RAG pipeline consistent with the pure-chat path by gating
`LOAD_HISTORY` on `hasHistory`.
- Removes the duplicated `Current Time: {{current_time}}` line from
`agent_system_prompt.yaml`. The agent already receives a fresh
`<runtime_context><current_time>` block with each turn from
`observe.buildRuntimeContextBlock`, so the static placeholder was
redundant.
The ReAct agent path (`session_agent_qa`) already checked `MultiTurnEnabled`
directly and is not affected.
Closes#1479
Previously the Langfuse timeline for a knowledge-chat request only
showed generations (chat.completion, embedding.embed, rerank), so the
work happening between them — query setup, vector/keyword search,
result merging, prompt assembly — appeared as unexplained gaps.
Add spans around the pieces that fill those gaps:
* qa.setup wraps request-time KB / model / search-target resolution
before the pipeline event loop starts, accounting for the visible
delay before the first chat.completion.
* pipeline.<event> wraps each pipeline stage's eventManager.Trigger
call so generations inside a stage nest under it. CHAT_COMPLETION_
STREAM is intentionally skipped because its OnEvent returns as soon
as the streaming goroutine starts; a stage span would always finish
before the chat.completion.stream generation it nominally parents.
* retrieve wraps the actual vector + keyword retrieve call inside
HybridSearch, exposing the DB round-trip that previously sat invisibly
between embedding generations and rerank.
* web_search wraps the external web-search HTTP call when enabled.
Quick-answer (RAG) agents can now configure a separate chat model for
the query-understanding stage (rewrite + intent classification),
decoupling it from the main conversation model so users can route the
lightweight rewrite call to a cheaper / faster model.
- Add CustomAgentConfig.QueryUnderstandModelID and plumb it through
PipelineRequest and ChatManage.Clone.
- query_understand plugin prefers QueryUnderstandModelID on the
text-only path; falls back to ChatModelID if the configured model
cannot be resolved (with a warn log). Multimodal path is unchanged
to keep vision-capable model selection intact.
- AgentEditorModal exposes a ModelSelector under the existing Query
Rewrite block; empty means reuse the main chat model.
- Add i18n strings (queryUnderstandModel / placeholder / desc) for
zh-CN, en-US, ko-KR, ru-RU.
- Extend Go SDK AgentConfig with the new field.
Closes#1113
The graph extraction pipeline relies on a strict markdown fence regex to
pull JSON out of LLM responses. In production the regex misses ~84% of
real-world responses, in three ways:
1. The LLM hits max_tokens mid-output and produces no closing fence.
2. The opening fence is malformed or surrounded by prose, so the
non-greedy regex fails to anchor a match.
3. The model returns raw JSON with stray backticks but no real fence.
In every case extractContent fell through to returning the raw text,
which then failed json.Unmarshal with errors like
"invalid character '` + "`" + "' looking for beginning of value".
This change keeps the existing happy path untouched and adds a
conservative recovery step in the default branch of extractContent:
- If an opening ``` is present, take everything after it, drop a
likely language tag on the first line, cut at any trailing closing
fence, and trim stray backticks/whitespace.
- Otherwise, look for an outermost JSON object/array in the text using
a small bracket-balanced scanner that respects string literals, so
embedded {} or [] inside JSON strings don't confuse it.
- Only fall back to the original raw-text behavior when neither
strategy yields anything plausible.
The recovery helpers (stripFencesAndExtract, extractJSONLike,
isLikelyLanguageTag) are package-private and have table-driven tests in
extract_entity_test.go covering the three failure patterns described in
the issue, plus the previously-working fenced and bare-JSON shapes to
guard against regressions.
AgentFinalAnswer may carry Done:true more than once (e.g. finish chunk
plus EOF). Guard AddEpisode with sync.Once so memory persists once per
turn, consistent with qa.go completion handling.
Co-authored-by: Cursor <cursoragent@cursor.com>
- Introduced `encodeSlugPath` function to safely encode wiki slugs for API requests, preserving hierarchical routing.
- Updated API functions to utilize the new slug encoding for `getWikiPage`, `updateWikiPage`, and `deleteWikiPage`.
- Enhanced `ListBySourceRef` method in the repository to safely handle source knowledge IDs with JSON marshalling, improving query security.
- Added `escapeLikePattern` function to ensure safe concatenation of LIKE patterns in SQL queries, preventing unintended matches.
- Updated `KnowledgePostProcessService` to include Redis client for task management, improving performance in handling wiki ingest tasks.
- Enhanced `WikiLintIssue` structure to include `TargetSlug` for better issue tracking and auto-fixing capabilities.
These changes significantly improve the robustness and security of the wiki management system, enhancing user experience and data integrity.
Added functionality to enable updates for the `wiki_config` field in the `KnowledgeBaseConfig` struct. This includes:
1. **Type Definition Update**: Added `WikiConfig` field to `KnowledgeBaseConfig` to allow JSON unmarshaling during update requests.
2. **Service Layer Enhancement**: Updated the `UpdateKnowledgeBase` method to persist changes to `wiki_config` when provided in the request.
3. **Frontend TypeScript Adjustments**: Enhanced type definitions in the frontend to support `wiki_config`, improving IDE autocomplete and type checking.
This fix addresses the critical issue where `wiki_config` was not being updated due to its absence in the `KnowledgeBaseConfig` struct, ensuring proper handling of wiki configurations during knowledge base updates.
This commit introduces the complete wiki feature for WeKnora, enabling AI-powered wiki page generation and management. The implementation includes:
**Backend Changes:**
- Wiki data model: WikiPage type with support for multiple page types (Summary, Entity, Concept, Index, Log)
- Database schema: wiki_pages table with full migration support
- WikiPageService: CRUD operations and page management
- WikiPageRepository: GORM-based persistence layer
- Wiki ingest pipeline: Automated generation of wiki pages from knowledge documents
* Summary page generation using LLM
* Entity and concept extraction in a single LLM call
* Synthesis opportunity detection
* Index page rebuilding
* Log page maintenance
- Wiki boost feature: Enhance chat retrieval with wiki context
- Wiki linting: Maintenance and validation utilities
- Agent wiki tools: Enable agents to query and interact with wiki pages
- Wiki prompts: Comprehensive LLM prompt templates for all wiki generation tasks
- Language support: Reuse existing middleware language infrastructure for LLM prompts
**Frontend Changes:**
- Wiki browser UI: View all wiki pages with filtering and search
- Wiki API client: Knowledge base wiki management endpoints
- Knowledge base editor: Configure wiki settings (language, auto-ingest, synthesis model)
- i18n updates: Support for English, Korean, Russian, and Chinese interfaces
**Configuration:**
- Container DI: Wire up all wiki services
- Router: Register wiki API endpoints
- Task handling: Support async wiki ingest tasks
**Testing:**
- Unit tests for wiki page types
- Service layer tests
- Endpoint tests for wiki operations
- Integration tests with LLM mocking
**Documentation:**
- Language refactoring analysis and guides
- Implementation completion reports
- Quick reference guides for developers
**Key Features:**
✅ LLM-powered wiki page generation from documents
✅ Multi-language support (9+ languages)
✅ Automatic extraction of entities and concepts
✅ Synthesis opportunity detection
✅ Index and log page maintenance
✅ Progressive wiki building across multiple documents
✅ Agent-based wiki interaction
✅ Chat retrieval enhancement with wiki context
✅ Full frontend UI for wiki browsing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Adjusted the expanded TopK limit to a maximum of 200 candidates to enhance HNSW efficiency.
- Updated the vector retrieval query to ensure the ORDER BY expression matches the indexed expression, improving index usage.
- Implemented local settings for HNSW's ef_search and iterative_scan within a transaction to optimize candidate retrieval.
- Added fallback logic to handle unsupported GUCs gracefully, ensuring query execution continues even with older pgvector versions.
These changes improve the performance and reliability of vector retrieval in the PostgreSQL implementation.
- Updated the prompt template to clarify the task of understanding and rewriting user questions, improving the instructions for coreference resolution and ellipsis completion.
- Renamed sections for better clarity, changing "Rewriting Goals" to "Query Understanding."
- Added critical notes for intent classification regarding image and document attachments, ensuring accurate processing based on the presence of attachments.
- Improved examples to reflect the updated intent classification logic, enhancing the overall user experience in query handling.
- Added a new logger for LLM calls, enabling detailed logging of model interactions, including request and response data.
- Introduced configuration options for enabling and specifying the log directory for LLM debug logs.
- Implemented cleanup for old log files to manage disk space effectively.
- Wrapped existing chat implementations to log calls when debug logging is enabled, enhancing traceability for model interactions.
- Updated the query content to include metadata for uploaded images and attachments, improving the context provided to the assistant.
- Modified the attachment prompt format to use XML-like tags for better structure and readability, including metadata for file type and size.
- Enhanced error handling for unsupported file content extraction, providing clearer feedback to users.
This update improves the assistant's ability to process and respond to queries involving multimedia content.
- Enhanced the logic for replacing child content with parent content to only apply to text chunks whose parent is a parent_text chunk.
- Added checks to skip non-text chunks and ensure that only appropriate parent chunks are used for replacement, improving content quality and integrity.
- Updated relevant files to include provider registration, implementation, and metadata.
- Enhanced frontend components to support provider management and configuration.
- Added localization for new provider settings and messages.
- Implemented backend repository methods for CRUD operations on web search providers.
- Updated context template to include runtime metadata such as current time and week, improving contextual awareness in user queries.
- Enhanced rewrite template with critical instructions for intent classification, ensuring that rewritten questions preserve essential entities and keywords.
- Refined intent classification logic to prioritize user intents more effectively, improving the accuracy of responses based on user queries.
- Added examples to clarify expected input and output formats for intent classification, enhancing usability for developers.
- Introduced a new `channel` field in the Knowledge struct and associated request types to track the source channel (e.g., "web", "api", "browser_extension").
- Updated various frontend components to display channel information and enhance user experience with channel labels.
- Enhanced localization files to support channel labels in English and Chinese.
- Modified backend services and database migrations to accommodate the new channel feature, ensuring consistent tracking across knowledge entries.
- Refactored related functions to integrate channel handling, improving overall knowledge management and context.
- Updated regex patterns for table separator and row matching to use [ \t] instead of \s, preventing newline consumption across rows.
- Enhanced comments for clarity on the changes made to the regex expressions.
- Updated the merging process in the chat pipeline to include a new step for re-merging overlapping ranges introduced by context expansion.
- Implemented a new function, removePartialOverlaps, to eliminate chunks that are largely contained within higher-scored chunks, improving deduplication accuracy.
- Enhanced normalization and overlap checking utilities to support the new deduplication logic.
- Updated the assistant descriptions in various prompt templates to specify that they are developed by Tencent.
- Added new intent-specific system prompt templates for greeting, chitchat, follow-up, image analysis, summarization, and web search unavailability scenarios.
- Enhanced citation rules in the system prompts to ensure proper inline citation formatting.
- Refactored the rewrite prompt to classify user intent more accurately and output structured JSON responses.