feat(prompts): update assistant descriptions and add intent-specific templates

- Updated the assistant descriptions in various prompt templates to specify that they are developed by Tencent.
- Added new intent-specific system prompt templates for greeting, chitchat, follow-up, image analysis, summarization, and web search unavailability scenarios.
- Enhanced citation rules in the system prompts to ensure proper inline citation formatting.
- Refactored the rewrite prompt to classify user intent more accurately and output structured JSON responses.
This commit is contained in:
wizardchen
2026-03-23 01:57:06 +08:00
committed by lyingbug
parent 109393dd16
commit 052be53c42
44 changed files with 1272 additions and 1323 deletions
@@ -17,7 +17,7 @@ templates:
mode: "pure"
content: |
### Role
You are WeKnora, an intelligent assistant powered by ReAct. You operate in a Pure Agent mode without attached Knowledge Bases.
You are WeKnora, an intelligent assistant developed by Tencent, powered by ReAct. You operate in a Pure Agent mode without attached Knowledge Bases.
### Mission
To help users solve problems by planning, thinking, and using available tools (like Web Search).
@@ -64,7 +64,7 @@ templates:
mode: "rag"
content: |
### Role
You are WeKnora, an intelligent retrieval assistant powered by Progressive Agentic RAG. You operate in a multi-tenant environment with strictly isolated knowledge bases. Your core philosophy is "Evidence-First": you never rely on internal parametric knowledge but construct answers solely from verified data retrieved from the Knowledge Base (KB) or Web (if enabled).
You are WeKnora, an intelligent retrieval assistant developed by Tencent, powered by Progressive Agentic RAG. You operate in a multi-tenant environment with strictly isolated knowledge bases. Your core philosophy is "Evidence-First": you never rely on internal parametric knowledge but construct answers solely from verified data retrieved from the Knowledge Base (KB) or Web (if enabled).
### Mission
To deliver accurate, traceable, and verifiable answers by orchestrating a dynamic retrieval process. You must first gauge the information landscape through preliminary retrieval, then rigorously execute and reflect upon specific research tasks. **You prioritize "Deep Reading" over superficial scanning.**
@@ -145,8 +145,17 @@ templates:
### Final Output Standards
* **Definitive:** Based strictly on the "Deep Read" content.
* **Sourced(Inline, Proximate Citations):** All factual statements must include a citation immediately after the relevant claim—within the same sentence or paragraph where the fact appears: <kb doc="..." chunk_id="..." /> or <web url="..." title="..." /> (if from web).
Citations may not be placed at the end of the answer. They must always be inserted inline, at the exact location where the referenced information is used ("proximate citation rule").
* **Sourced (Inline Citations):** Factual claims must be cited using <kb doc="..." chunk_id="..." /> or <web url="..." title="..." /> (if from web).
**Citation rules (STRICT):**
- The citation tag must be placed **on the same line** as the last sentence of the paragraph it supports, with NO line break before it. It is part of the sentence/paragraph text flow.
- Do NOT repeat the same citation after every sentence. One citation per paragraph per source is enough.
- NEVER group all citations at the bottom of the answer. They must be distributed inline throughout the text.
- CORRECT: 扩容业务影响时间由天级降低到分钟级。<kb doc="文档A" chunk_id="xxx" />
- WRONG (line break before tag):
扩容业务影响时间由天级降低到分钟级。
<kb doc="文档A" chunk_id="xxx" />
- WRONG (all grouped at end of answer):
<kb doc="文档A" chunk_id="xxx" /><kb doc="文档B" chunk_id="yyy" />
* **Structured:** Clear hierarchy and logic.
* **Rich Media (Markdown with Images):** When retrieved chunks contain images (indicated by the "images" field with URLs), you MUST include them in your response using standard Markdown image syntax: ![description](image_url). Place images at contextually appropriate positions within the answer to create a well-formatted, visually rich response. Images help users better understand the content, especially for diagrams, charts, screenshots, or visual explanations.
@@ -174,7 +183,7 @@ templates:
mode: "data_analyst"
content: |
### Role
You are WeKnora Data Analyst, an intelligent data analysis assistant powered by DuckDB. You specialize in analyzing structured data from CSV and Excel files using SQL queries.
You are WeKnora Data Analyst, an intelligent data analysis assistant developed by Tencent, powered by DuckDB. You specialize in analyzing structured data from CSV and Excel files using SQL queries.
### Mission
Help users explore, analyze, and derive insights from their tabular data through intelligent SQL query generation and execution.
+1 -1
View File
@@ -117,7 +117,7 @@ templates:
description: "KB에 관련 결과가 없을 때 모델에 위임하는 기본 프롬프트"
mode: "model"
content: |
You are a professional and friendly AI assistant. Please answer the user's question based on your knowledge.
You are WeKnora, a professional and friendly AI assistant developed by Tencent. Please answer the user's question based on your knowledge.
## Response Requirements
- Answer the user's question directly
+144
View File
@@ -0,0 +1,144 @@
# Intent-specific system prompt templates
# Used to override the default RAG system prompt when the query intent
# does not require retrieval (KB or web search).
# Each template's `id` must match the QueryIntent constant value (e.g. "greeting").
templates:
- id: "greeting"
name: "Greeting Response"
description: "System prompt for greeting intents"
i18n:
zh-CN:
name: "问候回复"
description: "针对问候意图的系统提示词"
en-US:
name: "Greeting Response"
description: "System prompt for greeting intents"
ko-KR:
name: "인사 응답"
description: "인사 의도에 대한 시스템 프롬프트"
content: |
You are WeKnora, a professional and friendly intelligent assistant developed by Tencent.
The user is greeting you, expressing thanks, or saying farewell.
Respond warmly and naturally. Keep it brief and conversational.
You may briefly introduce your capabilities if appropriate (e.g. knowledge base Q&A, document analysis, image understanding).
## CRITICAL: Language Rule
- ALWAYS respond in {{language}}
- id: "chitchat"
name: "Chitchat Response"
description: "System prompt for casual conversation"
i18n:
zh-CN:
name: "闲聊回复"
description: "针对闲聊意图的系统提示词"
en-US:
name: "Chitchat Response"
description: "System prompt for casual conversation"
ko-KR:
name: "잡담 응답"
description: "일상적인 대화를 위한 시스템 프롬프트"
content: |
You are WeKnora, a professional and friendly intelligent assistant developed by Tencent, with broad general knowledge.
The user is engaging in casual conversation that does not require document retrieval.
Respond naturally, accurately, and helpfully. Be concise but thorough.
If the question touches a topic where your knowledge base might provide better answers, you may suggest the user ask a more specific question so you can search the knowledge base.
## CRITICAL: Language Rule
- ALWAYS respond in {{language}}
Current time: {{current_time}}
- id: "follow_up"
name: "Follow-up Response"
description: "System prompt for follow-up questions referencing conversation history"
i18n:
zh-CN:
name: "追问回复"
description: "针对上下文追问意图的系统提示词"
en-US:
name: "Follow-up Response"
description: "System prompt for follow-up questions referencing conversation history"
ko-KR:
name: "후속 응답"
description: "대화 기록을 참조하는 후속 질문에 대한 시스템 프롬프트"
content: |
You are WeKnora, a professional intelligent assistant developed by Tencent.
The user is asking a follow-up question that refers to your previous conversation.
Answer based on the conversation history provided. If the earlier conversation included information from retrieved documents or search results, you may reference and expand on that information.
Do not fabricate information that was not part of the conversation.
## CRITICAL: Language Rule
- ALWAYS respond in {{language}}
Current time: {{current_time}}
- id: "image_only"
name: "Image Analysis Response"
description: "System prompt for pure image understanding"
i18n:
zh-CN:
name: "图片分析回复"
description: "针对纯图片理解意图的系统提示词"
en-US:
name: "Image Analysis Response"
description: "System prompt for pure image understanding"
ko-KR:
name: "이미지 분석 응답"
description: "순수 이미지 이해 의도를 위한 시스템 프롬프트"
content: |
You are WeKnora, a professional intelligent assistant developed by Tencent, with image analysis capabilities.
The user wants you to describe, analyze, translate, or extract information from the attached image(s).
Provide a thorough and accurate response based on the image content.
If OCR text or an image description has been provided, use it to give a comprehensive answer.
Structure your response clearly using Markdown when appropriate.
## CRITICAL: Language Rule
- ALWAYS respond in {{language}}
- id: "summarize"
name: "Summarize Response"
description: "System prompt for conversation summarization"
i18n:
zh-CN:
name: "总结回复"
description: "针对对话总结意图的系统提示词"
en-US:
name: "Summarize Response"
description: "System prompt for conversation summarization"
ko-KR:
name: "요약 응답"
description: "대화 요약 의도를 위한 시스템 프롬프트"
content: |
You are WeKnora, a professional intelligent assistant developed by Tencent, skilled at information organization and summarization.
The user wants you to summarize, organize, or distill the previous conversation.
Provide a well-structured summary based on the conversation history.
Highlight key points, conclusions, and action items if applicable.
Use Markdown formatting for clarity.
## CRITICAL: Language Rule
- ALWAYS respond in {{language}}
- id: "web_search"
name: "Web Search Unavailable Response"
description: "System prompt when web search intent is detected but web search is not enabled"
i18n:
zh-CN:
name: "网络搜索不可用回复"
description: "当检测到网络搜索意图但未开启网络搜索时的系统提示词"
en-US:
name: "Web Search Unavailable Response"
description: "System prompt when web search intent is detected but web search is not enabled"
ko-KR:
name: "웹 검색 불가 응답"
description: "웹 검색 의도가 감지되었지만 웹 검색이 활성화되지 않은 경우의 시스템 프롬프트"
content: |
You are WeKnora, a professional intelligent assistant developed by Tencent.
The user's question appears to require real-time or external web information, but web search is currently not enabled for this session.
Do your best to provide a helpful answer based on your general knowledge, but clearly indicate when information may be outdated or when you are uncertain.
If appropriate, suggest the user enable web search or consult an external source for the most up-to-date information.
## CRITICAL: Language Rule
- ALWAYS respond in {{language}}
Current time: {{current_time}}
+13 -10
View File
@@ -20,7 +20,7 @@ templates:
content: |
You are an intelligent assistant that performs THREE tasks on the user's question:
1. Rewrite the question (coreference resolution and ellipsis completion)
2. Classify whether the question requires knowledge base retrieval
2. Classify the intent of the question
3. Analyze attached images (when present)
## Task 1: Rewriting Goals
@@ -33,15 +33,18 @@ templates:
- IMPORTANT: The rewritten question must be in {{language}}
## Task 2: Intent Classification
Determine if the question requires knowledge base retrieval.
- Output a boolean field `skip_kb_search` instead of any prefix marker.
- Set `skip_kb_search=true` only when you are very confident retrieval is unnecessary.
Classify the user's intent into exactly ONE of the following categories.
Output the category name as the `intent` field in the JSON.
When to set skip_kb_search=true:
- Pure greetings, thanks, or farewell with no question ("谢谢", "你好", "再见")
- Requests to summarize or manipulate the previous conversation itself ("总结一下我们的对话")
- Pure image understanding with NO intent to search documents: describing, summarizing, translating, or extracting content from the image itself ("这张图片是什么", "描述一下图片内容", "帮我翻译图中文字", "图里的表格数据是什么", "帮我识别一下这张图")
- Follow-up questions that clearly refer to previous conversation content (especially previously uploaded images) and can be answered from dialogue context directly ("第一张图再详细描述一下", "第二张门上的字是什么意思", "这个再展开讲讲")
Intent types:
- `kb_search`: The question requires retrieving information from the knowledge base. This is the DEFAULT — use it when unsure.
- `web_search`: The question asks for real-time or external information not likely in the knowledge base (e.g. "今天天气怎么样", "最新的新闻").
- `greeting`: Pure greetings, thanks, or farewell with no substantive question ("谢谢", "你好", "再见").
- `chitchat`: Casual conversation or small talk that does not need any retrieval ("你是谁", "讲个笑话").
- `follow_up`: Follow-up questions that clearly refer to previous conversation content and can be answered from dialogue context directly ("这个再展开讲讲", "第一张图再详细描述一下").
- `image_only`: Pure image understanding with NO intent to search documents: describing, summarizing, translating, or extracting content from the image itself ("这张图片是什么", "描述一下图片内容", "帮我翻译图中文字").
- `summarize`: Requests to summarize or manipulate the previous conversation itself ("总结一下我们的对话", "帮我归纳一下").
- `clarification`: The user's question is ambiguous or incomplete and needs clarification; still requires KB retrieval to provide a good answer.
## Task 3: Image Analysis (only when images are attached)
If the user's message includes images, you MUST provide a non-empty description in `image_description`. It must NOT be empty when images are present.
@@ -53,7 +56,7 @@ templates:
You MUST output ONLY a single JSON object.
Do NOT output markdown, code fences, explanations, or any extra text.
JSON schema:
{"rewrite_query":"string","skip_kb_search":true|false,"image_description":"string"}
{"rewrite_query":"string","intent":"string","image_description":"string"}
## Conversation History
{{conversation}}
+6 -6
View File
@@ -16,7 +16,7 @@ templates:
default: true
has_knowledge_base: true
content: |
You are a professional intelligent information retrieval assistant named WeKnora. Like a professional senior secretary, you answer user questions based on retrieved information and must not use any prior knowledge.
You are WeKnora, a professional intelligent information retrieval assistant developed by Tencent. Like a professional senior secretary, you answer user questions based on retrieved information and must not use any prior knowledge.
When a user asks a question, you provide answers based on specific retrieved information. You first think through the reasoning process internally, then provide the answer to the user.
## Response Rules
@@ -49,7 +49,7 @@ templates:
description: "기술 또는 전문 분야에 적합한 전문적이고 심층적인 답변 스타일"
has_knowledge_base: true
content: |
You are a senior domain expert assistant with extensive professional knowledge and practical experience.
You are WeKnora, a senior domain expert assistant developed by Tencent, with extensive professional knowledge and practical experience.
Core Responsibilities:
1. Deeply analyze user questions and provide professional, comprehensive answers
@@ -82,7 +82,7 @@ templates:
description: "고객 서비스 시나리오에 적합한 친절하고 열정적인 서비스 스타일"
has_knowledge_base: true
content: |
You are a professional and friendly customer service assistant, dedicated to providing quality service experiences for users.
You are WeKnora, a professional and friendly customer service assistant developed by Tencent, dedicated to providing quality service experiences for users.
Service Guidelines:
1. Be warm and friendly, with polite and appropriate language
@@ -115,7 +115,7 @@ templates:
description: "코드 예제를 포함한 기술적인 질문에 대한 전문적인 답변"
has_knowledge_base: true
content: |
You are a professional technical support engineer responsible for answering technical questions.
You are WeKnora, a professional technical support engineer developed by Tencent, responsible for answering technical questions.
Responsibilities:
1. Accurately diagnose technical issues encountered by users
@@ -149,7 +149,7 @@ templates:
description: "지식베이스에 의존하지 않는 보편적인 대화 도우미"
has_knowledge_base: false
content: |
You are an intelligent conversational assistant capable of natural and fluent dialogue with users.
You are WeKnora, an intelligent conversational assistant developed by Tencent, capable of natural and fluent dialogue with users.
Features:
1. Understand user intent and provide helpful answers
@@ -178,7 +178,7 @@ templates:
has_knowledge_base: true
has_web_search: true
content: |
You are an intelligent assistant with web search capabilities, able to obtain the latest information to answer questions.
You are WeKnora, an intelligent assistant developed by Tencent with web search capabilities, able to obtain the latest information to answer questions.
How You Work:
1. Combine web search results and knowledge base content to answer questions
@@ -845,18 +845,36 @@ const buildFullEventList = (stream: any[]) => {
if (isThinkingLikeEvent(event) && result.length > 0) {
const prev = result[result.length - 1];
if (isThinkingLikeEvent(prev)) {
// Merge into previous: combine content
const prevContent = prev._mergedContent || getThinkingContent(prev);
const curContent = getThinkingContent(event);
// Deduplicate: when a tool_call thinking event's thought content was
// already delivered via streaming thinking events (same text), skip it.
if (curContent && prevContent && prevContent.includes(curContent)) {
continue;
}
if (curContent && prevContent && curContent.includes(prevContent)) {
// Current fully contains previous — replace instead of appending
result[result.length - 1] = {
type: 'thinking',
event_id: prev.event_id,
content: curContent,
thinking: prev.thinking || event.thinking,
timestamp: prev.timestamp,
_mergedContent: curContent,
};
continue;
}
// Normal merge: combine non-overlapping content
const merged = [prevContent, curContent].filter(Boolean).join('\n\n');
// Replace previous with a merged thinking event
result[result.length - 1] = {
type: 'thinking',
event_id: prev.event_id,
content: merged,
thinking: prev.thinking || event.thinking,
timestamp: prev.timestamp,
_mergedContent: merged, // track for further merges
_mergedContent: merged,
};
continue;
}
@@ -317,7 +317,7 @@ onBeforeUnmount(() => {
.content-wrapper {
background: var(--td-bg-color-container);
border-radius: 6px;
padding: 8px 12px;
padding: 8px 0px;
transition: all 0.2s ease;
}
-2
View File
@@ -210,8 +210,6 @@ func (e *AgentEngine) streamThinkingToEventBus(
}
if chunk.Content != "" {
// logger.Debugf(ctx, "[Agent][Thinking][Iteration-%d] Emitting thought chunk: %d chars",
// iteration+1, len(chunk.Content))
e.eventBus.Emit(ctx, event.Event{
ID: thinkingID, // Same ID for all chunks in this stream
Type: event.EventAgentThought,
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -6,7 +6,6 @@ import (
"fmt"
"github.com/Tencent/WeKnora/internal/event"
"github.com/Tencent/WeKnora/internal/logger"
"github.com/Tencent/WeKnora/internal/types"
"github.com/Tencent/WeKnora/internal/types/interfaces"
"github.com/google/uuid"
@@ -60,7 +59,9 @@ func (p *PluginChatCompletionStream) OnEvent(ctx context.Context,
"message_count": len(chatMessages),
"system_prompt": chatMessages[0].Content,
})
logger.Infof(ctx, "user message: %s", chatMessages[len(chatMessages)-1].Content)
pipelineInfo(ctx, "Stream", "user_message", map[string]interface{}{
"content": chatMessages[len(chatMessages)-1].Content,
})
// EventBus is required for event-driven streaming
if chatManage.EventBus == nil {
pipelineError(ctx, "Stream", "eventbus_missing", map[string]interface{}{
@@ -98,117 +99,123 @@ func (p *PluginChatCompletionStream) OnEvent(ctx context.Context,
"session_id": chatManage.SessionID,
})
// Start goroutine to consume channel and emit events directly
// For non-agent mode, thinking content is embedded in answer stream with <think> tags
// This ensures consistent display between streaming and history loading
// Start goroutine to consume channel and emit events directly.
// For non-agent mode, thinking content is embedded in answer stream with <think> tags.
// The goroutine monitors ctx.Done() to avoid leaking when the context is cancelled
// and the upstream channel is not closed promptly.
go func() {
answerID := fmt.Sprintf("%s-answer", uuid.New().String()[:8])
var finalContent string
var thinkingStarted bool
var thinkingEnded bool
for response := range responseChan {
// Handle error responses from the stream
if response.ResponseType == types.ResponseTypeError {
logger.Errorf(ctx, "Stream error received: %s", response.Content)
if err := eventBus.Emit(ctx, types.Event{
ID: fmt.Sprintf("%s-error", uuid.New().String()[:8]),
Type: types.EventType(event.EventError),
SessionID: chatManage.SessionID,
Data: event.ErrorData{
Error: response.Content,
Stage: "chat_completion_stream",
SessionID: chatManage.SessionID,
},
}); err != nil {
logger.Errorf(ctx, "Failed to emit error event: %v", err)
}
continue
}
// For non-agent mode: embed thinking content with <think> tags in answer stream
// This ensures the frontend uses deepThink.vue component consistently
if response.ResponseType == types.ResponseTypeThinking {
content := response.Content
// Add <think> tag at the beginning of thinking content
if !thinkingStarted {
content = "<think>" + content
thinkingStarted = true
}
// Add </think> tag at the end of thinking content
if response.Done && !thinkingEnded {
content = content + "</think>"
thinkingEnded = true
}
finalContent += content
if err := eventBus.Emit(ctx, types.Event{
ID: answerID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: content,
Done: false, // Thinking is not the final answer
},
}); err != nil {
logger.Errorf(ctx, "Failed to emit thinking as answer event: %v", err)
}
continue
}
// Emit event for each answer chunk
if response.ResponseType == types.ResponseTypeAnswer {
// If we had thinking but it wasn't explicitly ended, close the think tag
for {
select {
case <-ctx.Done():
if thinkingStarted && !thinkingEnded {
thinkingEnded = true
finalContent += "</think>"
if err := eventBus.Emit(ctx, types.Event{
eventBus.Emit(ctx, types.Event{
ID: answerID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: "</think>",
Done: true,
},
})
}
pipelineInfo(ctx, "Stream", "context_cancelled", map[string]interface{}{
"session_id": chatManage.SessionID,
})
return
case response, ok := <-responseChan:
if !ok {
if thinkingStarted && !thinkingEnded {
finalContent += "</think>"
eventBus.Emit(ctx, types.Event{
ID: answerID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: "</think>",
Done: true,
},
})
}
pipelineInfo(ctx, "Stream", "channel_close", map[string]interface{}{
"session_id": chatManage.SessionID,
})
return
}
if response.ResponseType == types.ResponseTypeError {
pipelineError(ctx, "Stream", "stream_error", map[string]interface{}{
"session_id": chatManage.SessionID,
"error": response.Content,
})
eventBus.Emit(ctx, types.Event{
ID: fmt.Sprintf("%s-error", uuid.New().String()[:8]),
Type: types.EventType(event.EventError),
SessionID: chatManage.SessionID,
Data: event.ErrorData{
Error: response.Content,
Stage: "chat_completion_stream",
SessionID: chatManage.SessionID,
},
})
continue
}
if response.ResponseType == types.ResponseTypeThinking {
content := response.Content
if !thinkingStarted {
content = "<think>" + content
thinkingStarted = true
}
if response.Done && !thinkingEnded {
content = content + "</think>"
thinkingEnded = true
}
finalContent += content
eventBus.Emit(ctx, types.Event{
ID: answerID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: content,
Done: false,
},
}); err != nil {
logger.Errorf(ctx, "Failed to emit think close tag: %v", err)
})
continue
}
if response.ResponseType == types.ResponseTypeAnswer {
if thinkingStarted && !thinkingEnded {
thinkingEnded = true
finalContent += "</think>"
eventBus.Emit(ctx, types.Event{
ID: answerID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: "</think>",
Done: false,
},
})
}
}
finalContent += response.Content
if err := eventBus.Emit(ctx, types.Event{
ID: answerID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: response.Content,
Done: response.Done,
},
}); err != nil {
logger.Errorf(ctx, "Failed to emit answer event: %v", err)
finalContent += response.Content
eventBus.Emit(ctx, types.Event{
ID: answerID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: response.Content,
Done: response.Done,
},
})
}
}
}
// Compensate for unclosed <think> tag when stream ends unexpectedly
// (e.g., context cancelled, upstream error) without an answer arriving.
if thinkingStarted && !thinkingEnded {
thinkingEnded = true
finalContent += "</think>"
if err := eventBus.Emit(ctx, types.Event{
ID: answerID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: "</think>",
Done: true,
},
}); err != nil {
logger.Errorf(ctx, "Failed to emit think close tag on stream end: %v", err)
}
}
pipelineInfo(ctx, "Stream", "channel_close", map[string]interface{}{
"session_id": chatManage.SessionID,
})
}()
return next()
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -1,8 +1,12 @@
package chatpipline
package chatpipeline
import (
"context"
"regexp"
"slices"
"sort"
"strings"
"sync"
"github.com/Tencent/WeKnora/internal/common"
"github.com/Tencent/WeKnora/internal/logger"
@@ -11,6 +15,8 @@ import (
"github.com/Tencent/WeKnora/internal/types/interfaces"
)
var regThinkTags = regexp.MustCompile(`(?s)<think>.*?</think>`)
// pipelineInfo logs pipeline info level entries.
func pipelineInfo(ctx context.Context, stage, action string, fields map[string]interface{}) {
common.PipelineInfo(ctx, stage, action, fields)
@@ -51,11 +57,16 @@ func prepareChatModel(ctx context.Context, modelService interfaces.ModelService,
return chatModel, opt, nil
}
// prepareMessagesWithHistory prepare complete messages including history
// prepareMessagesWithHistory prepare complete messages including history.
// When SystemPromptOverride is set (e.g. by intent-specific prompt logic),
// it takes precedence over the default SummaryConfig.Prompt.
func prepareMessagesWithHistory(chatManage *types.ChatManage) []chat.Message {
// Replace placeholders in system prompt
systemPrompt := renderSystemPromptPlaceholders(chatManage.SummaryConfig.Prompt, chatManage.Language)
base := chatManage.SummaryConfig.Prompt
if chatManage.SystemPromptOverride != "" {
base = chatManage.SystemPromptOverride
}
systemPrompt := renderSystemPromptPlaceholders(base, chatManage.Language)
chatMessages := []chat.Message{
{Role: "system", Content: systemPrompt},
}
@@ -77,6 +88,59 @@ func prepareMessagesWithHistory(chatManage *types.ChatManage) []chat.Message {
return chatMessages
}
// loadAndProcessHistory fetches recent messages, groups them into Q&A pairs,
// strips <think> tags from assistant answers, sorts by recency, and limits to maxRounds.
// fetchCount controls how many raw messages to fetch (typically maxRounds*2+10).
func loadAndProcessHistory(
ctx context.Context,
messageService interfaces.MessageService,
sessionID string,
maxRounds int,
fetchCount int,
) ([]*types.History, error) {
history, err := messageService.GetRecentMessagesBySession(ctx, sessionID, fetchCount)
if err != nil {
return nil, err
}
historyMap := make(map[string]*types.History)
for _, message := range history {
h, ok := historyMap[message.RequestID]
if !ok {
h = &types.History{}
}
if message.Role == "user" {
h.Query = message.Content
h.CreateAt = message.CreatedAt
if desc := extractImageCaptions(message.Images); desc != "" {
h.Query += "\n\n[用户上传图片内容]\n" + desc
}
} else {
h.Answer = regThinkTags.ReplaceAllString(message.Content, "")
h.KnowledgeReferences = message.KnowledgeReferences
}
historyMap[message.RequestID] = h
}
historyList := make([]*types.History, 0, len(historyMap))
for _, h := range historyMap {
if h.Answer != "" && h.Query != "" {
historyList = append(historyList, h)
}
}
sort.Slice(historyList, func(i, j int) bool {
return historyList[i].CreateAt.After(historyList[j].CreateAt)
})
if len(historyList) > maxRounds {
historyList = historyList[:maxRounds]
}
slices.Reverse(historyList)
return historyList, nil
}
// extractImageCaptions concatenates non-empty Caption fields from stored
// message images. Used when loading history so that previous turns' image
// descriptions are visible to the model.
@@ -101,3 +165,64 @@ func renderSystemPromptPlaceholders(prompt string, language ...string) string {
}
return types.RenderPromptPlaceholders(prompt, vals)
}
// ---------------------------------------------------------------------------
// Concurrency utilities
// ---------------------------------------------------------------------------
// ParallelTask represents a named unit of concurrent work.
type ParallelTask struct {
Name string
Run func() *PluginError
}
// RunParallel executes tasks concurrently.
// Returns a map of task name → error for tasks that returned non-nil errors.
func RunParallel(tasks ...ParallelTask) map[string]*PluginError {
errs := make(map[string]*PluginError)
var mu sync.Mutex
var wg sync.WaitGroup
wg.Add(len(tasks))
for _, task := range tasks {
go func(t ParallelTask) {
defer wg.Done()
if err := t.Run(); err != nil {
mu.Lock()
errs[t.Name] = err
mu.Unlock()
}
}(task)
}
wg.Wait()
return errs
}
// ParallelMap applies fn to each element of items concurrently (up to
// maxWorkers goroutines) and returns results in the same order as items.
// If maxWorkers <= 0, concurrency is unbounded (one goroutine per item).
func ParallelMap[T, R any](items []T, maxWorkers int, fn func(int, T) R) []R {
n := len(items)
if n == 0 {
return nil
}
results := make([]R, n)
if maxWorkers <= 0 || maxWorkers > n {
maxWorkers = n
}
var wg sync.WaitGroup
sem := make(chan struct{}, maxWorkers)
for i, item := range items {
wg.Add(1)
sem <- struct{}{}
go func(idx int, it T) {
defer func() { <-sem; wg.Done() }()
results[idx] = fn(idx, it)
}(i, item)
}
wg.Wait()
return results
}
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -53,6 +53,9 @@ func (p *PluginDataAnalysis) OnEvent(
chatManage *types.ChatManage,
next func() *PluginError,
) *PluginError {
if !chatManage.NeedsRetrieval() {
return next()
}
// 1. Check if there are any CSV/Excel files in MergeResult
var dataFiles []*types.SearchResult
for _, result := range chatManage.MergeResult {
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -47,14 +47,13 @@ func NewPluginExtractEntity(
return res
}
// ActivationEvents returns the list of event types this plugin responds to
// This plugin only responds to REWRITE_QUERY events
// ActivationEvents returns the list of event types this plugin responds to.
func (p *PluginExtractEntity) ActivationEvents() []types.EventType {
return []types.EventType{types.REWRITE_QUERY}
return []types.EventType{types.QUERY_UNDERSTAND}
}
// OnEvent processes triggered events
// When receiving a REWRITE_QUERY event, it rewrites the user query using conversation history and the language model
// When receiving a QUERY_UNDERSTAND event, it extracts entities from the query
func (p *PluginExtractEntity) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -26,6 +26,9 @@ func (p *PluginFilterTopK) ActivationEvents() []types.EventType {
func (p *PluginFilterTopK) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
if !chatManage.NeedsRetrieval() {
return next()
}
pipelineInfo(ctx, "FilterTopK", "input", map[string]interface{}{
"session_id": chatManage.SessionID,
"top_k": chatManage.RerankTopK,
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -74,7 +74,7 @@ func (p *PluginIntoChatMessage) OnEvent(ctx context.Context,
}
// Intent-based no-search path: bypass "reference materials" template entirely.
if chatManage.SkipKBSearch {
if !chatManage.NeedsRetrieval() {
// Prefer rewritten query in no-search mode; fallback to original query.
userContent := safeQuery
if rewrite := strings.TrimSpace(chatManage.RewriteQuery); rewrite != "" {
@@ -154,7 +154,7 @@ func (p *PluginIntoChatMessage) OnEvent(ctx context.Context,
"session_id": chatManage.SessionID,
"user_content_len": len(chatManage.UserContent),
"faq_priority": chatManage.FAQPriorityEnabled,
"skip_kb_search": chatManage.SkipKBSearch,
"intent": chatManage.Intent,
"image_description": chatManage.ImageDescription,
"chat_model_supports_vision": chatManage.ChatModelSupportsVision,
})
@@ -0,0 +1,63 @@
package chatpipeline
import (
"context"
"github.com/Tencent/WeKnora/internal/config"
"github.com/Tencent/WeKnora/internal/types"
"github.com/Tencent/WeKnora/internal/types/interfaces"
)
type PluginLoadHistory struct {
messageService interfaces.MessageService
config *config.Config
}
func NewPluginLoadHistory(eventManager *EventManager,
messageService interfaces.MessageService,
config *config.Config,
) *PluginLoadHistory {
res := &PluginLoadHistory{
messageService: messageService,
config: config,
}
eventManager.Register(res)
return res
}
func (p *PluginLoadHistory) ActivationEvents() []types.EventType {
return []types.EventType{types.LOAD_HISTORY}
}
func (p *PluginLoadHistory) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
maxRounds := p.config.Conversation.MaxRounds
if chatManage.MaxRounds > 0 {
maxRounds = chatManage.MaxRounds
}
pipelineInfo(ctx, "LoadHistory", "input", map[string]interface{}{
"session_id": chatManage.SessionID,
"max_rounds": maxRounds,
})
historyList, err := loadAndProcessHistory(ctx, p.messageService, chatManage.SessionID, maxRounds, maxRounds*2+10)
if err != nil {
pipelineWarn(ctx, "LoadHistory", "history_fetch", map[string]interface{}{
"session_id": chatManage.SessionID,
"error": err.Error(),
})
return next()
}
chatManage.History = historyList
pipelineInfo(ctx, "LoadHistory", "output", map[string]interface{}{
"session_id": chatManage.SessionID,
"history_rounds": len(historyList),
"max_rounds": maxRounds,
})
return next()
}
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -76,7 +76,7 @@ func (p *MemoryPlugin) handleRetrieval(
memoryStr += fmt.Sprintf("- %s (Summary: %s)\n", ep.CreatedAt.Format("2006-01-02"), ep.Summary)
}
chatManage.UserContent += memoryStr
logger.Info(ctx, "Retrieved memory: %s", memoryStr)
logger.Infof(ctx, "Retrieved memory: %s", memoryStr)
}
logger.Info(ctx, "End to retrieve memory")
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -43,6 +43,9 @@ func (p *PluginMerge) ActivationEvents() []types.EventType {
func (p *PluginMerge) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
if !chatManage.NeedsRetrieval() {
return next()
}
pipelineInfo(ctx, "Merge", "input", map[string]interface{}{
"session_id": chatManage.SessionID,
"candidate_cnt": len(chatManage.RerankResult),
@@ -154,33 +157,45 @@ func (p *PluginMerge) groupAndMergeOverlapping(ctx context.Context, results []*t
"knowledge_cnt": len(knowledgeGroup),
})
var mergedChunks []*types.SearchResult
// Flatten into independent (knowledgeID, chunks) work units for parallel merge.
type mergeUnit struct {
knowledgeID string
chunks []*types.SearchResult
}
var units []mergeUnit
for knowledgeID, chunkGroup := range knowledgeGroup {
for _, chunks := range chunkGroup {
pipelineInfo(ctx, "Merge", "group_process", map[string]interface{}{
"knowledge_id": knowledgeID,
"chunk_cnt": len(chunks),
})
// Sort by start position, then by end position
sort.Slice(chunks, func(i, j int) bool {
if chunks[i].StartAt == chunks[j].StartAt {
return chunks[i].EndAt < chunks[j].EndAt
}
return chunks[i].StartAt < chunks[j].StartAt
})
grouped := p.mergeOverlappingChunks(ctx, knowledgeID, chunks)
pipelineInfo(ctx, "Merge", "group_output", map[string]interface{}{
"knowledge_id": knowledgeID,
"merged_chunks": len(grouped),
})
mergedChunks = append(mergedChunks, grouped...)
units = append(units, mergeUnit{knowledgeID: knowledgeID, chunks: chunks})
}
}
groupResults := ParallelMap(units, 0, func(_ int, u mergeUnit) []*types.SearchResult {
pipelineInfo(ctx, "Merge", "group_process", map[string]interface{}{
"knowledge_id": u.knowledgeID,
"chunk_cnt": len(u.chunks),
})
sort.Slice(u.chunks, func(i, j int) bool {
if u.chunks[i].StartAt == u.chunks[j].StartAt {
return u.chunks[i].EndAt < u.chunks[j].EndAt
}
return u.chunks[i].StartAt < u.chunks[j].StartAt
})
grouped := p.mergeOverlappingChunks(ctx, u.knowledgeID, u.chunks)
pipelineInfo(ctx, "Merge", "group_output", map[string]interface{}{
"knowledge_id": u.knowledgeID,
"merged_chunks": len(grouped),
})
return grouped
})
var mergedChunks []*types.SearchResult
for _, g := range groupResults {
mergedChunks = append(mergedChunks, g...)
}
pipelineInfo(ctx, "Merge", "output", map[string]interface{}{
"merged_total": len(mergedChunks),
})
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -0,0 +1,147 @@
package chatpipeline
import (
"context"
"testing"
"github.com/Tencent/WeKnora/internal/types"
)
// --- IntoChatMessage tests ---
func TestIntoChatMessage_NoKBRetrieval(t *testing.T) {
cm := &types.ChatManage{
PipelineRequest: types.PipelineRequest{
Query: "hello world",
},
PipelineState: types.PipelineState{
Intent: types.IntentChitchat,
},
}
plugin := &PluginIntoChatMessage{}
nextCalled := false
err := plugin.OnEvent(context.Background(), types.INTO_CHAT_MESSAGE, cm, func() *PluginError {
nextCalled = true
return nil
})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !nextCalled {
t.Fatal("next() was not called")
}
if cm.UserContent != "hello world" {
t.Errorf("UserContent: got %q, want %q", cm.UserContent, "hello world")
}
}
func TestIntoChatMessage_WithMergeResults(t *testing.T) {
cm := &types.ChatManage{
PipelineRequest: types.PipelineRequest{
Query: "test query",
SummaryConfig: types.SummaryConfig{
ContextTemplate: "Question: {{query}}\n\nReferences:\n{{contexts}}",
},
},
PipelineState: types.PipelineState{
MergeResult: []*types.SearchResult{
{Content: "chunk A content"},
{Content: "chunk B content"},
},
},
}
plugin := &PluginIntoChatMessage{}
nextCalled := false
err := plugin.OnEvent(context.Background(), types.INTO_CHAT_MESSAGE, cm, func() *PluginError {
nextCalled = true
return nil
})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if !nextCalled {
t.Fatal("next() was not called")
}
if cm.UserContent == "" {
t.Fatal("expected UserContent to be populated")
}
if !contains(cm.UserContent, "test query") {
t.Errorf("UserContent should contain query, got: %s", cm.UserContent)
}
if !contains(cm.UserContent, "chunk A content") {
t.Errorf("UserContent should contain chunk A, got: %s", cm.UserContent)
}
}
func TestIntoChatMessage_ImageDescriptionAppended(t *testing.T) {
cm := &types.ChatManage{
PipelineRequest: types.PipelineRequest{
Query: "what is this?",
ChatModelSupportsVision: false,
},
PipelineState: types.PipelineState{
Intent: types.IntentChitchat,
ImageDescription: "a cat sitting on a mat",
},
}
plugin := &PluginIntoChatMessage{}
_ = plugin.OnEvent(context.Background(), types.INTO_CHAT_MESSAGE, cm, func() *PluginError {
return nil
})
if !contains(cm.UserContent, "a cat sitting on a mat") {
t.Errorf("UserContent should contain image description, got: %s", cm.UserContent)
}
}
// --- PipelineBuilder tests ---
func TestPipelineBuilder_Basic(t *testing.T) {
pipeline := types.NewPipelineBuilder().
Add(types.LOAD_HISTORY).
Add(types.CHAT_COMPLETION_STREAM).
Build()
if len(pipeline) != 2 {
t.Fatalf("expected 2 stages, got %d", len(pipeline))
}
if pipeline[0] != types.LOAD_HISTORY {
t.Errorf("stage 0: got %v, want %v", pipeline[0], types.LOAD_HISTORY)
}
}
func TestPipelineBuilder_AddIf(t *testing.T) {
pipeline := types.NewPipelineBuilder().
Add(types.LOAD_HISTORY).
AddIf(false, types.QUERY_UNDERSTAND).
AddIf(true, types.CHAT_COMPLETION_STREAM).
Build()
if len(pipeline) != 2 {
t.Fatalf("expected 2 stages (QUERY_UNDERSTAND skipped), got %d", len(pipeline))
}
if pipeline[1] != types.CHAT_COMPLETION_STREAM {
t.Errorf("stage 1: got %v, want %v", pipeline[1], types.CHAT_COMPLETION_STREAM)
}
}
func TestPipelineBuilder_Empty(t *testing.T) {
pipeline := types.NewPipelineBuilder().Build()
if len(pipeline) != 0 {
t.Fatalf("expected 0 stages, got %d", len(pipeline))
}
}
// --- helpers ---
func contains(s, substr string) bool {
return len(s) >= len(substr) && (s == substr || len(s) > 0 && containsString(s, substr))
}
func containsString(s, sub string) bool {
for i := 0; i <= len(s)-len(sub); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
}
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -1,13 +1,9 @@
// Package chatpipline provides chat pipeline processing capabilities
// Including query rewriting, history processing, model invocation and other features
package chatpipline
package chatpipeline
import (
"context"
"encoding/json"
"regexp"
"slices"
"sort"
"strings"
"time"
@@ -19,35 +15,30 @@ import (
"github.com/google/uuid"
)
// PluginRewrite is a plugin for rewriting user queries
// It uses historical dialog context and large language models to optimize the user's original query
type PluginRewrite struct {
modelService interfaces.ModelService // Model service for calling large language models
messageService interfaces.MessageService // Message service for retrieving historical messages
config *config.Config // System configuration
// PluginQueryUnderstand performs query rewriting and intent classification.
// It uses conversation history and an LLM to optimise the user's original query
// and determine the downstream pipeline behaviour.
type PluginQueryUnderstand struct {
modelService interfaces.ModelService
messageService interfaces.MessageService
config *config.Config
}
// reg is a regular expression used to match and remove content between <think></think> tags
var reg = regexp.MustCompile(`(?s)<think>.*?</think>`)
var rewriteImageSepPattern = regexp.MustCompile(`(?s)^(.*?)\s*\n?---\n(.*)$`)
const (
noSearchPrefix = "[NO_SEARCH]"
)
type rewriteOutput struct {
RewriteQuery string
SkipKBSearch bool
ImageDescription string
type queryUnderstandOutput struct {
RewriteQuery string `json:"rewrite_query"`
Intent types.QueryIntent `json:"intent"`
ImageDescription string `json:"image_description"`
}
// NewPluginRewrite creates a new query rewriting plugin instance
// Also registers the plugin with the event manager
func NewPluginRewrite(eventManager *EventManager,
// NewPluginQueryUnderstand creates a new query-understanding plugin instance
// and registers it with the event manager.
func NewPluginQueryUnderstand(eventManager *EventManager,
modelService interfaces.ModelService, messageService interfaces.MessageService,
config *config.Config,
) *PluginRewrite {
res := &PluginRewrite{
) *PluginQueryUnderstand {
res := &PluginQueryUnderstand{
modelService: modelService,
messageService: messageService,
config: config,
@@ -56,10 +47,9 @@ func NewPluginRewrite(eventManager *EventManager,
return res
}
// ActivationEvents returns the list of event types this plugin responds to
// This plugin only responds to REWRITE_QUERY events
func (p *PluginRewrite) ActivationEvents() []types.EventType {
return []types.EventType{types.REWRITE_QUERY}
// ActivationEvents returns the list of event types this plugin responds to.
func (p *PluginQueryUnderstand) ActivationEvents() []types.EventType {
return []types.EventType{types.QUERY_UNDERSTAND}
}
// OnEvent processes triggered events.
@@ -67,24 +57,22 @@ func (p *PluginRewrite) ActivationEvents() []types.EventType {
// - Text only: standard rewrite + intent classification (uses chat model)
// - Text + images: multimodal rewrite + intent + image description (uses VLM/vision model)
// - Images only: multimodal analysis + intent + image description (uses VLM/vision model)
func (p *PluginRewrite) OnEvent(ctx context.Context,
func (p *PluginQueryUnderstand) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
chatManage.RewriteQuery = chatManage.Query
hasImages := len(chatManage.Images) > 0
needRewrite := chatManage.EnableRewrite
// When images are present we always run the step for image analysis + intent,
// even without history or rewrite enabled.
if !needRewrite && !hasImages {
pipelineInfo(ctx, "Rewrite", "skip", map[string]interface{}{
pipelineInfo(ctx, "QueryUnderstand", "skip", map[string]interface{}{
"session_id": chatManage.SessionID,
"reason": "rewrite_disabled_no_images",
})
return next()
}
pipelineInfo(ctx, "Rewrite", "input", map[string]interface{}{
pipelineInfo(ctx, "QueryUnderstand", "input", map[string]interface{}{
"session_id": chatManage.SessionID,
"tenant_id": chatManage.TenantID,
"user_query": chatManage.Query,
@@ -93,13 +81,10 @@ func (p *PluginRewrite) OnEvent(ctx context.Context,
})
// --- Load and prepare conversation history ---
// If history was already loaded by a prior LOAD_HISTORY step, reuse it
// instead of fetching again. This allows rag_stream to work correctly
// even when rewrite is disabled.
var historyList []*types.History
if len(chatManage.History) > 0 {
historyList = chatManage.History
pipelineInfo(ctx, "Rewrite", "history_reused", map[string]interface{}{
pipelineInfo(ctx, "QueryUnderstand", "history_reused", map[string]interface{}{
"session_id": chatManage.SessionID,
"rounds": len(historyList),
})
@@ -107,19 +92,10 @@ func (p *PluginRewrite) OnEvent(ctx context.Context,
historyList = p.loadHistory(ctx, chatManage)
}
// Skip if there's nothing to do: no history to rewrite AND no images to analyse
if len(historyList) == 0 && !hasImages {
pipelineInfo(ctx, "Rewrite", "skip", map[string]interface{}{
"session_id": chatManage.SessionID,
"reason": "empty_history_no_images",
})
return next()
}
// --- Select the appropriate model ---
rewriteModel, useImages := p.selectModel(ctx, chatManage, hasImages)
if rewriteModel == nil {
pipelineError(ctx, "Rewrite", "get_model", map[string]interface{}{
pipelineError(ctx, "QueryUnderstand", "get_model", map[string]interface{}{
"session_id": chatManage.SessionID,
})
return next()
@@ -128,7 +104,6 @@ func (p *PluginRewrite) OnEvent(ctx context.Context,
// --- Build prompts ---
systemContent, userContent := p.buildPrompts(chatManage, historyList)
// Build user message (with images when using a vision-capable model)
userMsg := chat.Message{Role: "user", Content: userContent}
if useImages {
userMsg.Images = chatManage.Images
@@ -178,7 +153,7 @@ func (p *PluginRewrite) OnEvent(ctx context.Context,
},
})
}
pipelineError(ctx, "Rewrite", "model_call", map[string]interface{}{
pipelineError(ctx, "QueryUnderstand", "model_call", map[string]interface{}{
"session_id": chatManage.SessionID,
"error": err.Error(),
})
@@ -201,30 +176,42 @@ func (p *PluginRewrite) OnEvent(ctx context.Context,
}
// --- Parse structured output ---
p.parseRewriteOutput(chatManage, response.Content)
p.parseOutput(chatManage, response.Content)
// Persist image description back to the user message so that future turns
// can see it when loading conversation history.
// Persist image description asynchronously — this DB write does not affect
// the current pipeline result, so it can run in the background.
if chatManage.ImageDescription != "" && chatManage.UserMessageID != "" {
p.updateUserMessageImageCaption(ctx, chatManage)
go p.updateUserMessageImageCaption(context.WithoutCancel(ctx), chatManage)
}
pipelineInfo(ctx, "Rewrite", "output", map[string]interface{}{
"session_id": chatManage.SessionID,
"rewrite_query": chatManage.RewriteQuery,
"skip_kb_search": chatManage.SkipKBSearch,
"has_image_desc": chatManage.ImageDescription != "",
"original_output": response.Content,
// --- Apply intent-specific system prompt override ---
if !chatManage.NeedsRetrieval() {
if prompt, ok := p.config.Conversation.IntentSystemPrompts[string(chatManage.Intent)]; ok {
chatManage.SystemPromptOverride = prompt
pipelineInfo(ctx, "QueryUnderstand", "prompt_override", map[string]interface{}{
"session_id": chatManage.SessionID,
"intent": chatManage.Intent,
})
}
}
pipelineInfo(ctx, "QueryUnderstand", "output", map[string]interface{}{
"session_id": chatManage.SessionID,
"rewrite_query": chatManage.RewriteQuery,
"intent": chatManage.Intent,
"has_image_desc": chatManage.ImageDescription != "",
"has_prompt_override": chatManage.SystemPromptOverride != "",
"original_output": response.Content,
})
return next()
}
// updateUserMessageImageCaption writes the generated ImageDescription back to
// the stored user message's Images so that subsequent turns can see it in history.
func (p *PluginRewrite) updateUserMessageImageCaption(ctx context.Context, chatManage *types.ChatManage) {
// the stored user message so that subsequent turns can see it in history.
func (p *PluginQueryUnderstand) updateUserMessageImageCaption(ctx context.Context, chatManage *types.ChatManage) {
msg, err := p.messageService.GetMessage(ctx, chatManage.SessionID, chatManage.UserMessageID)
if err != nil {
pipelineWarn(ctx, "Rewrite", "get_user_message", map[string]interface{}{
pipelineWarn(ctx, "QueryUnderstand", "get_user_message", map[string]interface{}{
"session_id": chatManage.SessionID,
"user_message_id": chatManage.UserMessageID,
"error": err.Error(),
@@ -238,10 +225,8 @@ func (p *PluginRewrite) updateUserMessageImageCaption(ctx context.Context, chatM
msg.Images[0].Caption = chatManage.ImageDescription
// Use the targeted UpdateMessageImages to reliably persist the JSONB column.
// GORM's struct-based Updates may silently skip custom Valuer types.
if err := p.messageService.UpdateMessageImages(ctx, chatManage.SessionID, chatManage.UserMessageID, msg.Images); err != nil {
pipelineWarn(ctx, "Rewrite", "update_image_caption", map[string]interface{}{
pipelineWarn(ctx, "QueryUnderstand", "update_image_caption", map[string]interface{}{
"session_id": chatManage.SessionID,
"user_message_id": chatManage.UserMessageID,
"error": err.Error(),
@@ -250,58 +235,25 @@ func (p *PluginRewrite) updateUserMessageImageCaption(ctx context.Context, chatM
}
// loadHistory fetches and processes conversation history for rewrite context.
func (p *PluginRewrite) loadHistory(ctx context.Context, chatManage *types.ChatManage) []*types.History {
history, err := p.messageService.GetRecentMessagesBySession(ctx, chatManage.SessionID, 20)
if err != nil {
pipelineWarn(ctx, "Rewrite", "history_fetch", map[string]interface{}{
"session_id": chatManage.SessionID,
"error": err.Error(),
})
}
historyMap := make(map[string]*types.History)
for _, message := range history {
h, ok := historyMap[message.RequestID]
if !ok {
h = &types.History{}
}
if message.Role == "user" {
h.Query = message.Content
h.CreateAt = message.CreatedAt
if desc := extractImageCaptions(message.Images); desc != "" {
h.Query += "\n\n[用户上传图片内容]\n" + desc
}
} else {
h.Answer = reg.ReplaceAllString(message.Content, "")
h.KnowledgeReferences = message.KnowledgeReferences
}
historyMap[message.RequestID] = h
}
historyList := make([]*types.History, 0)
for _, h := range historyMap {
if h.Answer != "" && h.Query != "" {
historyList = append(historyList, h)
}
}
sort.Slice(historyList, func(i, j int) bool {
return historyList[i].CreateAt.After(historyList[j].CreateAt)
})
func (p *PluginQueryUnderstand) loadHistory(ctx context.Context, chatManage *types.ChatManage) []*types.History {
maxRounds := p.config.Conversation.MaxRounds
if chatManage.MaxRounds > 0 {
maxRounds = chatManage.MaxRounds
}
if len(historyList) > maxRounds {
historyList = historyList[:maxRounds]
historyList, err := loadAndProcessHistory(ctx, p.messageService, chatManage.SessionID, maxRounds, 20)
if err != nil {
pipelineWarn(ctx, "QueryUnderstand", "history_fetch", map[string]interface{}{
"session_id": chatManage.SessionID,
"error": err.Error(),
})
return nil
}
slices.Reverse(historyList)
chatManage.History = historyList
if len(historyList) > 0 {
pipelineInfo(ctx, "Rewrite", "history_ready", map[string]interface{}{
pipelineInfo(ctx, "QueryUnderstand", "history_ready", map[string]interface{}{
"session_id": chatManage.SessionID,
"history_rounds": len(historyList),
})
@@ -310,17 +262,16 @@ func (p *PluginRewrite) loadHistory(ctx context.Context, chatManage *types.ChatM
return historyList
}
// selectModel picks the model for rewrite. When images are present it prefers
// a vision-capable model (either the chat model itself, or the agent's VLM).
// Returns (model, useImages).
func (p *PluginRewrite) selectModel(ctx context.Context, chatManage *types.ChatManage, hasImages bool) (chat.Chat, bool) {
// selectModel picks the model for query understanding. When images are present
// it prefers a vision-capable model. Returns (model, useImages).
func (p *PluginQueryUnderstand) selectModel(ctx context.Context, chatManage *types.ChatManage, hasImages bool) (chat.Chat, bool) {
if hasImages {
if chatManage.ChatModelSupportsVision {
m, err := p.modelService.GetChatModel(ctx, chatManage.ChatModelID)
if err == nil {
return m, true
}
pipelineWarn(ctx, "Rewrite", "vision_model_fallback", map[string]interface{}{
pipelineWarn(ctx, "QueryUnderstand", "vision_model_fallback", map[string]interface{}{
"session_id": chatManage.SessionID,
"error": err.Error(),
})
@@ -330,21 +281,20 @@ func (p *PluginRewrite) selectModel(ctx context.Context, chatManage *types.ChatM
if err == nil {
return m, true
}
pipelineWarn(ctx, "Rewrite", "vlm_model_fallback", map[string]interface{}{
pipelineWarn(ctx, "QueryUnderstand", "vlm_model_fallback", map[string]interface{}{
"session_id": chatManage.SessionID,
"vlm_model_id": chatManage.VLMModelID,
"error": err.Error(),
})
}
pipelineWarn(ctx, "Rewrite", "no_vision_model", map[string]interface{}{
pipelineWarn(ctx, "QueryUnderstand", "no_vision_model", map[string]interface{}{
"session_id": chatManage.SessionID,
})
}
// Fallback: text-only rewrite with chat model
m, err := p.modelService.GetChatModel(ctx, chatManage.ChatModelID)
if err != nil {
pipelineError(ctx, "Rewrite", "get_model", map[string]interface{}{
pipelineError(ctx, "QueryUnderstand", "get_model", map[string]interface{}{
"session_id": chatManage.SessionID,
"chat_model_id": chatManage.ChatModelID,
"error": err.Error(),
@@ -355,7 +305,7 @@ func (p *PluginRewrite) selectModel(ctx context.Context, chatManage *types.ChatM
}
// buildPrompts constructs system and user prompts with placeholder replacement.
func (p *PluginRewrite) buildPrompts(chatManage *types.ChatManage, historyList []*types.History) (string, string) {
func (p *PluginQueryUnderstand) buildPrompts(chatManage *types.ChatManage, historyList []*types.History) (string, string) {
userPrompt := p.config.Conversation.RewritePromptUser
if chatManage.RewritePromptUser != "" {
userPrompt = chatManage.RewritePromptUser
@@ -377,54 +327,39 @@ func (p *PluginRewrite) buildPrompts(chatManage *types.ChatManage, historyList [
types.RenderPromptPlaceholders(userPrompt, vals)
}
// parseRewriteOutput extracts intent classification, rewritten query, and
// optional image description from the model's structured output.
// parseOutput extracts the rewritten query, intent classification, and optional
// image description from the model's structured JSON output.
//
// Expected formats:
//
// Preferred: {"rewrite_query":"...","skip_kb_search":false,"image_description":"..."}
// Legacy fallback:
// - Text only: "[NO_SEARCH] rewritten question" or "rewritten question"
// - With images: "[NO_SEARCH]\nrewritten question\n---\nimage description"
func (p *PluginRewrite) parseRewriteOutput(chatManage *types.ChatManage, raw string) {
// Expected format: {"rewrite_query":"...","intent":"kb_search","image_description":"..."}
func (p *PluginQueryUnderstand) parseOutput(chatManage *types.ChatManage, raw string) {
content := strings.TrimSpace(raw)
if content == "" {
return
}
if output, ok := parseStructuredRewriteOutput(content); ok {
if output, ok := parseStructuredQueryOutput(content); ok {
if rewrite := strings.TrimSpace(output.RewriteQuery); rewrite != "" {
chatManage.RewriteQuery = rewrite
}
chatManage.SkipKBSearch = output.SkipKBSearch
chatManage.Intent = output.Intent
chatManage.ImageDescription = strings.TrimSpace(output.ImageDescription)
return
}
// Legacy fallback parsing for older prompts/models.
if strings.HasPrefix(content, noSearchPrefix) {
chatManage.SkipKBSearch = true
content = strings.TrimSpace(strings.TrimPrefix(content, noSearchPrefix))
}
if m := rewriteImageSepPattern.FindStringSubmatch(content); len(m) == 3 {
chatManage.RewriteQuery = strings.TrimSpace(m[1])
chatManage.ImageDescription = strings.TrimSpace(m[2])
return
}
// If JSON parsing failed entirely, treat the raw text as the rewritten query
// and default to IntentKBSearch for safety.
if content != "" {
chatManage.RewriteQuery = content
}
}
func parseStructuredRewriteOutput(raw string) (rewriteOutput, bool) {
func parseStructuredQueryOutput(raw string) (queryUnderstandOutput, bool) {
content := strings.TrimSpace(raw)
if content == "" {
return rewriteOutput{}, false
return queryUnderstandOutput{}, false
}
var out rewriteOutput
if parsed, ok := parseStructuredRewriteOutputJSON(content); ok {
if parsed, ok := parseStructuredQueryOutputJSON(content); ok {
return parsed, true
}
@@ -432,31 +367,29 @@ func parseStructuredRewriteOutput(raw string) (rewriteOutput, bool) {
start := strings.Index(content, "{")
end := strings.LastIndex(content, "}")
if start < 0 || end <= start {
return rewriteOutput{}, false
return queryUnderstandOutput{}, false
}
candidate := content[start : end+1]
if parsed, ok := parseStructuredRewriteOutputJSON(candidate); ok {
if parsed, ok := parseStructuredQueryOutputJSON(candidate); ok {
return parsed, true
}
return out, false
return queryUnderstandOutput{}, false
}
func parseStructuredRewriteOutputJSON(content string) (rewriteOutput, bool) {
func parseStructuredQueryOutputJSON(content string) (queryUnderstandOutput, bool) {
var obj map[string]json.RawMessage
if err := json.Unmarshal([]byte(content), &obj); err != nil {
return rewriteOutput{}, false
return queryUnderstandOutput{}, false
}
out := rewriteOutput{
out := queryUnderstandOutput{
RewriteQuery: strings.TrimSpace(firstStringField(obj,
"rewrite_query", "rewritten_query", "query", "question")),
}
// Support common variants and semantic inversion for need_search.
if v, ok := firstBoolField(obj, "skip_kb_search", "skip_search", "no_search"); ok {
out.SkipKBSearch = v
} else if v, ok := firstBoolField(obj, "need_search", "requires_search"); ok {
out.SkipKBSearch = !v
intentStr := strings.TrimSpace(firstStringField(obj, "intent"))
if intentStr != "" {
out.Intent = types.QueryIntent(intentStr)
}
desc := strings.TrimSpace(firstStringField(obj,
@@ -486,43 +419,6 @@ func firstStringField(obj map[string]json.RawMessage, keys ...string) string {
return ""
}
func firstBoolField(obj map[string]json.RawMessage, keys ...string) (bool, bool) {
for _, key := range keys {
raw, ok := obj[key]
if !ok || len(raw) == 0 {
continue
}
if v, ok := parseBoolJSON(raw); ok {
return v, true
}
}
return false, false
}
func parseBoolJSON(raw json.RawMessage) (bool, bool) {
var b bool
if err := json.Unmarshal(raw, &b); err == nil {
return b, true
}
var s string
if err := json.Unmarshal(raw, &s); err == nil {
switch strings.ToLower(strings.TrimSpace(s)) {
case "true", "1", "yes", "y":
return true, true
case "false", "0", "no", "n":
return false, true
}
}
var n float64
if err := json.Unmarshal(raw, &n); err == nil {
return n != 0, true
}
return false, false
}
func mergeImageDescAndOCR(desc, ocr string) (string, bool) {
if desc == "" && ocr == "" {
return "", false
@@ -539,7 +435,7 @@ func mergeImageDescAndOCR(desc, ocr string) (string, bool) {
return desc + "\n\n[OCR]\n" + ocr, true
}
// formatConversationHistory formats conversation history for prompt template
// formatConversationHistory formats conversation history for prompt template.
func formatConversationHistory(historyList []*types.History) string {
if len(historyList) == 0 {
return ""
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -37,6 +37,9 @@ func (p *PluginRerank) ActivationEvents() []types.EventType {
func (p *PluginRerank) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
if !chatManage.NeedsRetrieval() {
return next()
}
pipelineInfo(ctx, "Rerank", "input", map[string]interface{}{
"session_id": chatManage.SessionID,
"candidate_cnt": len(chatManage.SearchResult),
@@ -114,10 +117,10 @@ func (p *PluginRerank) OnEvent(ctx context.Context,
degradedThreshold = 0.3
}
pipelineWarn(ctx, "Rerank", "threshold_degrade", map[string]interface{}{
"original": originalThreshold,
"degraded": degradedThreshold,
"candidate_cnt": len(candidatesToRerank),
"reason": "no results above original threshold, retrying with lower threshold",
"original": originalThreshold,
"degraded": degradedThreshold,
"candidate_cnt": len(candidatesToRerank),
"reason": "no results above original threshold, retrying with lower threshold",
})
chatManage.RerankThreshold = degradedThreshold
rerankResp = p.rerank(ctx, chatManage, rerankModel, chatManage.RewriteQuery, passages, candidatesToRerank)
@@ -356,11 +359,10 @@ func applyMMR(
"candidates": len(results),
})
// Pre-compute all token sets upfront (optimization)
allTokenSets := make([]map[string]struct{}, len(results))
for i, r := range results {
allTokenSets[i] = searchutil.TokenizeSimple(getEnrichedPassage(ctx, r))
}
// Pre-compute all token sets concurrently (CPU-bound tokenization)
allTokenSets := ParallelMap(results, 0, func(i int, r *types.SearchResult) map[string]struct{} {
return searchutil.TokenizeSimple(getEnrichedPassage(ctx, r))
})
selected := make([]*types.SearchResult, 0, k)
selectedTokenSets := make([]map[string]struct{}, 0, k)
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"testing"
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -1,4 +1,4 @@
package chatpipline
package chatpipeline
import (
"context"
@@ -1,8 +1,7 @@
package chatpipline
package chatpipeline
import (
"context"
"sync"
"github.com/Tencent/WeKnora/internal/config"
"github.com/Tencent/WeKnora/internal/logger"
@@ -89,8 +88,8 @@ func (p *PluginSearchParallel) ActivationEvents() []types.EventType {
func (p *PluginSearchParallel) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
// Intent-based skip: rewrite step determined KB retrieval is unnecessary
if chatManage.SkipKBSearch {
// Intent-based skip: query-understand step determined KB retrieval is unnecessary
if !chatManage.NeedsRetrieval() {
pipelineInfo(ctx, "SearchParallel", "skip", map[string]interface{}{
"session_id": chatManage.SessionID,
"reason": "intent_no_search",
@@ -104,88 +103,72 @@ func (p *PluginSearchParallel) OnEvent(ctx context.Context,
"rewrite_query": chatManage.RewriteQuery,
})
var wg sync.WaitGroup
var mu sync.Mutex
var chunkSearchErr *PluginError
var entitySearchErr *PluginError
// Deep-copy to avoid concurrent read/write on shared slice fields
chunkCM := chatManage.Clone()
chunkCM.SearchResult = nil
entityCM := chatManage.Clone()
entityCM.SearchResult = nil
// Use separate ChatManage copies to avoid concurrent write conflicts
chunkChatManage := *chatManage
chunkChatManage.SearchResult = nil
noop := func() *PluginError { return nil }
entityChatManage := *chatManage
entityChatManage.SearchResult = nil
tasks := []ParallelTask{
{
Name: "chunk_search",
Run: func() *PluginError {
err := p.searchPlugin.OnEvent(ctx, types.CHUNK_SEARCH, chunkCM, noop)
pipelineInfo(ctx, "SearchParallel", "chunk_search_done", map[string]interface{}{
"result_count": len(chunkCM.SearchResult),
"has_error": err != nil && err != ErrSearchNothing,
})
if err == ErrSearchNothing {
return nil
}
return err
},
},
{
Name: "entity_search",
Run: func() *PluginError {
if len(chatManage.Entity) == 0 {
pipelineInfo(ctx, "SearchParallel", "entity_search_skip", map[string]interface{}{
"reason": "no_entities",
})
return nil
}
err := p.searchEntityPlugin.OnEvent(ctx, types.ENTITY_SEARCH, entityCM, noop)
pipelineInfo(ctx, "SearchParallel", "entity_search_done", map[string]interface{}{
"result_count": len(entityCM.SearchResult),
"has_error": err != nil && err != ErrSearchNothing,
})
if err == ErrSearchNothing {
return nil
}
return err
},
},
}
// Run chunk search and entity search in parallel
wg.Add(2)
errs := RunParallel(tasks...)
// Goroutine 1: Chunk Search
go func() {
defer wg.Done()
err := p.searchPlugin.OnEvent(ctx, types.CHUNK_SEARCH, &chunkChatManage, func() *PluginError {
return nil
})
if err != nil && err != ErrSearchNothing {
mu.Lock()
chunkSearchErr = err
mu.Unlock()
}
pipelineInfo(ctx, "SearchParallel", "chunk_search_done", map[string]interface{}{
"result_count": len(chunkChatManage.SearchResult),
"has_error": err != nil && err != ErrSearchNothing,
})
}()
// Goroutine 2: Entity Search (only if entities are available)
go func() {
defer wg.Done()
if len(chatManage.Entity) == 0 {
pipelineInfo(ctx, "SearchParallel", "entity_search_skip", map[string]interface{}{
"reason": "no_entities",
})
return
}
err := p.searchEntityPlugin.OnEvent(ctx, types.ENTITY_SEARCH, &entityChatManage, func() *PluginError {
return nil
})
if err != nil && err != ErrSearchNothing {
mu.Lock()
entitySearchErr = err
mu.Unlock()
}
pipelineInfo(ctx, "SearchParallel", "entity_search_done", map[string]interface{}{
"result_count": len(entityChatManage.SearchResult),
"has_error": err != nil && err != ErrSearchNothing,
})
}()
wg.Wait()
// Merge results from both searches (no concurrent access now)
chatManage.SearchResult = append(chunkChatManage.SearchResult, entityChatManage.SearchResult...)
// Merge results from both searches
chatManage.SearchResult = append(chunkCM.SearchResult, entityCM.SearchResult...)
chatManage.SearchResult = removeDuplicateResults(chatManage.SearchResult)
// Log any errors but don't fail the pipeline if at least one search succeeded
if chunkSearchErr != nil {
logger.Warnf(ctx, "[SearchParallel] Chunk search error: %v", chunkSearchErr.Err)
}
if entitySearchErr != nil {
logger.Warnf(ctx, "[SearchParallel] Entity search error: %v", entitySearchErr.Err)
for name, err := range errs {
logger.Warnf(ctx, "[SearchParallel] %s error: %v", name, err.Err)
}
pipelineInfo(ctx, "SearchParallel", "complete", map[string]interface{}{
"session_id": chatManage.SessionID,
"chunk_results": len(chunkChatManage.SearchResult),
"entity_results": len(entityChatManage.SearchResult),
"total_results": len(chatManage.SearchResult),
"chunk_search_error": chunkSearchErr != nil,
"entity_search_error": entitySearchErr != nil,
"session_id": chatManage.SessionID,
"chunk_results": len(chunkCM.SearchResult),
"entity_results": len(entityCM.SearchResult),
"total_results": len(chatManage.SearchResult),
"error_count": len(errs),
})
// Return error only if both searches failed and we have no results
if len(chatManage.SearchResult) == 0 {
if chunkSearchErr != nil {
return chunkSearchErr
if err, ok := errs["chunk_search"]; ok {
return err
}
return ErrSearchNothing
}
@@ -1,128 +0,0 @@
// Package chatpipline provides chat pipeline processing capabilities
package chatpipline
import (
"context"
"regexp"
"slices"
"sort"
"github.com/Tencent/WeKnora/internal/config"
"github.com/Tencent/WeKnora/internal/types"
"github.com/Tencent/WeKnora/internal/types/interfaces"
)
// PluginLoadHistory is a plugin for loading conversation history without query rewriting
// It loads historical dialog context for multi-turn conversations
type PluginLoadHistory struct {
messageService interfaces.MessageService // Message service for retrieving historical messages
config *config.Config // System configuration
}
// regThink is a regular expression used to match and remove content between <think></think> tags
var regThink = regexp.MustCompile(`(?s)<think>.*?</think>`)
// NewPluginLoadHistory creates a new history loading plugin instance
// Also registers the plugin with the event manager
func NewPluginLoadHistory(eventManager *EventManager,
messageService interfaces.MessageService,
config *config.Config,
) *PluginLoadHistory {
res := &PluginLoadHistory{
messageService: messageService,
config: config,
}
eventManager.Register(res)
return res
}
// ActivationEvents returns the list of event types this plugin responds to
// This plugin only responds to LOAD_HISTORY events
func (p *PluginLoadHistory) ActivationEvents() []types.EventType {
return []types.EventType{types.LOAD_HISTORY}
}
// OnEvent processes triggered events
// When receiving a LOAD_HISTORY event, it loads conversation history without rewriting the query
func (p *PluginLoadHistory) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
// Determine max rounds from config or request
maxRounds := p.config.Conversation.MaxRounds
if chatManage.MaxRounds > 0 {
maxRounds = chatManage.MaxRounds
}
pipelineInfo(ctx, "LoadHistory", "input", map[string]interface{}{
"session_id": chatManage.SessionID,
"max_rounds": maxRounds,
})
// Get conversation history (fetch more to account for incomplete pairs)
history, err := p.messageService.GetRecentMessagesBySession(ctx, chatManage.SessionID, maxRounds*2+10)
if err != nil {
pipelineWarn(ctx, "LoadHistory", "history_fetch", map[string]interface{}{
"session_id": chatManage.SessionID,
"error": err.Error(),
})
return next()
}
pipelineInfo(ctx, "LoadHistory", "fetched", map[string]interface{}{
"session_id": chatManage.SessionID,
"message_count": len(history),
})
// Convert historical messages to conversation history structure
historyMap := make(map[string]*types.History)
// Process historical messages, grouped by requestID
for _, message := range history {
h, ok := historyMap[message.RequestID]
if !ok {
h = &types.History{}
}
if message.Role == "user" {
h.Query = message.Content
h.CreateAt = message.CreatedAt
if desc := extractImageCaptions(message.Images); desc != "" {
h.Query += "\n\n[用户上传图片内容]\n" + desc
}
} else {
// System message as answer, while removing thinking process
h.Answer = regThink.ReplaceAllString(message.Content, "")
h.KnowledgeReferences = message.KnowledgeReferences
}
historyMap[message.RequestID] = h
}
// Convert to list and filter incomplete conversations
historyList := make([]*types.History, 0)
for _, h := range historyMap {
if h.Answer != "" && h.Query != "" {
historyList = append(historyList, h)
}
}
// Sort by time, keep the most recent conversations
sort.Slice(historyList, func(i, j int) bool {
return historyList[i].CreateAt.After(historyList[j].CreateAt)
})
// Limit the number of historical records
if len(historyList) > maxRounds {
historyList = historyList[:maxRounds]
}
// Reverse to chronological order
slices.Reverse(historyList)
chatManage.History = historyList
pipelineInfo(ctx, "LoadHistory", "output", map[string]interface{}{
"session_id": chatManage.SessionID,
"history_rounds": len(historyList),
"max_rounds": maxRounds,
})
return next()
}
@@ -1,139 +0,0 @@
package chatpipline
import (
"context"
"errors"
"fmt"
"strings"
"github.com/Tencent/WeKnora/internal/event"
"github.com/Tencent/WeKnora/internal/types"
"github.com/google/uuid"
)
// PluginStreamFilter implements stream filtering functionality for chat pipeline
type PluginStreamFilter struct{}
// NewPluginStreamFilter creates a new stream filter plugin instance
func NewPluginStreamFilter(eventManager *EventManager) *PluginStreamFilter {
res := &PluginStreamFilter{}
eventManager.Register(res)
return res
}
// ActivationEvents returns the event types this plugin handles
func (p *PluginStreamFilter) ActivationEvents() []types.EventType {
return []types.EventType{types.STREAM_FILTER}
}
// OnEvent handles stream filtering events in the chat pipeline
func (p *PluginStreamFilter) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
pipelineInfo(ctx, "StreamFilter", "input", map[string]interface{}{
"session_id": chatManage.SessionID,
"has_event_bus": chatManage.EventBus != nil,
"no_match_prefix": chatManage.SummaryConfig.NoMatchPrefix,
})
// EventBus is required
if chatManage.EventBus == nil {
pipelineError(ctx, "StreamFilter", "eventbus_missing", map[string]interface{}{
"session_id": chatManage.SessionID,
})
return ErrModelCall.WithError(errors.New("EventBus is required for stream filtering"))
}
eventBus := chatManage.EventBus
// Check if no-match prefix filtering is needed
matchNoMatchBuilderPrefix := chatManage.SummaryConfig.NoMatchPrefix != ""
if matchNoMatchBuilderPrefix {
pipelineInfo(ctx, "StreamFilter", "enable_prefix_filter", map[string]interface{}{
"prefix": chatManage.SummaryConfig.NoMatchPrefix,
})
// Create an event interceptor for prefix filtering
return p.filterEventsWithPrefix(ctx, chatManage, eventBus, next)
}
// No filtering needed, just pass through
pipelineInfo(ctx, "StreamFilter", "passthrough", map[string]interface{}{
"session_id": chatManage.SessionID,
})
return next()
}
// filterEventsWithPrefix intercepts events, checks for NoMatchPrefix, and re-emits filtered events
func (p *PluginStreamFilter) filterEventsWithPrefix(
ctx context.Context,
chatManage *types.ChatManage,
originalEventBus types.EventBusInterface,
next func() *PluginError,
) *PluginError {
pipelineInfo(ctx, "StreamFilter", "setup_temp_bus", map[string]interface{}{
"session_id": chatManage.SessionID,
})
// Create a temporary EventBus to intercept events
tempEventBus := event.NewEventBus()
chatManage.EventBus = tempEventBus.AsEventBusInterface()
responseBuilder := &strings.Builder{}
matchFound := false
// Subscribe to answer events from temp bus
tempEventBus.On(event.EventAgentFinalAnswer, func(ctx context.Context, evt event.Event) error {
data, ok := evt.Data.(event.AgentFinalAnswerData)
if !ok {
return nil
}
responseBuilder.WriteString(data.Content)
// Check if content does NOT match the no-match prefix (meaning it's valid content)
if !strings.HasPrefix(chatManage.SummaryConfig.NoMatchPrefix, responseBuilder.String()) {
pipelineInfo(ctx, "StreamFilter", "emit_valid_chunk", map[string]interface{}{
"chunk_len": len(responseBuilder.String()),
})
// Emit the accumulated content as valid answer
originalEventBus.Emit(ctx, types.Event{
ID: evt.ID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: responseBuilder.String(),
Done: data.Done,
},
})
matchFound = true
}
return nil
})
// Call next to trigger pipeline stages that will emit to tempEventBus
err := next()
// After pipeline completes, check if we need fallback
if !matchFound && responseBuilder.Len() > 0 {
pipelineInfo(ctx, "StreamFilter", "emit_fallback", map[string]interface{}{
"session_id": chatManage.SessionID,
})
fallbackID := fmt.Sprintf("%s-fallback", uuid.New().String()[:8])
originalEventBus.Emit(ctx, types.Event{
ID: fallbackID,
Type: types.EventType(event.EventAgentFinalAnswer),
SessionID: chatManage.SessionID,
Data: event.AgentFinalAnswerData{
Content: chatManage.FallbackResponse,
Done: true,
},
})
}
// Restore original EventBus
chatManage.EventBus = originalEventBus
return err
}
@@ -1,283 +0,0 @@
package chatpipline
import (
"context"
"encoding/json"
"strings"
"time"
"github.com/Tencent/WeKnora/internal/event"
"github.com/Tencent/WeKnora/internal/logger"
"github.com/Tencent/WeKnora/internal/tracing"
"github.com/Tencent/WeKnora/internal/types"
"go.opentelemetry.io/otel/attribute"
)
// PluginTracing implements tracing functionality for chat pipeline events
type PluginTracing struct{}
// NewPluginTracing creates a new tracing plugin instance
func NewPluginTracing(eventManager *EventManager) *PluginTracing {
res := &PluginTracing{}
eventManager.Register(res)
return res
}
// ActivationEvents returns the event types this plugin handles
func (p *PluginTracing) ActivationEvents() []types.EventType {
return []types.EventType{
types.CHUNK_SEARCH,
types.CHUNK_RERANK,
types.CHUNK_MERGE,
types.INTO_CHAT_MESSAGE,
types.CHAT_COMPLETION,
types.CHAT_COMPLETION_STREAM,
types.FILTER_TOP_K,
types.REWRITE_QUERY,
types.CHUNK_SEARCH_PARALLEL,
}
}
// OnEvent handles incoming events and routes them to the appropriate tracing handler based on event type.
// It acts as the central dispatcher for all tracing-related events in the chat pipeline.
//
// Parameters:
// - ctx: context.Context for request-scoped values, cancellation signals, and deadlines
// - eventType: the type of event being processed (e.g., CHUNK_SEARCH, CHAT_COMPLETION)
// - chatManage: contains all the chat-related data and state for the current request
// - next: callback function to continue processing in the pipeline
//
// Returns:
// - *PluginError: error if any occurred during processing, or nil if successful
func (p *PluginTracing) OnEvent(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
switch eventType {
case types.CHUNK_SEARCH:
return p.Search(ctx, eventType, chatManage, next)
case types.CHUNK_RERANK:
return p.Rerank(ctx, eventType, chatManage, next)
case types.CHUNK_MERGE:
return p.Merge(ctx, eventType, chatManage, next)
case types.INTO_CHAT_MESSAGE:
return p.IntoChatMessage(ctx, eventType, chatManage, next)
case types.CHAT_COMPLETION:
return p.ChatCompletion(ctx, eventType, chatManage, next)
case types.CHAT_COMPLETION_STREAM:
return p.ChatCompletionStream(ctx, eventType, chatManage, next)
case types.FILTER_TOP_K:
return p.FilterTopK(ctx, eventType, chatManage, next)
case types.REWRITE_QUERY:
return p.RewriteQuery(ctx, eventType, chatManage, next)
case types.CHUNK_SEARCH_PARALLEL:
return p.SearchParallel(ctx, eventType, chatManage, next)
}
return next()
}
// Search traces search operations in the chat pipeline
func (p *PluginTracing) Search(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
_, span := tracing.ContextWithSpan(ctx, "PluginTracing.Search")
defer span.End()
span.SetAttributes(
attribute.String("query", chatManage.Query),
attribute.Float64("vector_threshold", chatManage.VectorThreshold),
attribute.Float64("keyword_threshold", chatManage.KeywordThreshold),
attribute.Int("match_count", chatManage.EmbeddingTopK),
)
err := next()
searchResultJson, _ := json.Marshal(chatManage.SearchResult)
unique := make(map[string]struct{})
for _, r := range chatManage.SearchResult {
unique[r.ID] = struct{}{}
}
span.SetAttributes(
attribute.String("hybrid_search", string(searchResultJson)),
attribute.Int("search_unique_count", len(unique)),
)
return err
}
// Rerank traces rerank operations in the chat pipeline
func (p *PluginTracing) Rerank(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
_, span := tracing.ContextWithSpan(ctx, "PluginTracing.Rerank")
defer span.End()
span.SetAttributes(
attribute.String("query", chatManage.Query),
attribute.Int("passages_count", len(chatManage.SearchResult)),
attribute.String("rerank_model_id", chatManage.RerankModelID),
attribute.Float64("rerank_filter_threshold", chatManage.RerankThreshold),
attribute.Int("rerank_filter_topk", chatManage.RerankTopK),
)
err := next()
resultJson, _ := json.Marshal(chatManage.RerankResult)
span.SetAttributes(
attribute.Int("rerank_resp_count", len(chatManage.RerankResult)),
attribute.String("rerank_resp_results", string(resultJson)),
)
return err
}
// Merge traces merge operations in the chat pipeline
func (p *PluginTracing) Merge(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
_, span := tracing.ContextWithSpan(ctx, "PluginTracing.Merge")
defer span.End()
span.SetAttributes(
attribute.Int("search_results_count", len(chatManage.SearchResult)),
attribute.Int("rerank_results_count", len(chatManage.RerankResult)),
)
err := next()
mergeResultJson, _ := json.Marshal(chatManage.MergeResult)
span.SetAttributes(
attribute.Int("merge_results_count", len(chatManage.MergeResult)),
attribute.String("merge_results", string(mergeResultJson)),
)
return err
}
// IntoChatMessage traces message conversion operations
func (p *PluginTracing) IntoChatMessage(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
_, span := tracing.ContextWithSpan(ctx, "PluginTracing.IntoChatMessage")
defer span.End()
span.SetAttributes(
attribute.Int("search_results_count", len(chatManage.SearchResult)),
attribute.Int("rerank_results_count", len(chatManage.RerankResult)),
attribute.Int("merge_results_count", len(chatManage.MergeResult)),
)
err := next()
span.SetAttributes(attribute.Int("generated_content_length", len(chatManage.UserContent)))
return err
}
// ChatCompletion traces chat completion operations
func (p *PluginTracing) ChatCompletion(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
_, span := tracing.ContextWithSpan(ctx, "PluginTracing.ChatCompletion")
defer span.End()
span.SetAttributes(
attribute.String("model_id", chatManage.ChatModelID),
attribute.String("system_prompt", chatManage.SummaryConfig.Prompt),
attribute.String("user_prompt", chatManage.UserContent),
attribute.Int("total_references", len(chatManage.RerankResult)),
)
err := next()
span.SetAttributes(
attribute.String("chat_response", chatManage.ChatResponse.Content),
attribute.Int("chat_response_tokens", chatManage.ChatResponse.Usage.TotalTokens),
attribute.Int("chat_response_prompt_tokens", chatManage.ChatResponse.Usage.PromptTokens),
attribute.Int("chat_response_completion_tokens", chatManage.ChatResponse.Usage.CompletionTokens),
)
return err
}
// ChatCompletionStream traces streaming chat completion operations
func (p *PluginTracing) ChatCompletionStream(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
ctx, span := tracing.ContextWithSpan(ctx, "PluginTracing.ChatCompletionStream")
startTime := time.Now()
span.SetAttributes(
attribute.String("model_id", chatManage.ChatModelID),
attribute.String("system_prompt", chatManage.SummaryConfig.Prompt),
attribute.String("user_prompt", chatManage.UserContent),
attribute.Int("total_references", len(chatManage.RerankResult)),
)
responseBuilder := &strings.Builder{}
// EventBus is required
if chatManage.EventBus == nil {
logger.Warn(ctx, "Tracing: EventBus not available, skipping metrics collection")
return next()
}
eventBus := chatManage.EventBus
// Subscribe to events and collect metrics
logger.Info(ctx, "Tracing: Subscribing to answer events for metrics collection")
eventBus.On(types.EventType(event.EventAgentFinalAnswer), func(ctx context.Context, evt types.Event) error {
data, ok := evt.Data.(event.AgentFinalAnswerData)
if ok {
responseBuilder.WriteString(data.Content)
// If this is the final chunk, record metrics
if data.Done {
elapsedMS := time.Since(startTime).Milliseconds()
span.SetAttributes(
attribute.Bool("chat_completion_success", true),
attribute.Int64("response_time_ms", elapsedMS),
attribute.String("chat_response", responseBuilder.String()),
attribute.Int("final_response_length", responseBuilder.Len()),
attribute.Float64("tokens_per_second", float64(responseBuilder.Len())/float64(elapsedMS)*1000),
)
span.End()
}
}
return nil
})
return next()
}
// FilterTopK traces filtering operations in the chat pipeline
func (p *PluginTracing) FilterTopK(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
_, span := tracing.ContextWithSpan(ctx, "PluginTracing.FilterTopK")
defer span.End()
span.SetAttributes(
attribute.Int("before_filter_search_results_count", len(chatManage.SearchResult)),
attribute.Int("before_filter_rerank_results_count", len(chatManage.RerankResult)),
attribute.Int("before_filter_merge_results_count", len(chatManage.MergeResult)),
)
err := next()
span.SetAttributes(
attribute.Int("after_filter_search_results_count", len(chatManage.SearchResult)),
attribute.Int("after_filter_rerank_results_count", len(chatManage.RerankResult)),
attribute.Int("after_filter_merge_results_count", len(chatManage.MergeResult)),
)
return err
}
// RewriteQuery traces query rewriting operations
func (p *PluginTracing) RewriteQuery(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
_, span := tracing.ContextWithSpan(ctx, "PluginTracing.RewriteQuery")
defer span.End()
span.SetAttributes(
attribute.String("query", chatManage.Query),
)
err := next()
span.SetAttributes(
attribute.String("rewrite_query", chatManage.RewriteQuery),
)
return err
}
// SearchParallel traces parallel search operations (chunk + entity)
func (p *PluginTracing) SearchParallel(ctx context.Context,
eventType types.EventType, chatManage *types.ChatManage, next func() *PluginError,
) *PluginError {
_, span := tracing.ContextWithSpan(ctx, "PluginTracing.SearchParallel")
defer span.End()
span.SetAttributes(
attribute.String("query", chatManage.Query),
attribute.String("rewrite_query", chatManage.RewriteQuery),
attribute.Int("entity_count", len(chatManage.Entity)),
)
err := next()
span.SetAttributes(
attribute.Int("search_result_count", len(chatManage.SearchResult)),
)
return err
}
+26 -24
View File
@@ -266,31 +266,33 @@ func (e *EvaluationService) Evaluation(ctx context.Context,
StartTime: time.Now(),
},
Params: &types.ChatManage{
VectorThreshold: e.config.Conversation.VectorThreshold,
KeywordThreshold: e.config.Conversation.KeywordThreshold,
EmbeddingTopK: e.config.Conversation.EmbeddingTopK,
MaxRounds: e.config.Conversation.MaxRounds,
RerankModelID: rerankModelID,
RerankTopK: e.config.Conversation.RerankTopK,
RerankThreshold: e.config.Conversation.RerankThreshold,
ChatModelID: chatModelID,
SummaryConfig: types.SummaryConfig{
MaxTokens: e.config.Conversation.Summary.MaxTokens,
RepeatPenalty: e.config.Conversation.Summary.RepeatPenalty,
TopK: e.config.Conversation.Summary.TopK,
TopP: e.config.Conversation.Summary.TopP,
Prompt: e.config.Conversation.Summary.Prompt,
ContextTemplate: e.config.Conversation.Summary.ContextTemplate,
FrequencyPenalty: e.config.Conversation.Summary.FrequencyPenalty,
PresencePenalty: e.config.Conversation.Summary.PresencePenalty,
NoMatchPrefix: e.config.Conversation.Summary.NoMatchPrefix,
Temperature: e.config.Conversation.Summary.Temperature,
Seed: e.config.Conversation.Summary.Seed,
MaxCompletionTokens: e.config.Conversation.Summary.MaxCompletionTokens,
PipelineRequest: types.PipelineRequest{
VectorThreshold: e.config.Conversation.VectorThreshold,
KeywordThreshold: e.config.Conversation.KeywordThreshold,
EmbeddingTopK: e.config.Conversation.EmbeddingTopK,
MaxRounds: e.config.Conversation.MaxRounds,
RerankModelID: rerankModelID,
RerankTopK: e.config.Conversation.RerankTopK,
RerankThreshold: e.config.Conversation.RerankThreshold,
ChatModelID: chatModelID,
SummaryConfig: types.SummaryConfig{
MaxTokens: e.config.Conversation.Summary.MaxTokens,
RepeatPenalty: e.config.Conversation.Summary.RepeatPenalty,
TopK: e.config.Conversation.Summary.TopK,
TopP: e.config.Conversation.Summary.TopP,
Prompt: e.config.Conversation.Summary.Prompt,
ContextTemplate: e.config.Conversation.Summary.ContextTemplate,
FrequencyPenalty: e.config.Conversation.Summary.FrequencyPenalty,
PresencePenalty: e.config.Conversation.Summary.PresencePenalty,
NoMatchPrefix: e.config.Conversation.Summary.NoMatchPrefix,
Temperature: e.config.Conversation.Summary.Temperature,
Seed: e.config.Conversation.Summary.Seed,
MaxCompletionTokens: e.config.Conversation.Summary.MaxCompletionTokens,
},
FallbackResponse: e.config.Conversation.FallbackResponse,
RewritePromptSystem: e.config.Conversation.RewritePromptSystem,
RewritePromptUser: e.config.Conversation.RewritePromptUser,
},
FallbackResponse: e.config.Conversation.FallbackResponse,
RewritePromptSystem: e.config.Conversation.RewritePromptSystem,
RewritePromptUser: e.config.Conversation.RewritePromptUser,
},
}
+3 -3
View File
@@ -10,7 +10,7 @@ import (
"github.com/Tencent/WeKnora/internal/agent/tools"
filesvc "github.com/Tencent/WeKnora/internal/application/service/file"
chatpipline "github.com/Tencent/WeKnora/internal/application/service/chat_pipline"
chatpipeline "github.com/Tencent/WeKnora/internal/application/service/chat_pipeline"
"github.com/Tencent/WeKnora/internal/application/service/retriever"
"github.com/Tencent/WeKnora/internal/config"
"github.com/Tencent/WeKnora/internal/logger"
@@ -155,7 +155,7 @@ func NewChunkExtractService(
chunkRepo interfaces.ChunkRepository,
graphEngine interfaces.RetrieveGraphRepository,
) interfaces.TaskHandler {
// generator := chatpipline.NewQAPromptGenerator(chatpipline.NewFormater(), config.ExtractManager.ExtractGraph)
// generator := chatpipeline.NewQAPromptGenerator(chatpipeline.NewFormater(), config.ExtractManager.ExtractGraph)
// ctx := context.Background()
// logger.Debugf(ctx, "chunk extract system prompt: %s", generator.System(ctx))
// logger.Debugf(ctx, "chunk extract user prompt: %s", generator.User(ctx, "demo"))
@@ -211,7 +211,7 @@ func (s *ChunkExtractService) Handle(ctx context.Context, t *asynq.Task) error {
},
},
}
extractor := chatpipline.NewExtractor(chatModel, template)
extractor := chatpipeline.NewExtractor(chatModel, template)
graph, err := extractor.Extract(ctx, chunk.Content)
if err != nil {
return err
-1
View File
@@ -126,7 +126,6 @@ func (s *modelService) GetModelByID(ctx context.Context, id string) (*types.Mode
// Check model status
if model.Status == types.ModelStatusActive {
logger.Info(ctx, "Model is active and ready to use")
return model, nil
}
+3 -3
View File
@@ -14,7 +14,7 @@ import (
"github.com/Tencent/WeKnora/internal/types/interfaces"
"github.com/google/uuid"
chatpipline "github.com/Tencent/WeKnora/internal/application/service/chat_pipline"
chatpipeline "github.com/Tencent/WeKnora/internal/application/service/chat_pipeline"
llmcontext "github.com/Tencent/WeKnora/internal/application/service/llmcontext"
)
@@ -31,7 +31,7 @@ type sessionService struct {
knowledgeBaseService interfaces.KnowledgeBaseService // Service for knowledge base operations
modelService interfaces.ModelService // Service for model operations
tenantService interfaces.TenantService // Service for tenant operations
eventManager *chatpipline.EventManager // Event manager for chat pipeline
eventManager *chatpipeline.EventManager // Event manager for chat pipeline
agentService interfaces.AgentService // Service for agent operations
sessionStorage llmcontext.ContextStorage // Session storage
knowledgeService interfaces.KnowledgeService // Service for knowledge operations
@@ -50,7 +50,7 @@ func NewSessionService(cfg *config.Config,
chunkService interfaces.ChunkService,
modelService interfaces.ModelService,
tenantService interfaces.TenantService,
eventManager *chatpipline.EventManager,
eventManager *chatpipeline.EventManager,
agentService interfaces.AgentService,
sessionStorage llmcontext.ContextStorage,
webSearchStateRepo interfaces.WebSearchStateService,
@@ -4,15 +4,14 @@ import (
"context"
"fmt"
"strings"
"time"
chatpipline "github.com/Tencent/WeKnora/internal/application/service/chat_pipline"
chatpipeline "github.com/Tencent/WeKnora/internal/application/service/chat_pipeline"
"github.com/Tencent/WeKnora/internal/common"
"github.com/Tencent/WeKnora/internal/event"
"github.com/Tencent/WeKnora/internal/logger"
"github.com/Tencent/WeKnora/internal/models/chat"
"github.com/Tencent/WeKnora/internal/tracing"
"github.com/Tencent/WeKnora/internal/types"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/codes"
)
// KnowledgeQA performs knowledge base question answering with LLM summarization
@@ -91,40 +90,45 @@ func (s *sessionService) KnowledgeQA(
userID, _ := types.UserIDFromContext(ctx)
chatManage := &types.ChatManage{
Query: req.Query,
RewriteQuery: req.Query,
SessionID: req.Session.ID,
UserID: userID,
MessageID: req.AssistantMessageID,
KnowledgeBaseIDs: knowledgeBaseIDs,
KnowledgeIDs: knowledgeIDs,
SearchTargets: searchTargets,
VectorThreshold: s.cfg.Conversation.VectorThreshold,
KeywordThreshold: s.cfg.Conversation.KeywordThreshold,
EmbeddingTopK: s.cfg.Conversation.EmbeddingTopK,
RerankTopK: s.cfg.Conversation.RerankTopK,
RerankThreshold: s.cfg.Conversation.RerankThreshold,
MaxRounds: s.cfg.Conversation.MaxRounds,
ChatModelID: chatModelID,
SummaryConfig: summaryConfig,
FallbackStrategy: fallbackStrategy,
FallbackResponse: s.cfg.Conversation.FallbackResponse,
FallbackPrompt: s.cfg.Conversation.FallbackPrompt,
EventBus: eventBus.AsEventBusInterface(),
WebSearchEnabled: req.WebSearchEnabled,
EnableMemory: req.EnableMemory,
TenantID: retrievalTenantID,
RewritePromptSystem: s.cfg.Conversation.RewritePromptSystem,
RewritePromptUser: s.cfg.Conversation.RewritePromptUser,
EnableRewrite: s.cfg.Conversation.EnableRewrite,
EnableQueryExpansion: s.cfg.Conversation.EnableQueryExpansion,
// Image support
UserMessageID: req.UserMessageID,
Images: req.ImageURLs,
ImageDescription: req.ImageDescription,
VLMModelID: vlmModelID,
ChatModelSupportsVision: chatModelSupportsVision,
Language: types.LanguageNameFromContext(ctx),
PipelineRequest: types.PipelineRequest{
Query: req.Query,
SessionID: req.Session.ID,
UserID: userID,
EnableMemory: req.EnableMemory,
MaxRounds: s.cfg.Conversation.MaxRounds,
KnowledgeBaseIDs: knowledgeBaseIDs,
KnowledgeIDs: knowledgeIDs,
SearchTargets: searchTargets,
VectorThreshold: s.cfg.Conversation.VectorThreshold,
KeywordThreshold: s.cfg.Conversation.KeywordThreshold,
EmbeddingTopK: s.cfg.Conversation.EmbeddingTopK,
RerankTopK: s.cfg.Conversation.RerankTopK,
RerankThreshold: s.cfg.Conversation.RerankThreshold,
ChatModelID: chatModelID,
SummaryConfig: summaryConfig,
FallbackStrategy: fallbackStrategy,
FallbackResponse: s.cfg.Conversation.FallbackResponse,
FallbackPrompt: s.cfg.Conversation.FallbackPrompt,
EnableRewrite: s.cfg.Conversation.EnableRewrite,
EnableQueryExpansion: s.cfg.Conversation.EnableQueryExpansion,
RewritePromptSystem: s.cfg.Conversation.RewritePromptSystem,
RewritePromptUser: s.cfg.Conversation.RewritePromptUser,
WebSearchEnabled: req.WebSearchEnabled,
TenantID: retrievalTenantID,
Images: req.ImageURLs,
VLMModelID: vlmModelID,
ChatModelSupportsVision: chatModelSupportsVision,
Language: types.LanguageNameFromContext(ctx),
},
PipelineState: types.PipelineState{
RewriteQuery: req.Query,
ImageDescription: req.ImageDescription,
},
PipelineContext: types.PipelineContext{
EventBus: eventBus.AsEventBusInterface(),
MessageID: req.AssistantMessageID,
UserMessageID: req.UserMessageID,
},
}
// Apply custom agent overrides (system prompt, temperature, retrieval params,
@@ -132,36 +136,43 @@ func (s *sessionService) KnowledgeQA(
s.applyAgentOverridesToChatManage(ctx, req.CustomAgent, chatManage)
// Determine pipeline based on knowledge bases availability and web search setting
// If no knowledge bases are selected AND web search is disabled, use pure chat pipeline
// Otherwise use rag_stream pipeline (which handles both KB search and web search)
hasKB := len(knowledgeBaseIDs) > 0 || len(knowledgeIDs) > 0
needsRAG := hasKB || req.WebSearchEnabled
hasHistory := chatManage.MaxRounds > 0
var pipeline []types.EventType
if len(knowledgeBaseIDs) == 0 && len(knowledgeIDs) == 0 && !req.WebSearchEnabled {
logger.Info(ctx, "No knowledge bases selected and web search disabled, using chat pipeline")
// For pure chat, UserContent is the Query (since INTO_CHAT_MESSAGE is skipped)
// Only append image text description for non-vision models; vision models see images directly
if !needsRAG {
// Pure chat — no retrieval needed.
userContent := req.Query
if req.ImageDescription != "" && !chatModelSupportsVision {
userContent += "\n\n[用户上传图片内容]\n" + req.ImageDescription
}
chatManage.UserContent = userContent
// Use chat_history_stream if multi-turn is enabled, otherwise use chat_stream
if chatManage.MaxRounds > 0 {
logger.Infof(ctx, "Multi-turn enabled with maxRounds=%d, using chat_history_stream pipeline", chatManage.MaxRounds)
pipeline = types.Pipline["chat_history_stream"]
} else {
logger.Info(ctx, "Multi-turn disabled, using chat_stream pipeline")
pipeline = types.Pipline["chat_stream"]
}
pipeline = types.NewPipelineBuilder().
AddIf(hasHistory, types.LOAD_HISTORY).
AddIf(chatManage.EnableMemory, types.MEMORY_RETRIEVAL).
Add(types.CHAT_COMPLETION_STREAM).
AddIf(chatManage.EnableMemory, types.MEMORY_STORAGE).
Build()
} else {
if req.WebSearchEnabled && len(knowledgeBaseIDs) == 0 && len(knowledgeIDs) == 0 {
logger.Info(ctx, "Web search enabled without knowledge bases, using rag_stream pipeline for web search only")
} else {
logger.Info(ctx, "Knowledge bases selected, using rag_stream pipeline")
}
pipeline = types.Pipline["rag_stream"]
// RAG — dynamically assemble based on feature flags.
pipeline = types.NewPipelineBuilder().
Add(types.LOAD_HISTORY).
Add(types.QUERY_UNDERSTAND).
Add(types.CHUNK_SEARCH_PARALLEL).
Add(types.CHUNK_RERANK).
Add(types.CHUNK_MERGE).
Add(types.FILTER_TOP_K).
Add(types.DATA_ANALYSIS).
Add(types.INTO_CHAT_MESSAGE).
Add(types.CHAT_COMPLETION_STREAM).
Build()
}
logger.Infof(ctx, "Assembled pipeline (%d stages), hasKB=%v, webSearch=%v, history=%v",
len(pipeline), hasKB, req.WebSearchEnabled, hasHistory)
// Start knowledge QA event processing (set session tenant so pipeline session/message lookups use session owner)
ctx = context.WithValue(ctx, types.SessionTenantIDContextKey, req.Session.TenantID)
logger.Info(ctx, "Triggering question answering event")
@@ -465,57 +476,54 @@ func (s *sessionService) buildSearchTargets(
func (s *sessionService) KnowledgeQAByEvent(ctx context.Context,
chatManage *types.ChatManage, eventList []types.EventType,
) error {
ctx, span := tracing.ContextWithSpan(ctx, "SessionService.KnowledgeQAByEvent")
defer span.End()
logger.Info(ctx, "Start processing knowledge base question answering through events")
logger.Infof(ctx, "Knowledge base question answering parameters, session ID: %s, query: %s",
logger.Infof(ctx, "Knowledge base question answering parameters, session ID: %s, query: %s",
chatManage.SessionID, chatManage.Query)
// Prepare method list for logging and tracing
methods := []string{}
for _, event := range eventList {
methods = append(methods, string(event))
methods := make([]string, len(eventList))
for i, event := range eventList {
methods[i] = string(event)
}
// Set up tracing attributes
logger.Infof(ctx, "Trigger event list: %v", methods)
span.SetAttributes(
attribute.String("request_id", func() string { id, _ := types.RequestIDFromContext(ctx); return id }()),
attribute.String("query", chatManage.Query),
attribute.String("method", strings.Join(methods, ",")),
)
// Process each event in sequence
pipelineStart := time.Now()
for _, eventType := range eventList {
logger.Infof(ctx, "Starting to trigger event: %v", eventType)
stageStart := time.Now()
err := s.eventManager.Trigger(ctx, eventType, chatManage)
stageDuration := time.Since(stageStart)
// Handle case where search returns no results
if err == chatpipline.ErrSearchNothing {
logger.Warnf(
ctx,
"Event %v triggered, search result is empty, using fallback response, strategy: %v",
eventType,
chatManage.FallbackStrategy,
)
if err == chatpipeline.ErrSearchNothing {
common.PipelineWarn(ctx, "Pipeline", "stage_fallback", map[string]interface{}{
"event": string(eventType),
"duration_ms": stageDuration.Milliseconds(),
"reason": "search_nothing",
"strategy": string(chatManage.FallbackStrategy),
})
s.handleFallbackResponse(ctx, chatManage)
return nil
}
// Handle other errors
if err != nil {
logger.Errorf(ctx, "Event triggering failed, event: %v, error type: %s, description: %s, error: %v",
eventType, err.ErrorType, err.Description, err.Err)
span.RecordError(err.Err)
span.SetStatus(codes.Error, err.Description)
span.SetAttributes(attribute.String("error_type", err.ErrorType))
common.PipelineError(ctx, "Pipeline", "stage_failed", map[string]interface{}{
"event": string(eventType),
"duration_ms": stageDuration.Milliseconds(),
"error_type": err.ErrorType,
"description": err.Description,
})
return err.Err
}
logger.Infof(ctx, "Event %v triggered successfully", eventType)
common.PipelineInfo(ctx, "Pipeline", "stage_complete", map[string]interface{}{
"event": string(eventType),
"duration_ms": stageDuration.Milliseconds(),
})
}
logger.Info(ctx, "All events triggered successfully")
common.PipelineInfo(ctx, "Pipeline", "all_stages_complete", map[string]interface{}{
"session_id": chatManage.SessionID,
"total_stages": len(eventList),
"total_duration_ms": time.Since(pipelineStart).Milliseconds(),
})
return nil
}
@@ -557,18 +565,22 @@ func (s *sessionService) SearchKnowledge(ctx context.Context,
}
chatManage := &types.ChatManage{
Query: query,
RewriteQuery: query,
UserID: userID,
KnowledgeBaseIDs: knowledgeBaseIDs,
KnowledgeIDs: knowledgeIDs,
SearchTargets: searchTargets,
MaxRounds: s.cfg.Conversation.MaxRounds,
EmbeddingTopK: rc.GetEffectiveEmbeddingTopK(),
VectorThreshold: rc.GetEffectiveVectorThreshold(),
KeywordThreshold: rc.GetEffectiveKeywordThreshold(),
RerankTopK: rc.GetEffectiveRerankTopK(),
RerankThreshold: rc.GetEffectiveRerankThreshold(),
PipelineRequest: types.PipelineRequest{
Query: query,
UserID: userID,
KnowledgeBaseIDs: knowledgeBaseIDs,
KnowledgeIDs: knowledgeIDs,
SearchTargets: searchTargets,
MaxRounds: s.cfg.Conversation.MaxRounds,
EmbeddingTopK: rc.GetEffectiveEmbeddingTopK(),
VectorThreshold: rc.GetEffectiveVectorThreshold(),
KeywordThreshold: rc.GetEffectiveKeywordThreshold(),
RerankTopK: rc.GetEffectiveRerankTopK(),
RerankThreshold: rc.GetEffectiveRerankThreshold(),
},
PipelineState: types.PipelineState{
RewriteQuery: query,
},
}
// Get default models
@@ -601,42 +613,20 @@ func (s *sessionService) SearchKnowledge(ctx context.Context,
types.FILTER_TOP_K, // Filter top K results
}
ctx, span := tracing.ContextWithSpan(ctx, "SessionService.SearchKnowledge")
defer span.End()
logger.Infof(ctx, "Trigger search event list: %v", searchEvents)
// Prepare method list for logging and tracing
methods := []string{}
for _, event := range searchEvents {
methods = append(methods, string(event))
}
// Set up tracing attributes
logger.Infof(ctx, "Trigger search event list: %v", methods)
span.SetAttributes(
attribute.String("query", query),
attribute.StringSlice("knowledge_base_ids", knowledgeBaseIDs),
attribute.StringSlice("knowledge_ids", knowledgeIDs),
attribute.String("method", strings.Join(methods, ",")),
)
// Process each search event in sequence
for _, event := range searchEvents {
logger.Infof(ctx, "Starting to trigger search event: %v", event)
err := s.eventManager.Trigger(ctx, event, chatManage)
// Handle case where search returns no results
if err == chatpipline.ErrSearchNothing {
if err == chatpipeline.ErrSearchNothing {
logger.Warnf(ctx, "Event %v triggered, search result is empty", event)
return []*types.SearchResult{}, nil
}
// Handle other errors
if err != nil {
logger.Errorf(ctx, "Event triggering failed, event: %v, error type: %s, description: %s, error: %v",
event, err.ErrorType, err.Description, err.Err)
span.RecordError(err.Err)
span.SetStatus(codes.Error, err.Description)
span.SetAttributes(attribute.String("error_type", err.ErrorType))
return nil, err.Err
}
logger.Infof(ctx, "Event %v triggered successfully", event)
+19
View File
@@ -102,6 +102,10 @@ type ConversationConfig struct {
ExtractEntitiesPrompt string `yaml:"-" json:"extract_entities_prompt"`
ExtractRelationshipsPrompt string `yaml:"-" json:"extract_relationships_prompt"`
GenerateQuestionsPrompt string `yaml:"-" json:"generate_questions_prompt"`
// IntentSystemPrompts maps intent values (e.g. "greeting", "chitchat") to
// system prompt text. Populated by backfill from IntentPrompts templates.
IntentSystemPrompts map[string]string `yaml:"-" json:"-"`
}
// SummaryConfig 摘要配置
@@ -201,6 +205,8 @@ type PromptTemplatesConfig struct {
AgentSystemPrompt []PromptTemplate `yaml:"agent_system_prompt" json:"agent_system_prompt,omitempty"`
GraphExtraction []PromptTemplate `yaml:"graph_extraction" json:"graph_extraction,omitempty"`
GenerateQuestions []PromptTemplate `yaml:"generate_questions" json:"generate_questions,omitempty"`
// IntentPrompts holds per-intent system prompt overrides (template ID = intent value).
IntentPrompts []PromptTemplate `yaml:"intent_prompts" json:"intent_prompts,omitempty"`
}
// DefaultTemplate returns the first template marked as default in the list,
@@ -504,6 +510,17 @@ func backfillConversationDefaults(cfg *Config) {
}
}
}
// Build intent→system-prompt map from IntentPrompts templates.
// Template ID must equal the QueryIntent string value (e.g. "greeting").
if len(pt.IntentPrompts) > 0 {
conv.IntentSystemPrompts = make(map[string]string, len(pt.IntentPrompts))
for _, t := range pt.IntentPrompts {
if t.ID != "" && t.Content != "" {
conv.IntentSystemPrompts[t.ID] = t.Content
}
}
}
}
// FindTemplateByID searches across all template lists for a template with the given ID.
@@ -524,6 +541,7 @@ func FindTemplateByID(pt *PromptTemplatesConfig, id string) *PromptTemplate {
pt.AgentSystemPrompt,
pt.GraphExtraction,
pt.GenerateQuestions,
pt.IntentPrompts,
} {
for i := range list {
if list[i].ID == id {
@@ -574,6 +592,7 @@ func loadPromptTemplates(configDir string) (*PromptTemplatesConfig, error) {
"agent_system_prompt.yaml": &config.AgentSystemPrompt,
"graph_extraction.yaml": &config.GraphExtraction,
"generate_questions.yaml": &config.GenerateQuestions,
"intent_prompts.yaml": &config.IntentPrompts,
}
// 加载每个模板文件
+16 -18
View File
@@ -42,7 +42,7 @@ import (
sqliteRetrieverRepo "github.com/Tencent/WeKnora/internal/application/repository/retriever/sqlite"
weaviateRepo "github.com/Tencent/WeKnora/internal/application/repository/retriever/weaviate"
"github.com/Tencent/WeKnora/internal/application/service"
chatpipline "github.com/Tencent/WeKnora/internal/application/service/chat_pipline"
chatpipeline "github.com/Tencent/WeKnora/internal/application/service/chat_pipeline"
"github.com/Tencent/WeKnora/internal/application/service/file"
"github.com/Tencent/WeKnora/internal/application/service/llmcontext"
memoryService "github.com/Tencent/WeKnora/internal/application/service/memory"
@@ -203,23 +203,21 @@ func BuildContainer(container *dig.Container) *dig.Container {
// Chat pipeline components for processing chat requests
logger.Debugf(ctx, "[Container] Registering chat pipeline plugins...")
must(container.Provide(chatpipline.NewEventManager))
must(container.Invoke(chatpipline.NewPluginTracing))
must(container.Invoke(chatpipline.NewPluginSearch))
must(container.Invoke(chatpipline.NewPluginRerank))
must(container.Invoke(chatpipline.NewPluginMerge))
must(container.Invoke(chatpipline.NewPluginDataAnalysis))
must(container.Invoke(chatpipline.NewPluginIntoChatMessage))
must(container.Invoke(chatpipline.NewPluginChatCompletion))
must(container.Invoke(chatpipline.NewPluginChatCompletionStream))
must(container.Invoke(chatpipline.NewPluginStreamFilter))
must(container.Invoke(chatpipline.NewPluginFilterTopK))
must(container.Invoke(chatpipline.NewPluginRewrite))
must(container.Invoke(chatpipline.NewPluginLoadHistory))
must(container.Invoke(chatpipline.NewPluginExtractEntity))
must(container.Invoke(chatpipline.NewPluginSearchEntity))
must(container.Invoke(chatpipline.NewPluginSearchParallel))
must(container.Invoke(chatpipline.NewMemoryPlugin))
must(container.Provide(chatpipeline.NewEventManager))
must(container.Invoke(chatpipeline.NewPluginSearch))
must(container.Invoke(chatpipeline.NewPluginRerank))
must(container.Invoke(chatpipeline.NewPluginMerge))
must(container.Invoke(chatpipeline.NewPluginDataAnalysis))
must(container.Invoke(chatpipeline.NewPluginIntoChatMessage))
must(container.Invoke(chatpipeline.NewPluginChatCompletion))
must(container.Invoke(chatpipeline.NewPluginChatCompletionStream))
must(container.Invoke(chatpipeline.NewPluginFilterTopK))
must(container.Invoke(chatpipeline.NewPluginQueryUnderstand))
must(container.Invoke(chatpipeline.NewPluginLoadHistory))
must(container.Invoke(chatpipeline.NewPluginExtractEntity))
must(container.Invoke(chatpipeline.NewPluginSearchEntity))
must(container.Invoke(chatpipeline.NewPluginSearchParallel))
must(container.Invoke(chatpipeline.NewMemoryPlugin))
logger.Debugf(ctx, "[Container] Chat pipeline plugins registered")
// HTTP handlers layer
+2 -2
View File
@@ -13,7 +13,7 @@ import (
"sync"
"time"
chatpipline "github.com/Tencent/WeKnora/internal/application/service/chat_pipline"
chatpipeline "github.com/Tencent/WeKnora/internal/application/service/chat_pipeline"
"github.com/Tencent/WeKnora/internal/config"
"github.com/Tencent/WeKnora/internal/errors"
"github.com/Tencent/WeKnora/internal/logger"
@@ -2092,7 +2092,7 @@ func (h *InitializationHandler) extractRelationsFromText(
Examples: h.config.ExtractManager.ExtractGraph.Examples,
}
extractor := chatpipline.NewExtractor(chatModel, template)
extractor := chatpipeline.NewExtractor(chatModel, template)
graph, err := extractor.Extract(ctx, text)
if err != nil {
logger.Error(ctx, "文本关系提取失败", err)
+217 -141
View File
@@ -1,87 +1,140 @@
package types
// ChatManage represents the configuration and state for a chat session
// including query processing, search parameters, and model configurations
type ChatManage struct {
SessionID string `json:"session_id"` // Unique identifier for the chat session
UserID string `json:"user_id"` // Unique identifier for the user
Query string `json:"query,omitempty"` // Original user query
RewriteQuery string `json:"rewrite_query,omitempty"` // Query after rewriting for better retrieval
EnableMemory bool `json:"enable_memory"` // Whether memory feature is enabled
History []*History `json:"history,omitempty"` // Chat history for context
// PipelineRequest holds immutable configuration set once at the request entry point.
type PipelineRequest struct {
SessionID string `json:"session_id"`
UserID string `json:"user_id"`
Query string `json:"query,omitempty"`
EnableMemory bool `json:"enable_memory"`
MaxRounds int `json:"max_rounds"`
KnowledgeBaseIDs []string `json:"knowledge_base_ids"` // IDs of knowledge bases to search (multi-KB support)
KnowledgeIDs []string `json:"knowledge_ids,omitempty"` // IDs of specific files to search (optional)
// SearchTargets is the pre-computed unified search targets
// Computed once at request entry point, used throughout the pipeline
// Knowledge base retrieval parameters
KnowledgeBaseIDs []string `json:"knowledge_base_ids"`
KnowledgeIDs []string `json:"knowledge_ids,omitempty"`
SearchTargets SearchTargets `json:"-"`
VectorThreshold float64 `json:"vector_threshold"` // Minimum score threshold for vector search results
KeywordThreshold float64 `json:"keyword_threshold"` // Minimum score threshold for keyword search results
EmbeddingTopK int `json:"embedding_top_k"` // Number of top results to retrieve from embedding search
VectorDatabase string `json:"vector_database"` // Vector database type/name to use
VectorThreshold float64 `json:"vector_threshold"`
KeywordThreshold float64 `json:"keyword_threshold"`
EmbeddingTopK int `json:"embedding_top_k"`
VectorDatabase string `json:"vector_database"`
RerankModelID string `json:"rerank_model_id"` // Model ID for reranking search results
RerankTopK int `json:"rerank_top_k"` // Number of top results after reranking
RerankThreshold float64 `json:"rerank_threshold"` // Minimum score threshold for reranked results
// Rerank parameters
RerankModelID string `json:"rerank_model_id"`
RerankTopK int `json:"rerank_top_k"`
RerankThreshold float64 `json:"rerank_threshold"`
MaxRounds int `json:"max_rounds"` // Maximum history rounds used for rewrite/context
// Chat model parameters
ChatModelID string `json:"chat_model_id"`
SummaryConfig SummaryConfig `json:"summary_config"`
FallbackStrategy FallbackStrategy `json:"fallback_strategy"`
FallbackResponse string `json:"fallback_response"`
FallbackPrompt string `json:"fallback_prompt"`
ChatModelID string `json:"chat_model_id"` // ID of the chat model to use
SummaryConfig SummaryConfig `json:"summary_config"` // Configuration for summary generation
FallbackStrategy FallbackStrategy `json:"fallback_strategy"` // Strategy when no relevant results are found
FallbackResponse string `json:"fallback_response"` // Default response when fallback occurs
FallbackPrompt string `json:"fallback_prompt"` // Prompt for model-based fallback response
// Rewrite parameters
EnableRewrite bool `json:"enable_rewrite"`
EnableQueryExpansion bool `json:"enable_query_expansion"`
RewritePromptSystem string `json:"rewrite_prompt_system"`
RewritePromptUser string `json:"rewrite_prompt_user"`
EnableRewrite bool `json:"enable_rewrite"` // Whether to enable rewrite
EnableQueryExpansion bool `json:"enable_query_expansion"` // Whether to enable query expansion with LLM
RewritePromptSystem string `json:"rewrite_prompt_system"` // Custom system prompt for rewrite stage
RewritePromptUser string `json:"rewrite_prompt_user"` // Custom user prompt for rewrite stage
// FAQ strategy
FAQPriorityEnabled bool `json:"-"`
FAQDirectAnswerThreshold float64 `json:"-"`
FAQScoreBoost float64 `json:"-"`
// Internal fields for pipeline data processing
SearchResult []*SearchResult `json:"-"` // Results from search phase
RerankResult []*SearchResult `json:"-"` // Results after reranking
MergeResult []*SearchResult `json:"-"` // Final merged results after all processing
Entity []string `json:"-"` // List of identified entities
EntityKBIDs []string `json:"-"` // Knowledge base IDs with ExtractConfig enabled
EntityKnowledge map[string]string `json:"-"` // KnowledgeID -> KnowledgeBaseID mapping for graph-enabled files
GraphResult *GraphData `json:"-"` // Graph data from search phase
UserContent string `json:"-"` // Processed user content
ChatResponse *ChatResponse `json:"-"` // Final response from chat model
// Image / multimodal support
Images []string `json:"-"`
VLMModelID string `json:"-"`
ChatModelSupportsVision bool `json:"-"`
// Event system for streaming responses
EventBus EventBusInterface `json:"-"` // EventBus for emitting streaming events
MessageID string `json:"-"` // Assistant message ID for event emission
// Web search configuration (internal use)
TenantID uint64 `json:"-"` // Tenant ID for retrieving web search config
WebSearchEnabled bool `json:"-"` // Whether web search is enabled for this request
// FAQ Strategy Settings
FAQPriorityEnabled bool `json:"-"` // Whether FAQ priority strategy is enabled
FAQDirectAnswerThreshold float64 `json:"-"` // Threshold for direct FAQ answer (similarity > this value)
FAQScoreBoost float64 `json:"-"` // Score multiplier for FAQ results
// Image support for multimodal chat
UserMessageID string `json:"-"` // User message ID for updating image captions after rewrite
Images []string `json:"-"` // Image URLs for MultiContent in current user message
ImageDescription string `json:"-"` // Image description (visual details + OCR text) generated by VLM (used as fallback for non-vision models)
VLMModelID string `json:"-"` // Agent-configured VLM model ID for image analysis
ChatModelSupportsVision bool `json:"-"` // Whether the chat model accepts multimodal/image input
SkipKBSearch bool `json:"-"` // Set by rewrite intent classification: true = skip KB retrieval
Language string `json:"-"` // User language name for prompt placeholder (e.g. "Chinese (Simplified)", "English")
// Misc request-scoped config
TenantID uint64 `json:"-"`
WebSearchEnabled bool `json:"-"`
Language string `json:"-"`
}
// Clone creates a deep copy of the ChatManage object
// QueryIntent represents the classified intent of a user query.
type QueryIntent string
const (
IntentKBSearch QueryIntent = "kb_search"
IntentWebSearch QueryIntent = "web_search"
IntentGreeting QueryIntent = "greeting"
IntentChitchat QueryIntent = "chitchat"
IntentFollowUp QueryIntent = "follow_up"
IntentImageOnly QueryIntent = "image_only"
IntentSummarize QueryIntent = "summarize"
IntentClarification QueryIntent = "clarification"
)
// NeedsKBRetrieval returns true when the intent requires knowledge base search.
// The zero value (empty string) is treated as needing retrieval for safety.
// Note: IntentWebSearch is NOT included — use ChatManage.NeedsRetrieval()
// which also considers the WebSearchEnabled flag.
func (i QueryIntent) NeedsKBRetrieval() bool {
switch i {
case IntentKBSearch, IntentClarification, "":
return true
default:
return false
}
}
// PipelineState holds mutable intermediate data that plugins read and write
// as the pipeline progresses.
type PipelineState struct {
RewriteQuery string `json:"rewrite_query,omitempty"`
Intent QueryIntent `json:"intent,omitempty"`
History []*History `json:"history,omitempty"`
SearchResult []*SearchResult `json:"-"`
RerankResult []*SearchResult `json:"-"`
MergeResult []*SearchResult `json:"-"`
Entity []string `json:"-"`
EntityKBIDs []string `json:"-"`
EntityKnowledge map[string]string `json:"-"`
GraphResult *GraphData `json:"-"`
UserContent string `json:"-"`
ChatResponse *ChatResponse `json:"-"`
ImageDescription string `json:"-"`
SystemPromptOverride string `json:"-"`
}
// PipelineContext holds runtime context for the current pipeline execution.
type PipelineContext struct {
EventBus EventBusInterface `json:"-"`
MessageID string `json:"-"`
UserMessageID string `json:"-"`
}
// ChatManage represents the full configuration, state and runtime context
// for a chat pipeline execution. It embeds PipelineRequest (immutable config),
// PipelineState (mutable intermediate data), and PipelineContext (runtime handles).
type ChatManage struct {
PipelineRequest
PipelineState
PipelineContext
}
// NeedsRetrieval returns true when the current pipeline execution should
// run the retrieval stages (search, rerank, merge, etc.).
// For IntentWebSearch, retrieval is only needed if web search is enabled;
// for all other intents it delegates to QueryIntent.NeedsKBRetrieval().
func (c *ChatManage) NeedsRetrieval() bool {
if c.Intent == IntentWebSearch {
return c.WebSearchEnabled
}
return c.Intent.NeedsKBRetrieval()
}
// Clone creates a deep copy of the ChatManage object.
// PipelineContext fields (EventBus, MessageID, etc.) are NOT copied because they
// are per-execution handles that should not be shared across clones.
func (c *ChatManage) Clone() *ChatManage {
// Deep copy knowledge base IDs slice
knowledgeBaseIDs := make([]string, len(c.KnowledgeBaseIDs))
copy(knowledgeBaseIDs, c.KnowledgeBaseIDs)
// Deep copy knowledge IDs slice
knowledgeIDs := make([]string, len(c.KnowledgeIDs))
copy(knowledgeIDs, c.KnowledgeIDs)
// Deep copy search targets slice
searchTargets := make(SearchTargets, len(c.SearchTargets))
for i, t := range c.SearchTargets {
if t != nil {
@@ -96,55 +149,47 @@ func (c *ChatManage) Clone() *ChatManage {
}
return &ChatManage{
Query: c.Query,
RewriteQuery: c.RewriteQuery,
SessionID: c.SessionID,
KnowledgeBaseIDs: knowledgeBaseIDs,
KnowledgeIDs: knowledgeIDs,
SearchTargets: searchTargets,
VectorThreshold: c.VectorThreshold,
KeywordThreshold: c.KeywordThreshold,
EmbeddingTopK: c.EmbeddingTopK,
MaxRounds: c.MaxRounds,
VectorDatabase: c.VectorDatabase,
RerankModelID: c.RerankModelID,
RerankTopK: c.RerankTopK,
RerankThreshold: c.RerankThreshold,
ChatModelID: c.ChatModelID,
SummaryConfig: SummaryConfig{
MaxTokens: c.SummaryConfig.MaxTokens,
RepeatPenalty: c.SummaryConfig.RepeatPenalty,
TopK: c.SummaryConfig.TopK,
TopP: c.SummaryConfig.TopP,
FrequencyPenalty: c.SummaryConfig.FrequencyPenalty,
PresencePenalty: c.SummaryConfig.PresencePenalty,
Prompt: c.SummaryConfig.Prompt,
ContextTemplate: c.SummaryConfig.ContextTemplate,
NoMatchPrefix: c.SummaryConfig.NoMatchPrefix,
Temperature: c.SummaryConfig.Temperature,
Seed: c.SummaryConfig.Seed,
MaxCompletionTokens: c.SummaryConfig.MaxCompletionTokens,
Thinking: c.SummaryConfig.Thinking,
PipelineRequest: PipelineRequest{
Query: c.Query,
SessionID: c.SessionID,
UserID: c.UserID,
EnableMemory: c.EnableMemory,
MaxRounds: c.MaxRounds,
KnowledgeBaseIDs: knowledgeBaseIDs,
KnowledgeIDs: knowledgeIDs,
SearchTargets: searchTargets,
VectorThreshold: c.VectorThreshold,
KeywordThreshold: c.KeywordThreshold,
EmbeddingTopK: c.EmbeddingTopK,
VectorDatabase: c.VectorDatabase,
RerankModelID: c.RerankModelID,
RerankTopK: c.RerankTopK,
RerankThreshold: c.RerankThreshold,
ChatModelID: c.ChatModelID,
SummaryConfig: c.SummaryConfig,
FallbackStrategy: c.FallbackStrategy,
FallbackResponse: c.FallbackResponse,
FallbackPrompt: c.FallbackPrompt,
EnableRewrite: c.EnableRewrite,
EnableQueryExpansion: c.EnableQueryExpansion,
RewritePromptSystem: c.RewritePromptSystem,
RewritePromptUser: c.RewritePromptUser,
FAQPriorityEnabled: c.FAQPriorityEnabled,
FAQDirectAnswerThreshold: c.FAQDirectAnswerThreshold,
FAQScoreBoost: c.FAQScoreBoost,
Images: append([]string(nil), c.Images...),
VLMModelID: c.VLMModelID,
ChatModelSupportsVision: c.ChatModelSupportsVision,
TenantID: c.TenantID,
WebSearchEnabled: c.WebSearchEnabled,
Language: c.Language,
},
PipelineState: PipelineState{
RewriteQuery: c.RewriteQuery,
Intent: c.Intent,
ImageDescription: c.ImageDescription,
SystemPromptOverride: c.SystemPromptOverride,
},
FallbackStrategy: c.FallbackStrategy,
FallbackResponse: c.FallbackResponse,
FallbackPrompt: c.FallbackPrompt,
RewritePromptSystem: c.RewritePromptSystem,
RewritePromptUser: c.RewritePromptUser,
EnableRewrite: c.EnableRewrite,
EnableQueryExpansion: c.EnableQueryExpansion,
TenantID: c.TenantID,
// FAQ Strategy Settings
FAQPriorityEnabled: c.FAQPriorityEnabled,
FAQDirectAnswerThreshold: c.FAQDirectAnswerThreshold,
FAQScoreBoost: c.FAQScoreBoost,
UserMessageID: c.UserMessageID,
Images: append([]string(nil), c.Images...),
ImageDescription: c.ImageDescription,
VLMModelID: c.VLMModelID,
ChatModelSupportsVision: c.ChatModelSupportsVision,
SkipKBSearch: c.SkipKBSearch,
Language: c.Language,
}
}
@@ -152,56 +197,87 @@ func (c *ChatManage) Clone() *ChatManage {
type EventType string
const (
LOAD_HISTORY EventType = "load_history" // Load conversation history without rewriting
REWRITE_QUERY EventType = "rewrite_query" // Query rewriting for better retrieval
CHUNK_SEARCH EventType = "chunk_search" // Search for relevant chunks
CHUNK_SEARCH_PARALLEL EventType = "chunk_search_parallel" // Parallel search: chunks + entities
ENTITY_SEARCH EventType = "entity_search" // Search for relevant entities
CHUNK_RERANK EventType = "chunk_rerank" // Rerank search results
CHUNK_MERGE EventType = "chunk_merge" // Merge similar chunks
DATA_ANALYSIS EventType = "data_analysis" // Data analysis for CSV/Excel files
INTO_CHAT_MESSAGE EventType = "into_chat_message" // Convert chunks into chat messages
CHAT_COMPLETION EventType = "chat_completion" // Generate chat completion
CHAT_COMPLETION_STREAM EventType = "chat_completion_stream" // Stream chat completion
STREAM_FILTER EventType = "stream_filter" // Filter streaming output
FILTER_TOP_K EventType = "filter_top_k" // Keep only top K results
MEMORY_RETRIEVAL EventType = "memory_retrieval" // Retrieve memory context
MEMORY_STORAGE EventType = "memory_storage" // Store conversation to memory
LOAD_HISTORY EventType = "load_history"
QUERY_UNDERSTAND EventType = "query_understand"
CHUNK_SEARCH EventType = "chunk_search"
CHUNK_SEARCH_PARALLEL EventType = "chunk_search_parallel"
ENTITY_SEARCH EventType = "entity_search"
CHUNK_RERANK EventType = "chunk_rerank"
CHUNK_MERGE EventType = "chunk_merge"
DATA_ANALYSIS EventType = "data_analysis"
INTO_CHAT_MESSAGE EventType = "into_chat_message"
CHAT_COMPLETION EventType = "chat_completion"
CHAT_COMPLETION_STREAM EventType = "chat_completion_stream"
FILTER_TOP_K EventType = "filter_top_k"
MEMORY_RETRIEVAL EventType = "memory_retrieval"
MEMORY_STORAGE EventType = "memory_storage"
)
// Pipline defines the sequence of events for different chat modes
var Pipline = map[string][]EventType{
"chat": { // Simple chat without retrieval
// PipelineBuilder dynamically assembles a pipeline as an ordered list of EventTypes.
type PipelineBuilder struct {
stages []EventType
}
// NewPipelineBuilder returns an empty builder.
func NewPipelineBuilder() *PipelineBuilder {
return &PipelineBuilder{}
}
// Add appends one or more stages unconditionally.
func (b *PipelineBuilder) Add(stages ...EventType) *PipelineBuilder {
b.stages = append(b.stages, stages...)
return b
}
// AddIf appends stages only when the condition is true.
func (b *PipelineBuilder) AddIf(cond bool, stages ...EventType) *PipelineBuilder {
if cond {
b.stages = append(b.stages, stages...)
}
return b
}
// Build returns the final event list. The builder must not be reused.
func (b *PipelineBuilder) Build() []EventType {
out := make([]EventType, len(b.stages))
copy(out, b.stages)
return out
}
// Pipeline defines the sequence of events for different chat modes.
// Kept as a convenience lookup for callers that don't need dynamic composition.
var Pipeline = map[string][]EventType{
"chat": {
CHAT_COMPLETION,
},
"chat_stream": { // Streaming chat without retrieval (no history)
"chat_stream": {
CHAT_COMPLETION_STREAM,
STREAM_FILTER,
},
"chat_history_stream": { // Streaming chat with conversation history
"chat_history_stream": {
LOAD_HISTORY,
MEMORY_RETRIEVAL,
CHAT_COMPLETION_STREAM,
STREAM_FILTER,
MEMORY_STORAGE,
},
"rag": { // Retrieval Augmented Generation
"rag": {
CHUNK_SEARCH,
CHUNK_RERANK,
CHUNK_MERGE,
INTO_CHAT_MESSAGE,
CHAT_COMPLETION,
},
"rag_stream": { // Streaming Retrieval Augmented Generation
LOAD_HISTORY, // Load conversation history independently (works even when rewrite is disabled)
REWRITE_QUERY,
CHUNK_SEARCH_PARALLEL, // Parallel: CHUNK_SEARCH + ENTITY_SEARCH
"rag_stream": {
LOAD_HISTORY,
QUERY_UNDERSTAND,
CHUNK_SEARCH_PARALLEL,
CHUNK_RERANK,
CHUNK_MERGE,
FILTER_TOP_K,
DATA_ANALYSIS,
INTO_CHAT_MESSAGE,
CHAT_COMPLETION_STREAM,
STREAM_FILTER,
},
}
// Pipline is a deprecated alias for Pipeline (kept for backward compatibility).
var Pipline = Pipeline