fix: strip thinking content before indexing to chat history KB

The assistant message content may contain <think>...</think> tags from
extended thinking mode. Previously this raw thinking content was
indexed into the chat history knowledge base, polluting embeddings
and degrading RAG retrieval quality with intermediate reasoning text.

Now we strip thinking blocks before building the passage for
IndexMessageToKB, consistent with how load_history and rewrite
plugins already strip thinking from conversation history.
This commit is contained in:
lyingbug
2026-03-20 22:35:45 +00:00
committed by lyingbug
parent cbee01f5e9
commit 1c5013e559
+11 -1
View File
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"regexp"
"sort"
"strings"
"time"
@@ -13,6 +14,9 @@ import (
"github.com/Tencent/WeKnora/internal/types/interfaces"
)
// regThinkIndex matches <think>...</think> blocks for stripping from KB index content.
var regThinkIndex = regexp.MustCompile(`(?s)<think>.*?</think>`)
// messageService implements the MessageService interface for managing messaging operations
// It handles creating, retrieving, updating, and deleting messages within sessions.
// It reads the chat history knowledge base configuration from the tenant's ChatHistoryConfig,
@@ -342,7 +346,13 @@ func (s *messageService) getRetrievalConfig(ctx context.Context) *types.Retrieva
// then links the message to the Knowledge entry via the knowledge_id field.
// The KB ID is read from the tenant's ChatHistoryConfig — if not configured, indexing is skipped.
func (s *messageService) IndexMessageToKB(ctx context.Context, userQuery string, assistantAnswer string, messageID string, sessionID string) {
if strings.TrimSpace(userQuery) == "" && strings.TrimSpace(assistantAnswer) == "" {
// Strip thinking content (<think>...</think>) before indexing to avoid
// polluting the knowledge base with intermediate reasoning that would
// degrade retrieval quality.
assistantAnswer = regThinkIndex.ReplaceAllString(assistantAnswer, "")
assistantAnswer = strings.TrimSpace(assistantAnswer)
if strings.TrimSpace(userQuery) == "" && assistantAnswer == "" {
return
}