feat: Enhance wiki content processing with enriched image information and XML output

- Introduced `enrichChunkImageInfo` function to populate image information for text chunks by retrieving data from child chunks, ensuring that image-related content is not lost during processing.
- Updated `reconstructContent` to `reconstructEnrichedContent`, allowing for the inclusion of OCR text and captions from images, improving the quality of content sent to the LLM.
- Added `writeDedupItemXML` and `xmlEscape` functions to format entity and concept entries as structured XML, enhancing clarity and preventing nonsensical merges in deduplication prompts.
- Refined the `deduplicateExtractedBatch` method to utilize the new XML formatting for better output consistency.

These changes significantly improve the handling of image data and the overall clarity of deduplication prompts, contributing to a more effective wiki content management system.
This commit is contained in:
wizardchen
2026-04-20 17:55:30 +08:00
parent 173547a17a
commit 53ecd5df5d
4 changed files with 155 additions and 62 deletions
+19 -7
View File
@@ -115,6 +115,15 @@ Output ONLY valid JSON. Example:
// WikiPageModifyPrompt updates an existing wiki page with new additions and removes stale/deleted information in a single pass.
const WikiPageModifyPrompt = `You are a wiki editor tasked with updating an existing wiki page. You must process a set of NEW information to add, AND/OR a set of deleted documents whose exclusive contributions must be REMOVED.
<page_metadata>
<slug>{{.PageSlug}}</slug>
<title>{{.PageTitle}}</title>
<type>{{.PageType}}</type>{{if .PageAliases}}
<aliases>{{.PageAliases}}</aliases>{{end}}
</page_metadata>
This wiki page is specifically about **{{.PageTitle}}** (a {{.PageType}}). Every statement on the page MUST be directly about this exact {{.PageType}} — not about related, adjacent, or similarly-named things.
<existing_page_content>
{{.ExistingContent}}
</existing_page_content>
@@ -146,15 +155,15 @@ const WikiPageModifyPrompt = `You are a wiki editor tasked with updating an exis
{{end}}
{{if .HasAdditions}}
3. ADD and MERGE the facts, details, and context from the <new_information> into the page.
- **CRITICAL CONFLICT CHECK**: First verify that the <new_information> describes the EXACT SAME core entity/concept as this page. If the new info clearly belongs to a DIFFERENT but related product/entity (e.g., this page is about "Hunyuan Model" but the new info is about "Qwen3"), you MUST REJECT that part of the new information and DO NOT add it.
- If it is the same core entity and contradicts old content, prefer the newer information.
- **CRITICAL CONFLICT CHECK**: First verify that the <new_information> is actually about **{{.PageTitle}}** (as declared in <page_metadata>). If a piece of new info clearly belongs to a DIFFERENT but related thing (e.g., this page is about "Hunyuan Model" but the new info is about "Qwen3"; or this page is about "居民身份证" but the new info is about "工作居住证"), you MUST REJECT that part of the new information and DO NOT add it.
- If it is genuinely about {{.PageTitle}} and contradicts old content, prefer the newer information.
{{end}}
4. Preserve existing information that is still valid.
5. Keep [[slug|name]] wiki-link references ONLY if the slug appears in the <valid_wiki_links> list above. Remove any [[slug|name]] whose slug is NOT in that list. Do NOT invent new wiki-link slugs.
6. Maintain the existing page structure and formatting style.
4. Preserve existing information that is still valid and still about {{.PageTitle}}.
5. Keep [[slug|name]] wiki-link references ONLY if the slug appears in the <valid_wiki_links> list above. Remove any [[slug|name]] whose slug is NOT in that list. Do NOT invent new wiki-link slugs. The page's own slug ({{.PageSlug}}) MUST NOT appear as a [[...]] link inside its own content.
6. Maintain the existing page structure and formatting style. Use "# {{.PageTitle}}" as the top-level heading if the page does not already have one.
7. **Image rule**: Include relevant images using Markdown syntax: ![caption](url) from new information if applicable.
{{if .HasRetractions}}
8. If after removing deleted content the page becomes nearly empty and there is no new information to add, output just: "SUMMARY: (empty page)\n# [Title]\n\n*This page's primary source document was removed.*"
8. If after removing deleted content the page becomes nearly empty and there is no new information to add, output just: "SUMMARY: (empty page)\n# {{.PageTitle}}\n\n*This page's primary source document was removed.*"
{{end}}
9. Write in {{.Language}}.
</instructions>
@@ -243,8 +252,11 @@ const WikiDeduplicationPrompt = `You are a strict deduplication system. Given a
- "Competition Categories" → "Age Groups" (age groups are one aspect of categories, not the same concept)
- "Performance Standard" → "Competition Rounds" (both relate to competitions, but are different concepts)
- "Machine Learning" → "Neural Networks" (neural networks are a subset of ML, not the same concept)
- "居民身份证 / Resident ID Card" → "工作居住证 / Work Residence Permit" (both are government-issued documents but completely different credentials)
- "驾驶证 / Driver's License" → "行驶证 / Vehicle Registration" (both are car-related certificates but different documents)
- "学位证 / Degree Certificate" → "毕业证 / Graduation Certificate" (both educational documents but distinct)
### Key principle: **related ≠ same**. **ABSOLUTELY DO NOT** merge different products, different companies, or different versions just because they belong to the same industry. When in doubt, do NOT merge. It is far better to have two separate pages for the same thing than to wrongly merge two different things.
### Key principle: **related ≠ same**. Two items sharing a few characters in their name, or belonging to the same domain / document family / industry, is NOT a reason to merge. **ABSOLUTELY DO NOT** merge different products, different companies, different versions, or different certificates/documents just because they belong to the same category. When in doubt, do NOT merge. It is far better to have two separate pages for the same thing than to wrongly merge two different things.
Return a JSON object with a "merges" map. The key is the NEW item's slug, the value is the EXISTING page's slug that it should merge into. Only include items where you are highly confident they are the same thing.
@@ -7,6 +7,7 @@ import (
"regexp"
"strings"
"github.com/Tencent/WeKnora/internal/searchutil"
"github.com/Tencent/WeKnora/internal/types"
"github.com/Tencent/WeKnora/internal/types/interfaces"
)
@@ -53,6 +54,41 @@ If neither query nor range is provided, it returns the beginning of the document
}
}
// enrichChunkImageInfo populates chunk.ImageInfo for a batch of parent text
// chunks by looking up their image_ocr / image_caption children. Chunks that
// already have a non-empty ImageInfo are left untouched.
func enrichChunkImageInfo(
ctx context.Context,
chunkRepo interfaces.ChunkRepository,
tenantID uint64,
chunks []*types.Chunk,
) {
if len(chunks) == 0 || chunkRepo == nil {
return
}
ids := make([]string, 0, len(chunks))
for _, c := range chunks {
if c.ImageInfo == "" && c.ID != "" {
ids = append(ids, c.ID)
}
}
if len(ids) == 0 {
return
}
infoMap := searchutil.CollectImageInfoByChunkIDs(ctx, chunkRepo, tenantID, ids)
if len(infoMap) == 0 {
return
}
for _, c := range chunks {
if c.ImageInfo != "" {
continue
}
if merged, ok := infoMap[c.ID]; ok && merged != "" {
c.ImageInfo = merged
}
}
}
func enrichChunkContent(c *types.Chunk) string {
content := c.Content
if c.ImageInfo != "" {
@@ -164,6 +200,12 @@ func (t *wikiReadSourceDocTool) Execute(ctx context.Context, args json.RawMessag
break
}
// Lazily enrich ImageInfo from image_ocr / image_caption child chunks.
// Parent text chunks don't carry image_info themselves (see
// image_multimodal.go); without this the source-doc reader would
// silently drop OCR text and captions for image-heavy documents.
enrichChunkImageInfo(ctx, t.chunkService.GetRepository(), knowledge.TenantID, chunks)
for _, c := range chunks {
chunkNum := c.ChunkIndex + 1
chunkContent := enrichChunkContent(c)
+66 -53
View File
@@ -12,6 +12,7 @@ import (
"github.com/Tencent/WeKnora/internal/agent"
"github.com/Tencent/WeKnora/internal/logger"
"github.com/Tencent/WeKnora/internal/models/chat"
"github.com/Tencent/WeKnora/internal/searchutil"
"github.com/Tencent/WeKnora/internal/types"
"github.com/Tencent/WeKnora/internal/types/interfaces"
"github.com/hibiken/asynq"
@@ -718,6 +719,32 @@ func (s *wikiIngestService) publishDraftPages(ctx context.Context, kbID string,
}
}
// writeDedupItemXML renders a single entity/concept entry as a structured XML
// block for the deduplication prompt. Structured form (versus a single
// pipe-separated line) helps the LLM reliably tell name / aliases / type apart
// and reduces nonsensical merges like "居民身份证" → "工作居住证".
func writeDedupItemXML(buf *strings.Builder, slug, name, itemType string, aliases []string) {
fmt.Fprintf(buf, " <item slug=%q type=%q>\n", slug, itemType)
fmt.Fprintf(buf, " <name>%s</name>\n", xmlEscape(name))
for _, alias := range aliases {
if alias == "" {
continue
}
fmt.Fprintf(buf, " <alias>%s</alias>\n", xmlEscape(alias))
}
buf.WriteString(" </item>\n")
}
// xmlEscape escapes the minimal set of characters that can break XML text
// content. Slugs are ASCII-only so they don't need escaping when used as
// attribute values.
func xmlEscape(s string) string {
s = strings.ReplaceAll(s, "&", "&amp;")
s = strings.ReplaceAll(s, "<", "&lt;")
s = strings.ReplaceAll(s, ">", "&gt;")
return s
}
// deduplicateExtractedBatch deduplicates both entities and concepts against
// existing wiki pages in a single LLM call. Uses pre-loaded allPages to avoid
// redundant DB queries. This replaces the two separate deduplicateItems calls
@@ -737,11 +764,7 @@ func (s *wikiIngestService) deduplicateExtractedBatch(
if p.PageType != types.WikiPageTypeEntity && p.PageType != types.WikiPageTypeConcept {
continue
}
aliases := ""
if len(p.Aliases) > 0 {
aliases = fmt.Sprintf(" | aliases: %s", strings.Join(p.Aliases, ", "))
}
fmt.Fprintf(&existingBuf, "- slug: %s | title: %s | type: %s%s\n", p.Slug, p.Title, p.PageType, aliases)
writeDedupItemXML(&existingBuf, p.Slug, p.Title, string(p.PageType), []string(p.Aliases))
}
if existingBuf.Len() == 0 {
return entities, concepts
@@ -749,18 +772,10 @@ func (s *wikiIngestService) deduplicateExtractedBatch(
var newBuf strings.Builder
for _, item := range entities {
aliases := ""
if len(item.Aliases) > 0 {
aliases = fmt.Sprintf(" | aliases: %s", strings.Join(item.Aliases, ", "))
}
fmt.Fprintf(&newBuf, "- slug: %s | name: %s | type: entity%s\n", item.Slug, item.Name, aliases)
writeDedupItemXML(&newBuf, item.Slug, item.Name, "entity", item.Aliases)
}
for _, item := range concepts {
aliases := ""
if len(item.Aliases) > 0 {
aliases = fmt.Sprintf(" | aliases: %s", strings.Join(item.Aliases, ", "))
}
fmt.Fprintf(&newBuf, "- slug: %s | name: %s | type: concept%s\n", item.Slug, item.Name, aliases)
writeDedupItemXML(&newBuf, item.Slug, item.Name, "concept", item.Aliases)
}
dedupeJSON, err := s.generateWithTemplate(ctx, chatModel, agent.WikiDeduplicationPrompt, map[string]string{
@@ -850,7 +865,14 @@ func (s *wikiIngestService) generateWithTemplate(ctx context.Context, chatModel
// --- Helpers ---
// reconstructContent rebuilds document text from chunks
// reconstructContent rebuilds document text from chunks.
//
// This only concatenates text-type chunks — image OCR / caption information is
// stored on image_ocr / image_caption child chunks (see image_multimodal.go),
// not on the parent text chunk's ImageInfo field. Callers that need the full
// enriched content (with OCR / captions inlined) should call
// reconstructEnrichedContent instead so image info is fetched from child
// chunks and embedded alongside Markdown image links.
func reconstructContent(chunks []*types.Chunk) string {
var textChunks []*types.Chunk
for _, c := range chunks {
@@ -899,50 +921,41 @@ func reconstructContent(chunks []*types.Chunk) string {
// If c.EndAt <= lastEndAt, it's fully contained, so skip appending text
}
// Append image information at the end to avoid interrupting text flow
var hasImages bool
seenURLs := make(map[string]bool)
for _, c := range textChunks {
if c.ImageInfo != "" {
var imageInfos []types.ImageInfo
if err := json.Unmarshal([]byte(c.ImageInfo), &imageInfos); err == nil && len(imageInfos) > 0 {
for _, img := range imageInfos {
// Deduplicate images by URL to avoid printing the same image multiple times from overlapping chunks
if img.URL != "" {
if seenURLs[img.URL] {
continue
}
seenURLs[img.URL] = true
}
return sb.String()
}
if !hasImages {
sb.WriteString("\n\n<images>\n")
hasImages = true
} else {
sb.WriteString("\n")
}
// reconstructEnrichedContent rebuilds document text and inlines image_info
// (OCR text + caption) pulled from image_ocr / image_caption child chunks.
//
// Without this enrichment, image-heavy documents (e.g. a scanned PDF or a
// standalone .jpg) reach the LLM as bare Markdown image links, causing
// extraction / summarization to produce empty or "no textual content" output.
func reconstructEnrichedContent(
ctx context.Context,
chunkRepo interfaces.ChunkRepository,
tenantID uint64,
chunks []*types.Chunk,
) string {
content := reconstructContent(chunks)
sb.WriteString("<image>\n")
if img.URL != "" {
sb.WriteString(fmt.Sprintf(" <url>%s</url>\n", img.URL))
}
if img.Caption != "" {
sb.WriteString(fmt.Sprintf(" <caption>%s</caption>\n", img.Caption))
}
if img.OCRText != "" {
sb.WriteString(fmt.Sprintf(" <ocr_text>%s</ocr_text>\n", img.OCRText))
}
sb.WriteString("</image>\n")
}
var textChunkIDs []string
for _, c := range chunks {
if c.ChunkType == types.ChunkTypeText || c.ChunkType == "" {
if c.ID != "" {
textChunkIDs = append(textChunkIDs, c.ID)
}
}
}
if hasImages {
sb.WriteString("</images>\n")
if len(textChunkIDs) == 0 || chunkRepo == nil {
return content
}
return sb.String()
imageInfoMap := searchutil.CollectImageInfoByChunkIDs(ctx, chunkRepo, tenantID, textChunkIDs)
mergedImageInfo := searchutil.MergeImageInfoJSON(imageInfoMap)
if mergedImageInfo == "" {
return content
}
return searchutil.EnrichContentWithImageInfo(content, mergedImageInfo)
}
// slugify creates a URL-friendly slug from a string
@@ -381,7 +381,7 @@ func (s *wikiIngestService) mapOneDocument(
return nil, nil, nil
}
content := reconstructContent(chunks)
content := reconstructEnrichedContent(ctx, s.chunkRepo, payload.TenantID, chunks)
rawRuneCount := len([]rune(content))
if len([]rune(content)) > maxContentForWiki {
content = string([]rune(content)[:maxContentForWiki])
@@ -538,14 +538,22 @@ func (s *wikiIngestService) extractEntitiesAndConceptsNoUpsert(
oldPageSlugs map[string]bool,
batchCtx *WikiBatchContext,
) ([]extractedItem, []extractedItem, map[string]extractedItem, error) {
// Only entity/* and concept/* slugs are relevant for LLM slug-continuity —
// summary slugs are code-generated from the document title and never appear
// in the extraction output, so including them just wastes tokens and risks
// confusing the model.
var prevSlugsText string
if len(oldPageSlugs) > 0 {
var sb strings.Builder
for slug := range oldPageSlugs {
if !strings.HasPrefix(slug, "entity/") && !strings.HasPrefix(slug, "concept/") {
continue
}
fmt.Fprintf(&sb, "- %s\n", slug)
}
prevSlugsText = sb.String()
} else {
}
if prevSlugsText == "" {
prevSlugsText = "(none — this is a new document)"
}
@@ -756,9 +764,27 @@ func (s *wikiIngestService) reduceSlugUpdates(
hasRetractionsStr = "1"
}
// Fall back gracefully if title/type are still unset (shouldn't happen
// for well-formed updates — both get populated from `additions` above,
// and retract-only paths require an existing page — but stay defensive
// so we never feed the LLM an empty identity block).
pageTitle := page.Title
if pageTitle == "" {
pageTitle = slug
}
pageType := string(page.PageType)
if pageType == "" {
pageType = "wiki page"
}
pageAliases := strings.Join(page.Aliases, ", ")
updatedContent, err := s.generateWithTemplate(ctx, chatModel, agent.WikiPageModifyPrompt, map[string]string{
"HasAdditions": hasAdditionsStr,
"HasRetractions": hasRetractionsStr,
"PageSlug": slug,
"PageTitle": pageTitle,
"PageType": pageType,
"PageAliases": pageAliases,
"ExistingContent": existingContent,
"NewContent": newContentBuilder.String(),
"DeletedContent": deletedContent.String(),