Files
Enterprise-Knowledge-Base/WIKI_README.md
T
wizardchen 28deb8282f feat: Add comprehensive documentation for WeKnora knowledge base and wiki features
- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings.
- The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities.
- Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system.

These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
2026-04-22 21:18:10 +08:00

11 KiB

WeKnora Wiki System - Complete Documentation

Welcome! This directory contains comprehensive documentation of the WeKnora wiki knowledge base system.

Quick Start

New to the wiki system? Start here:

  1. Read WIKI_SYSTEM_SUMMARY.md (5-10 minutes)

    • Understand what the wiki system does
    • Learn the key architecture concepts
    • Get familiar with common workflows
  2. Browse WIKI_DOCS_INDEX.md (2 minutes)

    • Find what you're looking for
    • Navigate between documents
    • Make decisions about which doc to read next
  3. Dive into specific docs

    • Implementing a feature? → WIKI_IMPLEMENTATION_GUIDE.md
    • Designing changes? → WIKI_TECHNICAL_ANALYSIS.md
    • Need complete reference? → WIKI_SYSTEM_DEEP_DIVE.md

Document Overview

Document Purpose Length Audience
WIKI_SYSTEM_SUMMARY.md Executive overview of wiki functionality 470 lines Everyone
WIKI_SYSTEM_DEEP_DIVE.md Comprehensive technical reference 950 lines Developers, Architects
WIKI_TECHNICAL_ANALYSIS.md Deep architectural analysis with edge cases 1,170 lines Advanced developers, Architects
WIKI_IMPLEMENTATION_GUIDE.md Practical how-to guide with code examples 750 lines Feature developers
WIKI_DOCS_INDEX.md Navigation and cross-references 480 lines Everyone
WIKI_README.md This file - First-time readers

Total: ~5,800 lines of documentation + ~7,300 lines of source code

Key Concepts at a Glance

What is Wiki?

A knowledge graph system that:

  • Extracts entities and concepts from uploaded documents
  • Synthesizes multi-document knowledge into canonical pages
  • Maintains bidirectional link references for graph traversal
  • Auto-links page content via pattern matching
  • Integrates with chat retrieval for enhanced results

Core Architecture

Document Upload
    ↓
Queue (Redis, 30s debounce)
    ↓
Async Batch Processing (MAP-REDUCE)
    ├─ Extract: LLM calls for entities/concepts
    ├─ Synthesize: Merge multi-document knowledge
    ├─ Post-process: Inject cross-links, rebuild index
    ↓
Database Storage + Retrieval Indexing
    ↓
Chat Integration (1.3x score boost for wiki pages)

Key Design Decisions

  1. Wiki as Feature Flag — Not a KB type, but an option on any KB
  2. Async with Debouncing — Batches documents for efficiency
  3. MAP-REDUCE Pattern — Parallel extraction and synthesis
  4. Bidirectional Links — Automatic link maintenance for graph queries
  5. Text-Based Injection — Pattern matching for auto-linking
  6. Optimistic Locking — Version-based conflict detection

FAQ

Q: Is wiki enabled for my knowledge base?
A: Check if kb.WikiConfig.Enabled is true. Use GetKnowledgeBaseByID to verify.

Q: Why is my page not showing in search?
A: Pages must have status=published. Check RunLint for health issues. Wiki page chunks need to be synchronized (should happen automatically).

Q: How long does wiki ingestion take?
A: Documents queue with a 30-second debounce. Typically 5-30 seconds per batch depending on LLM latency.

Q: Can I update a page while ingestion is running?
A: Yes, but you might get ErrWikiPageConflict if concurrent updates hit the same page. Retry with fresh version.

Q: How do I fix broken links?
A: Run WikiLintService.RunLint() to detect them. Use AutoFix() to repair automatically, or manually update page content.

Q: Can wiki pages link to each other?
A: Yes! Links are parsed from [[slug]] or [[slug|display text]] markdown syntax in page content.

Q: What's the difference between UpdatePage and UpdatePageMeta?
A: UpdatePage is for content changes and bumps version. UpdatePageMeta is for internal bookkeeping (links, status) without version bump.

Q: How does cross-link injection work?
A: System scans page content for matches of entity/concept names, wraps unlinked mentions as [[slug|name]]. Smart enough to skip code blocks and existing links.

Q: What if LLM extraction fails?
A: Document is logged as processed anyway. No automatic retry. Document can be re-uploaded if needed.

Q: How can I monitor wiki health?
A: Use WikiLintService.RunLint() which returns a health score (0-100), statistics, and detected issues.

Common Tasks

Check if Wiki is Enabled

kb, _ := kbService.GetKnowledgeBaseByID(ctx, kbID)
if kb.IsWikiEnabled() {
    // Wiki is active
}

Create a Wiki Page

page := &types.WikiPage{
    Slug:    "entity/acme-corp",
    Title:   "Acme Corporation",
    Content: "[[entity/bob]] founded Acme in 2020.",
    PageType: types.WikiPageTypeEntity,
}
created, _ := wikiService.CreatePage(ctx, page)

Get All Pages

pages, _ := wikiService.ListAllPages(ctx, kbID)
for _, p := range pages {
    fmt.Printf("%s: %s (%d inbound, %d outbound links)\n", 
        p.Slug, p.Title, len(p.InLinks), len(p.OutLinks))
}

Check Page Health

report, _ := lintService.RunLint(ctx, kbID)
fmt.Printf("Health Score: %d/100\n", report.HealthScore)
if report.HealthScore < 70 {
    fixed, _ := lintService.AutoFix(ctx, kbID)
    fmt.Printf("Fixed %d issues\n", fixed)
}

Update Page Content

page, _ := wikiService.GetPageBySlug(ctx, kbID, slug)
page.Content = newContent  // Update content
updated, err := wikiService.UpdatePage(ctx, page)
if err != nil {
    // Handle conflict retry
}

File Structure

WeKnora/
├── WIKI_README.md                               ← You are here
├── WIKI_DOCS_INDEX.md                           ← Navigation hub
├── WIKI_SYSTEM_SUMMARY.md                       ← Start here (5 min)
├── WIKI_SYSTEM_DEEP_DIVE.md                     ← Comprehensive reference
├── WIKI_TECHNICAL_ANALYSIS.md                   ← Deep architecture
├── WIKI_IMPLEMENTATION_GUIDE.md                 ← How-to guide
│
├── internal/types/
│   ├── wiki_page.go                             ← Data models
│   └── wiki_page_test.go                        ← Model tests
│
├── internal/application/service/
│   ├── wiki_page.go                             ← Core service (628 lines)
│   ├── wiki_ingest.go                           ← Async pipeline (1,053 lines)
│   ├── wiki_ingest_batch.go                     ← MAP-REDUCE handler (796 lines)
│   ├── wiki_linkify.go                          ← Cross-link injection (565 lines)
│   ├── wiki_lint.go                             ← Health checking (323 lines)
│   ├── wiki_page_test.go                        ← Service tests
│   ├── wiki_ingest_test.go                      ← Ingest tests
│   ├── wiki_linkify_test.go                     ← Linkify tests (285 lines)
│   ├── chat_pipeline/
│   │   └── wiki_boost.go                        ← Retrieval plugin
│   └── ...
│
├── internal/application/repository/
│   └── wiki_page.go                             ← Data access layer (357 lines)
│
├── internal/handler/
│   └── wiki_page.go                             ← HTTP endpoints (560 lines)
│
├── internal/agent/
│   ├── prompts_wiki.go                          ← LLM prompts (258 lines)
│   └── tools/
│       ├── wiki_write_page.go                   ← Create/update tool
│       ├── wiki_delete_page.go                  ← Delete tool
│       ├── wiki_replace_text.go                 ← Find/replace tool
│       └── ...
│
└── ... (other WeKnora files)

Reading Paths

Path 1: Quick Orientation (15 minutes)

  1. WIKI_README.md (this file) — 3 min
  2. WIKI_SYSTEM_SUMMARY.md — 5 min
  3. WIKI_DOCS_INDEX.md → choose next doc — 5 min

Path 2: Implementing a Feature (90 minutes)

  1. WIKI_SYSTEM_SUMMARY.md — 5 min
  2. WIKI_IMPLEMENTATION_GUIDE.md → relevant task — 20 min
  3. Source code reading (related service) — 30 min
  4. Write tests — 20 min
  5. Review WIKI_TECHNICAL_ANALYSIS.md for edge cases — 15 min

Path 3: Architectural Review (2-3 hours)

  1. WIKI_SYSTEM_SUMMARY.md — 10 min
  2. WIKI_SYSTEM_DEEP_DIVE.md (full) — 45 min
  3. WIKI_TECHNICAL_ANALYSIS.md (full) — 60 min
  4. Source code deep dive — 45 min

Path 4: Debugging (30-60 minutes)

  1. WIKI_TECHNICAL_ANALYSIS.md → Part 4 (Edge Cases) — 15 min
  2. WIKI_IMPLEMENTATION_GUIDE.md → Part 5 (Debugging) — 10 min
  3. SQL queries in appendix — 5 min
  4. Source code inspection — 20-30 min

Key Files to Know

Most Important:

  • wiki_page.go (service) — Core operations
  • wiki_ingest.go — Async orchestration
  • wiki_ingest_batch.go — Batch processor
  • wiki_linkify.go — Link injection algorithm

Important:

  • wiki_page.go (types) — Data structures
  • wiki_lint.go — Health checking
  • wiki_page.go (repository) — Data access
  • wiki_page.go (handler) — HTTP API

Reference:

  • prompts_wiki.go — LLM prompts
  • wiki_boost.go — Retrieval integration

Testing

All wiki functionality has tests:

  • Unit tests: wiki_linkify_test.go (285 lines, 13 test functions)
  • Service tests: wiki_page_test.go (100 lines)
  • Integration tests: wiki_ingest_test.go

Run tests:

go test ./internal/application/service/...
go test ./internal/application/repository/...
go test ./internal/types/...

Performance Notes

  • Debounce window: 30 seconds
  • Batch size: 5 documents per batch
  • Parallel goroutines: 10 (extraction), 10 (synthesis)
  • Lock TTL: 5 minutes
  • Pending operations TTL: 24 hours
  • Typical page size: 2-50 KB
  • Cross-link injection: O(n*m) where n=pages, m=slug count

Operational Monitoring

Key metrics to watch:

  • wiki:pending:{kbID} list length (pending documents)
  • wiki:active:{kbID} presence (processing running?)
  • Health score from RunLint (0-100)
  • Orphan page count (pages with no inbound links)
  • Broken link count (pages linking to nonexistent targets)

Getting Help

  1. API question? → WIKI_SYSTEM_DEEP_DIVE.md (API Endpoints section)
  2. Implementation question? → WIKI_IMPLEMENTATION_GUIDE.md
  3. Architecture question? → WIKI_TECHNICAL_ANALYSIS.md
  4. Can't find something? → WIKI_DOCS_INDEX.md (Navigation)
  5. Need to run SQL query? → WIKI_IMPLEMENTATION_GUIDE.md (Appendix C)

Contributing to Documentation

If you:

  • Find inaccuracies → update the doc
  • Find gaps → add examples or explanations
  • Find unclear sections → simplify or add context
  • Implement new features → document them here

Keep all docs in / (root directory) for easy access.


Last Updated: April 20, 2026
Total Documentation: ~5,800 lines
Total Source Code: ~7,300 lines (wiki-related files only)