- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings. - The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities. - Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system. These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
11 KiB
WeKnora Wiki System - Complete Documentation
Welcome! This directory contains comprehensive documentation of the WeKnora wiki knowledge base system.
Quick Start
New to the wiki system? Start here:
-
Read WIKI_SYSTEM_SUMMARY.md (5-10 minutes)
- Understand what the wiki system does
- Learn the key architecture concepts
- Get familiar with common workflows
-
Browse WIKI_DOCS_INDEX.md (2 minutes)
- Find what you're looking for
- Navigate between documents
- Make decisions about which doc to read next
-
Dive into specific docs
- Implementing a feature? → WIKI_IMPLEMENTATION_GUIDE.md
- Designing changes? → WIKI_TECHNICAL_ANALYSIS.md
- Need complete reference? → WIKI_SYSTEM_DEEP_DIVE.md
Document Overview
| Document | Purpose | Length | Audience |
|---|---|---|---|
| WIKI_SYSTEM_SUMMARY.md | Executive overview of wiki functionality | 470 lines | Everyone |
| WIKI_SYSTEM_DEEP_DIVE.md | Comprehensive technical reference | 950 lines | Developers, Architects |
| WIKI_TECHNICAL_ANALYSIS.md | Deep architectural analysis with edge cases | 1,170 lines | Advanced developers, Architects |
| WIKI_IMPLEMENTATION_GUIDE.md | Practical how-to guide with code examples | 750 lines | Feature developers |
| WIKI_DOCS_INDEX.md | Navigation and cross-references | 480 lines | Everyone |
| WIKI_README.md | This file | - | First-time readers |
Total: ~5,800 lines of documentation + ~7,300 lines of source code
Key Concepts at a Glance
What is Wiki?
A knowledge graph system that:
- Extracts entities and concepts from uploaded documents
- Synthesizes multi-document knowledge into canonical pages
- Maintains bidirectional link references for graph traversal
- Auto-links page content via pattern matching
- Integrates with chat retrieval for enhanced results
Core Architecture
Document Upload
↓
Queue (Redis, 30s debounce)
↓
Async Batch Processing (MAP-REDUCE)
├─ Extract: LLM calls for entities/concepts
├─ Synthesize: Merge multi-document knowledge
├─ Post-process: Inject cross-links, rebuild index
↓
Database Storage + Retrieval Indexing
↓
Chat Integration (1.3x score boost for wiki pages)
Key Design Decisions
- Wiki as Feature Flag — Not a KB type, but an option on any KB
- Async with Debouncing — Batches documents for efficiency
- MAP-REDUCE Pattern — Parallel extraction and synthesis
- Bidirectional Links — Automatic link maintenance for graph queries
- Text-Based Injection — Pattern matching for auto-linking
- Optimistic Locking — Version-based conflict detection
FAQ
Q: Is wiki enabled for my knowledge base?
A: Check if kb.WikiConfig.Enabled is true. Use GetKnowledgeBaseByID to verify.
Q: Why is my page not showing in search?
A: Pages must have status=published. Check RunLint for health issues. Wiki page chunks need to be synchronized (should happen automatically).
Q: How long does wiki ingestion take?
A: Documents queue with a 30-second debounce. Typically 5-30 seconds per batch depending on LLM latency.
Q: Can I update a page while ingestion is running?
A: Yes, but you might get ErrWikiPageConflict if concurrent updates hit the same page. Retry with fresh version.
Q: How do I fix broken links?
A: Run WikiLintService.RunLint() to detect them. Use AutoFix() to repair automatically, or manually update page content.
Q: Can wiki pages link to each other?
A: Yes! Links are parsed from [[slug]] or [[slug|display text]] markdown syntax in page content.
Q: What's the difference between UpdatePage and UpdatePageMeta?
A: UpdatePage is for content changes and bumps version. UpdatePageMeta is for internal bookkeeping (links, status) without version bump.
Q: How does cross-link injection work?
A: System scans page content for matches of entity/concept names, wraps unlinked mentions as [[slug|name]]. Smart enough to skip code blocks and existing links.
Q: What if LLM extraction fails?
A: Document is logged as processed anyway. No automatic retry. Document can be re-uploaded if needed.
Q: How can I monitor wiki health?
A: Use WikiLintService.RunLint() which returns a health score (0-100), statistics, and detected issues.
Common Tasks
Check if Wiki is Enabled
kb, _ := kbService.GetKnowledgeBaseByID(ctx, kbID)
if kb.IsWikiEnabled() {
// Wiki is active
}
Create a Wiki Page
page := &types.WikiPage{
Slug: "entity/acme-corp",
Title: "Acme Corporation",
Content: "[[entity/bob]] founded Acme in 2020.",
PageType: types.WikiPageTypeEntity,
}
created, _ := wikiService.CreatePage(ctx, page)
Get All Pages
pages, _ := wikiService.ListAllPages(ctx, kbID)
for _, p := range pages {
fmt.Printf("%s: %s (%d inbound, %d outbound links)\n",
p.Slug, p.Title, len(p.InLinks), len(p.OutLinks))
}
Check Page Health
report, _ := lintService.RunLint(ctx, kbID)
fmt.Printf("Health Score: %d/100\n", report.HealthScore)
if report.HealthScore < 70 {
fixed, _ := lintService.AutoFix(ctx, kbID)
fmt.Printf("Fixed %d issues\n", fixed)
}
Update Page Content
page, _ := wikiService.GetPageBySlug(ctx, kbID, slug)
page.Content = newContent // Update content
updated, err := wikiService.UpdatePage(ctx, page)
if err != nil {
// Handle conflict retry
}
File Structure
WeKnora/
├── WIKI_README.md ← You are here
├── WIKI_DOCS_INDEX.md ← Navigation hub
├── WIKI_SYSTEM_SUMMARY.md ← Start here (5 min)
├── WIKI_SYSTEM_DEEP_DIVE.md ← Comprehensive reference
├── WIKI_TECHNICAL_ANALYSIS.md ← Deep architecture
├── WIKI_IMPLEMENTATION_GUIDE.md ← How-to guide
│
├── internal/types/
│ ├── wiki_page.go ← Data models
│ └── wiki_page_test.go ← Model tests
│
├── internal/application/service/
│ ├── wiki_page.go ← Core service (628 lines)
│ ├── wiki_ingest.go ← Async pipeline (1,053 lines)
│ ├── wiki_ingest_batch.go ← MAP-REDUCE handler (796 lines)
│ ├── wiki_linkify.go ← Cross-link injection (565 lines)
│ ├── wiki_lint.go ← Health checking (323 lines)
│ ├── wiki_page_test.go ← Service tests
│ ├── wiki_ingest_test.go ← Ingest tests
│ ├── wiki_linkify_test.go ← Linkify tests (285 lines)
│ ├── chat_pipeline/
│ │ └── wiki_boost.go ← Retrieval plugin
│ └── ...
│
├── internal/application/repository/
│ └── wiki_page.go ← Data access layer (357 lines)
│
├── internal/handler/
│ └── wiki_page.go ← HTTP endpoints (560 lines)
│
├── internal/agent/
│ ├── prompts_wiki.go ← LLM prompts (258 lines)
│ └── tools/
│ ├── wiki_write_page.go ← Create/update tool
│ ├── wiki_delete_page.go ← Delete tool
│ ├── wiki_replace_text.go ← Find/replace tool
│ └── ...
│
└── ... (other WeKnora files)
Reading Paths
Path 1: Quick Orientation (15 minutes)
- WIKI_README.md (this file) — 3 min
- WIKI_SYSTEM_SUMMARY.md — 5 min
- WIKI_DOCS_INDEX.md → choose next doc — 5 min
Path 2: Implementing a Feature (90 minutes)
- WIKI_SYSTEM_SUMMARY.md — 5 min
- WIKI_IMPLEMENTATION_GUIDE.md → relevant task — 20 min
- Source code reading (related service) — 30 min
- Write tests — 20 min
- Review WIKI_TECHNICAL_ANALYSIS.md for edge cases — 15 min
Path 3: Architectural Review (2-3 hours)
- WIKI_SYSTEM_SUMMARY.md — 10 min
- WIKI_SYSTEM_DEEP_DIVE.md (full) — 45 min
- WIKI_TECHNICAL_ANALYSIS.md (full) — 60 min
- Source code deep dive — 45 min
Path 4: Debugging (30-60 minutes)
- WIKI_TECHNICAL_ANALYSIS.md → Part 4 (Edge Cases) — 15 min
- WIKI_IMPLEMENTATION_GUIDE.md → Part 5 (Debugging) — 10 min
- SQL queries in appendix — 5 min
- Source code inspection — 20-30 min
Key Files to Know
Most Important:
wiki_page.go(service) — Core operationswiki_ingest.go— Async orchestrationwiki_ingest_batch.go— Batch processorwiki_linkify.go— Link injection algorithm
Important:
wiki_page.go(types) — Data structureswiki_lint.go— Health checkingwiki_page.go(repository) — Data accesswiki_page.go(handler) — HTTP API
Reference:
prompts_wiki.go— LLM promptswiki_boost.go— Retrieval integration
Testing
All wiki functionality has tests:
- Unit tests:
wiki_linkify_test.go(285 lines, 13 test functions) - Service tests:
wiki_page_test.go(100 lines) - Integration tests:
wiki_ingest_test.go
Run tests:
go test ./internal/application/service/...
go test ./internal/application/repository/...
go test ./internal/types/...
Performance Notes
- Debounce window: 30 seconds
- Batch size: 5 documents per batch
- Parallel goroutines: 10 (extraction), 10 (synthesis)
- Lock TTL: 5 minutes
- Pending operations TTL: 24 hours
- Typical page size: 2-50 KB
- Cross-link injection: O(n*m) where n=pages, m=slug count
Operational Monitoring
Key metrics to watch:
wiki:pending:{kbID}list length (pending documents)wiki:active:{kbID}presence (processing running?)- Health score from
RunLint(0-100) - Orphan page count (pages with no inbound links)
- Broken link count (pages linking to nonexistent targets)
Getting Help
- API question? → WIKI_SYSTEM_DEEP_DIVE.md (API Endpoints section)
- Implementation question? → WIKI_IMPLEMENTATION_GUIDE.md
- Architecture question? → WIKI_TECHNICAL_ANALYSIS.md
- Can't find something? → WIKI_DOCS_INDEX.md (Navigation)
- Need to run SQL query? → WIKI_IMPLEMENTATION_GUIDE.md (Appendix C)
Contributing to Documentation
If you:
- Find inaccuracies → update the doc
- Find gaps → add examples or explanations
- Find unclear sections → simplify or add context
- Implement new features → document them here
Keep all docs in / (root directory) for easy access.
Last Updated: April 20, 2026
Total Documentation: ~5,800 lines
Total Source Code: ~7,300 lines (wiki-related files only)