Files
Enterprise-Knowledge-Base/WIKI_README.md
T
wizardchen 28deb8282f feat: Add comprehensive documentation for WeKnora knowledge base and wiki features
- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings.
- The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities.
- Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system.

These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
2026-04-22 21:18:10 +08:00

302 lines
11 KiB
Markdown

# WeKnora Wiki System - Complete Documentation
Welcome! This directory contains comprehensive documentation of the WeKnora wiki knowledge base system.
## Quick Start
**New to the wiki system?** Start here:
1. **Read WIKI_SYSTEM_SUMMARY.md** (5-10 minutes)
- Understand what the wiki system does
- Learn the key architecture concepts
- Get familiar with common workflows
2. **Browse WIKI_DOCS_INDEX.md** (2 minutes)
- Find what you're looking for
- Navigate between documents
- Make decisions about which doc to read next
3. **Dive into specific docs**
- **Implementing a feature?** → WIKI_IMPLEMENTATION_GUIDE.md
- **Designing changes?** → WIKI_TECHNICAL_ANALYSIS.md
- **Need complete reference?** → WIKI_SYSTEM_DEEP_DIVE.md
## Document Overview
| Document | Purpose | Length | Audience |
|----------|---------|--------|----------|
| **WIKI_SYSTEM_SUMMARY.md** | Executive overview of wiki functionality | 470 lines | Everyone |
| **WIKI_SYSTEM_DEEP_DIVE.md** | Comprehensive technical reference | 950 lines | Developers, Architects |
| **WIKI_TECHNICAL_ANALYSIS.md** | Deep architectural analysis with edge cases | 1,170 lines | Advanced developers, Architects |
| **WIKI_IMPLEMENTATION_GUIDE.md** | Practical how-to guide with code examples | 750 lines | Feature developers |
| **WIKI_DOCS_INDEX.md** | Navigation and cross-references | 480 lines | Everyone |
| **WIKI_README.md** | This file | - | First-time readers |
**Total:** ~5,800 lines of documentation + ~7,300 lines of source code
## Key Concepts at a Glance
### What is Wiki?
A knowledge graph system that:
- Extracts entities and concepts from uploaded documents
- Synthesizes multi-document knowledge into canonical pages
- Maintains bidirectional link references for graph traversal
- Auto-links page content via pattern matching
- Integrates with chat retrieval for enhanced results
### Core Architecture
```
Document Upload
↓
Queue (Redis, 30s debounce)
↓
Async Batch Processing (MAP-REDUCE)
├─ Extract: LLM calls for entities/concepts
├─ Synthesize: Merge multi-document knowledge
├─ Post-process: Inject cross-links, rebuild index
↓
Database Storage + Retrieval Indexing
↓
Chat Integration (1.3x score boost for wiki pages)
```
### Key Design Decisions
1. **Wiki as Feature Flag** — Not a KB type, but an option on any KB
2. **Async with Debouncing** — Batches documents for efficiency
3. **MAP-REDUCE Pattern** — Parallel extraction and synthesis
4. **Bidirectional Links** — Automatic link maintenance for graph queries
5. **Text-Based Injection** — Pattern matching for auto-linking
6. **Optimistic Locking** — Version-based conflict detection
## FAQ
**Q: Is wiki enabled for my knowledge base?**
A: Check if `kb.WikiConfig.Enabled` is true. Use `GetKnowledgeBaseByID` to verify.
**Q: Why is my page not showing in search?**
A: Pages must have status=`published`. Check `RunLint` for health issues. Wiki page chunks need to be synchronized (should happen automatically).
**Q: How long does wiki ingestion take?**
A: Documents queue with a 30-second debounce. Typically 5-30 seconds per batch depending on LLM latency.
**Q: Can I update a page while ingestion is running?**
A: Yes, but you might get `ErrWikiPageConflict` if concurrent updates hit the same page. Retry with fresh version.
**Q: How do I fix broken links?**
A: Run `WikiLintService.RunLint()` to detect them. Use `AutoFix()` to repair automatically, or manually update page content.
**Q: Can wiki pages link to each other?**
A: Yes! Links are parsed from `[[slug]]` or `[[slug|display text]]` markdown syntax in page content.
**Q: What's the difference between UpdatePage and UpdatePageMeta?**
A: `UpdatePage` is for content changes and bumps version. `UpdatePageMeta` is for internal bookkeeping (links, status) without version bump.
**Q: How does cross-link injection work?**
A: System scans page content for matches of entity/concept names, wraps unlinked mentions as `[[slug|name]]`. Smart enough to skip code blocks and existing links.
**Q: What if LLM extraction fails?**
A: Document is logged as processed anyway. No automatic retry. Document can be re-uploaded if needed.
**Q: How can I monitor wiki health?**
A: Use `WikiLintService.RunLint()` which returns a health score (0-100), statistics, and detected issues.
## Common Tasks
### Check if Wiki is Enabled
```go
kb, _ := kbService.GetKnowledgeBaseByID(ctx, kbID)
if kb.IsWikiEnabled() {
// Wiki is active
}
```
### Create a Wiki Page
```go
page := &types.WikiPage{
Slug: "entity/acme-corp",
Title: "Acme Corporation",
Content: "[[entity/bob]] founded Acme in 2020.",
PageType: types.WikiPageTypeEntity,
}
created, _ := wikiService.CreatePage(ctx, page)
```
### Get All Pages
```go
pages, _ := wikiService.ListAllPages(ctx, kbID)
for _, p := range pages {
fmt.Printf("%s: %s (%d inbound, %d outbound links)\n",
p.Slug, p.Title, len(p.InLinks), len(p.OutLinks))
}
```
### Check Page Health
```go
report, _ := lintService.RunLint(ctx, kbID)
fmt.Printf("Health Score: %d/100\n", report.HealthScore)
if report.HealthScore < 70 {
fixed, _ := lintService.AutoFix(ctx, kbID)
fmt.Printf("Fixed %d issues\n", fixed)
}
```
### Update Page Content
```go
page, _ := wikiService.GetPageBySlug(ctx, kbID, slug)
page.Content = newContent // Update content
updated, err := wikiService.UpdatePage(ctx, page)
if err != nil {
// Handle conflict retry
}
```
## File Structure
```
WeKnora/
├── WIKI_README.md ← You are here
├── WIKI_DOCS_INDEX.md ← Navigation hub
├── WIKI_SYSTEM_SUMMARY.md ← Start here (5 min)
├── WIKI_SYSTEM_DEEP_DIVE.md ← Comprehensive reference
├── WIKI_TECHNICAL_ANALYSIS.md ← Deep architecture
├── WIKI_IMPLEMENTATION_GUIDE.md ← How-to guide
│
├── internal/types/
│ ├── wiki_page.go ← Data models
│ └── wiki_page_test.go ← Model tests
│
├── internal/application/service/
│ ├── wiki_page.go ← Core service (628 lines)
│ ├── wiki_ingest.go ← Async pipeline (1,053 lines)
│ ├── wiki_ingest_batch.go ← MAP-REDUCE handler (796 lines)
│ ├── wiki_linkify.go ← Cross-link injection (565 lines)
│ ├── wiki_lint.go ← Health checking (323 lines)
│ ├── wiki_page_test.go ← Service tests
│ ├── wiki_ingest_test.go ← Ingest tests
│ ├── wiki_linkify_test.go ← Linkify tests (285 lines)
│ ├── chat_pipeline/
│ │ └── wiki_boost.go ← Retrieval plugin
│ └── ...
│
├── internal/application/repository/
│ └── wiki_page.go ← Data access layer (357 lines)
│
├── internal/handler/
│ └── wiki_page.go ← HTTP endpoints (560 lines)
│
├── internal/agent/
│ ├── prompts_wiki.go ← LLM prompts (258 lines)
│ └── tools/
│ ├── wiki_write_page.go ← Create/update tool
│ ├── wiki_delete_page.go ← Delete tool
│ ├── wiki_replace_text.go ← Find/replace tool
│ └── ...
│
└── ... (other WeKnora files)
```
## Reading Paths
### Path 1: Quick Orientation (15 minutes)
1. WIKI_README.md (this file) — 3 min
2. WIKI_SYSTEM_SUMMARY.md — 5 min
3. WIKI_DOCS_INDEX.md → choose next doc — 5 min
### Path 2: Implementing a Feature (90 minutes)
1. WIKI_SYSTEM_SUMMARY.md — 5 min
2. WIKI_IMPLEMENTATION_GUIDE.md → relevant task — 20 min
3. Source code reading (related service) — 30 min
4. Write tests — 20 min
5. Review WIKI_TECHNICAL_ANALYSIS.md for edge cases — 15 min
### Path 3: Architectural Review (2-3 hours)
1. WIKI_SYSTEM_SUMMARY.md — 10 min
2. WIKI_SYSTEM_DEEP_DIVE.md (full) — 45 min
3. WIKI_TECHNICAL_ANALYSIS.md (full) — 60 min
4. Source code deep dive — 45 min
### Path 4: Debugging (30-60 minutes)
1. WIKI_TECHNICAL_ANALYSIS.md → Part 4 (Edge Cases) — 15 min
2. WIKI_IMPLEMENTATION_GUIDE.md → Part 5 (Debugging) — 10 min
3. SQL queries in appendix — 5 min
4. Source code inspection — 20-30 min
## Key Files to Know
**Most Important:**
- `wiki_page.go` (service) — Core operations
- `wiki_ingest.go` — Async orchestration
- `wiki_ingest_batch.go` — Batch processor
- `wiki_linkify.go` — Link injection algorithm
**Important:**
- `wiki_page.go` (types) — Data structures
- `wiki_lint.go` — Health checking
- `wiki_page.go` (repository) — Data access
- `wiki_page.go` (handler) — HTTP API
**Reference:**
- `prompts_wiki.go` — LLM prompts
- `wiki_boost.go` — Retrieval integration
## Testing
All wiki functionality has tests:
- Unit tests: `wiki_linkify_test.go` (285 lines, 13 test functions)
- Service tests: `wiki_page_test.go` (100 lines)
- Integration tests: `wiki_ingest_test.go`
Run tests:
```bash
go test ./internal/application/service/...
go test ./internal/application/repository/...
go test ./internal/types/...
```
## Performance Notes
- **Debounce window:** 30 seconds
- **Batch size:** 5 documents per batch
- **Parallel goroutines:** 10 (extraction), 10 (synthesis)
- **Lock TTL:** 5 minutes
- **Pending operations TTL:** 24 hours
- **Typical page size:** 2-50 KB
- **Cross-link injection:** O(n*m) where n=pages, m=slug count
## Operational Monitoring
Key metrics to watch:
- `wiki:pending:{kbID}` list length (pending documents)
- `wiki:active:{kbID}` presence (processing running?)
- Health score from `RunLint` (0-100)
- Orphan page count (pages with no inbound links)
- Broken link count (pages linking to nonexistent targets)
## Getting Help
1. **API question?** → WIKI_SYSTEM_DEEP_DIVE.md (API Endpoints section)
2. **Implementation question?** → WIKI_IMPLEMENTATION_GUIDE.md
3. **Architecture question?** → WIKI_TECHNICAL_ANALYSIS.md
4. **Can't find something?** → WIKI_DOCS_INDEX.md (Navigation)
5. **Need to run SQL query?** → WIKI_IMPLEMENTATION_GUIDE.md (Appendix C)
## Contributing to Documentation
If you:
- Find inaccuracies → update the doc
- Find gaps → add examples or explanations
- Find unclear sections → simplify or add context
- Implement new features → document them here
Keep all docs in `/` (root directory) for easy access.
---
**Last Updated:** April 20, 2026
**Total Documentation:** ~5,800 lines
**Total Source Code:** ~7,300 lines (wiki-related files only)