Files
Enterprise-Knowledge-Base/KB_WIKI_FILE_INDEX.md
T
wizardchen 28deb8282f feat: Add comprehensive documentation for WeKnora knowledge base and wiki features
- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings.
- The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities.
- Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system.

These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
2026-04-22 21:18:10 +08:00

14 KiB

Knowledge Base & Wiki System - File Index & Cross-Reference

COMPLETE FILE LISTING BY COMPONENT

1. DATA MODELS (internal/types/)

File Purpose Key Types
knowledgebase.go KB data model KnowledgeBase, WikiConfig, ChunkingConfig, VLMConfig, ASRConfig, ExtractConfig, FAQConfig
knowledge.go Document/knowledge model Knowledge, ManualKnowledgeMetadata, Knowledge.Type (manual, faq)
wiki_page.go Wiki page model WikiPage, WikiPageIssue, WikiConfig, WikiStats, WikiGraphData
agent.go Agent configuration AgentConfig, SessionAgentConfig (includes KnowledgeBases)
chunk.go Chunk model Indexed content units (not detailed in search but referenced)
message.go Chat messages Not detailed - used by sessions
tag.go Tags/categories For knowledge organization

2. INTERFACES (internal/types/interfaces/)

File Purpose Contracts
knowledgebase.go KB service interface KnowledgeBaseService, KnowledgeBaseRepository
knowledge.go Knowledge service interface KnowledgeService, KnowledgeRepository
wiki_page.go Wiki service interface WikiPageService, WikiPageRepository
agent.go Agent tool interface Generic tool registration
chunk.go Chunk access interface ChunkRepository, ChunkService

3. SERVICES (internal/application/service/)

File Purpose Methods
knowledgebase.go KB business logic Create, Update, Get, List, HybridSearch
knowledge.go Doc ingest pipeline CreateKnowledge, UpdateKnowledge, Delete, Search
wiki_ingest.go Wiki LLM generation WikiIngestService, ProcessWikiIngest, ScheduleFollowUp
wiki_ingest_batch.go Wiki batch processing Batch LLM calls, link rebuilding
wiki_page.go Wiki CRUD operations CreatePage, UpdatePage, GetPageBySlug, ListPages, RebuildLinks
wiki_linkify.go Wiki link parsing Parse wiki-links syntax
wiki_lint.go Wiki quality checks Validate, find broken links, orphaned pages
knowledge_post_process.go Post-processing docs Summary generation, extraction
chat_pipeline/search.go Retrieval in chat Knowledge search during chat
chat_pipeline/wiki_boost.go Wiki integration in chat Boost wiki results in chat context
chunk.go Chunk service Chunk CRUD, retrieval
retriever/keywords_vector_hybrid_indexer.go Hybrid search engine Index, BatchIndex, Retrieve

4. REPOSITORIES (internal/application/repository/)

File Purpose Storage
knowledgebase.go KB persistence GORM: knowledge_bases table
knowledge.go Knowledge persistence GORM: knowledges table
wiki_page.go Wiki persistence GORM: wiki_pages, wiki_page_issues
chunk.go Chunk persistence GORM: chunks table
chunk_sqlite_test.go SQLite chunk backend For testing
retriever/weaviate/repository.go Weaviate backend Vector search
retriever/elasticsearch/v7/repository.go Elasticsearch v7 Vector + keyword
retriever/elasticsearch/v8/repository.go Elasticsearch v8 Vector + keyword
retriever/qdrant/repository.go Qdrant backend Vector search
retriever/milvus/repository.go Milvus backend Vector search
retriever/postgres/repository.go PostgreSQL backend Vector search with pgvector
retriever/sqlite/repository.go SQLite backend Local vector search
tag.go Tag persistence GORM: tags table

5. HTTP HANDLERS (internal/handler/)

File Purpose Endpoints
knowledgebase.go KB HTTP API POST/PUT/GET/DELETE /api/v1/knowledge-bases
wiki_page.go Wiki HTTP API GET/POST/PUT/DELETE /api/v1/knowledgebase/{id}/wiki/*
knowledge.go Knowledge HTTP API File upload, URL add, manual create
chunk.go Chunk HTTP API Chunk retrieval, preview
session/qa.go Session chat QA Knowledge search in chat context

6. ROUTING (internal/router/)

File Purpose Route Registration
router.go Main router RegisterWikiPageRoutes (~line 839)
task.go Task routing Async task registration

7. AGENT TOOLS (internal/agent/tools/)

File Purpose Tool Name
wiki_write_page.go Agent creates page wiki_write_page
wiki_read_source_doc.go Agent reads original doc wiki_read_source_doc
wiki_tools.go Tool registration WikiToolRegistry
wiki_delete_page.go Agent deletes page wiki_delete_page
wiki_replace_text.go Agent edits page wiki_replace_text
wiki_rename_page.go Agent renames page wiki_rename_page
wiki_update_issue.go Agent resolves issue wiki_update_issue
wiki_read_issue.go Agent checks issue wiki_read_issue
wiki_flag_issue.go Agent flags problem wiki_flag_issue
knowledge_search.go Agent searches KB knowledge_search
grep_chunks.go Agent regex search grep_chunks
list_knowledge_chunks.go Agent lists docs list_knowledge_chunks
get_document_info.go Agent gets doc info get_document_info
web_search.go Agent web search web_search
definitions.go Tool definitions Tool registry

8. AGENT CONFIG (internal/agent/)

File Purpose Content
prompts_wiki.go Wiki-specific prompts Agent instructions for wiki tools
prompts.go General prompts System prompt templates
engine.go Agent loop ReAct execution engine
observe.go Agent observation Tool result interpretation

9. ASYNC TASKS (internal/types/ & services)

Task Type File Handler Payload
TypeWikiIngest wiki_ingest.go ProcessWikiIngest WikiIngestPayload
(Knowledge ingest) knowledge.go Various KnowledgePayload

10. FRONTEND (frontend/src/)

File Purpose Exports
api/knowledge-base/index.ts KB API client createKnowledgeBase, updateKnowledgeBase, getKnowledgeBaseById, etc.
api/wiki/index.ts Wiki API client listWikiPages, getWikiPage, updateWikiPage, etc.
stores/knowledge.ts State management Knowledge store (Pinia)
stores/settings.ts Settings store App settings
hooks/useKnowledgeBase.ts KB hook KB operations hook
hooks/useKnowledgeBaseCreationNavigation.ts KB creation Navigation after KB creation

KEY DATA FLOWS BY FILE

Document Ingestion Flow

1. frontend/src/api/knowledge-base/index.ts
   → uploadKnowledgeFile(kbId, file)

2. internal/handler/knowledge.go
   → KnowledgeHandler.CreateKnowledge()

3. internal/application/service/knowledge.go
   → KnowledgeService.CreateKnowledge()
   → Creates Knowledge entity (ParseStatus=pending)
   → Enqueues ingest task

4. Async task processed:
   → Chunks created (chunks table)
   → Embeddings indexed (vector DB: Weaviate, Elasticsearch, etc.)
   → IF WikiConfig.AutoIngest:
      → internal/application/service/wiki_ingest.go
      → Enqueue WikiIngest task (30s delay)

5. Wiki ingest task:
   → internal/application/service/wiki_ingest_batch.go
   → ProcessWikiIngest()
   → LLM generates pages
   → internal/application/service/wiki_page.go
   → CreatePage() for each generated wiki page
   → internal/application/repository/wiki_page.go
   → Save to wiki_pages table

6. Link maintenance:
   → internal/application/service/wiki_linkify.go
   → Parse [[wiki-links]]
   → internal/application/service/wiki_page.go
   → RebuildLinks() updates InLinks/OutLinks

Settings Update Flow

1. frontend/src/api/knowledge-base/index.ts
   → updateKnowledgeBase(id, config)

2. internal/handler/knowledgebase.go
   → KnowledgeBaseHandler.UpdateKnowledgeBase()
   → Validates config

3. internal/application/service/knowledgebase.go
   → KnowledgeBaseService.UpdateKnowledgeBase()
   → Merges config (including wiki_config)
   → Calls EnsureDefaults()

4. internal/application/repository/knowledgebase.go
   → KnowledgeBaseRepository.UpdateKnowledgeBase()
   → UPDATE knowledge_bases SET wiki_config = JSON_VALUE

Agent Using Wiki

1. Session starts:
   → internal/handler/session/handler.go
   → Load session with agent_config

2. Agent receives query:
   → internal/types/agent.go
   → AgentConfig.KnowledgeBases → resolved to KB objects
   → For each wiki-enabled KB:
      → Add wiki tools to available tools

3. Agent calls wiki_read_page:
   → internal/agent/tools/wiki_tools.go
   → Call WikiPageService.GetPageBySlug()
   → internal/application/service/wiki_page.go
   → internal/application/repository/wiki_page.go
   → SELECT FROM wiki_pages WHERE knowledge_base_id = ? AND slug = ?

4. Agent calls wiki_write_page:
   → internal/agent/tools/wiki_write_page.go
   → Call WikiPageService.CreatePage()
   → Same flow as #5-6 in document ingestion

DIRECT FILE RELATIONSHIPS

KnowledgeBase ↔ WikiConfig

internal/types/knowledgebase.go::KnowledgeBase
    └─ WikiConfig *types.WikiConfig
       └─ Stored in knowledge_bases.wiki_config (JSON)

Update path:
    PUT handler (knowledgebase.go)
    → Service (knowledgebase.go)
    → Repository (knowledgebase.go)
    → Database: UPDATE knowledge_bases SET wiki_config = ?

Knowledge ↔ Chunks

internal/types/knowledge.go::Knowledge
    └─ FK: knowledge_base_id

internal/types/chunk.go::Chunk
    └─ FK: knowledge_id → knowledge.id
    └─ Indexed in multiple backends:
        ├─ elasticsearch/v7/repository.go
        ├─ elasticsearch/v8/repository.go
        ├─ weaviate/repository.go
        ├─ qdrant/repository.go
        ├─ milvus/repository.go
        ├─ postgres/repository.go
        └─ sqlite/repository.go

WikiPage ↔ KnowledgeBase

internal/types/wiki_page.go::WikiPage
    └─ FK: knowledge_base_id
    └─ Unique constraint: (knowledge_base_id, slug)
    └─ SourceRefs: knowledge IDs that contributed

Agent ↔ KnowledgeBase

internal/types/agent.go::AgentConfig
    └─ KnowledgeBases []string (KB IDs)
    └─ Resolved at runtime to KnowledgeBase objects
    └─ Tool selection based on KB.IsWikiEnabled()

SERVICE DEPENDENCY INJECTION

Container (internal/container/container.go)

Provides instances of all services:
    - KnowledgeBaseService
    - KnowledgeService
    - WikiPageService
    - WikiLintService
    - Retrieval engines
    - Repositories
    - HTTP handlers

Service Constructor Dependencies

KnowledgeBaseService:

repo KnowledgeBaseRepository
kgRepo KnowledgeRepository
chunkRepo ChunkRepository
shareRepo KBShareRepository
kbShareService KBShareService
modelService ModelService
retrieveEngine RetrieveEngineRegistry
tenantRepo TenantRepository
fileSvc FileService
graphEngine RetrieveGraphRepository
asynqClient TaskEnqueuer

WikiPageService:

repo WikiPageRepository
chunkRepo ChunkRepository
kbService KnowledgeBaseService
redisClient *redis.Client

WikiIngestService:

wikiService WikiPageService
kbService KnowledgeBaseService
knowledgeSvc KnowledgeService
chunkRepo ChunkRepository
modelService ModelService
task TaskEnqueuer
redisClient *redis.Client

CONFIGURATION FILES & ENVIRONMENT

Backend Config (internal/config/config.go)

Reads from:
    - .env file
    - Environment variables
    - YAML config files

Key config sections:
    - Database (PostgreSQL, MySQL, SQLite)
    - Redis (for async tasks)
    - Retrieval engines (Elasticsearch, Weaviate, etc.)
    - LLM models (OpenAI, Claude, local, etc.)
    - Storage (local, S3, Minio, etc.)

Environment Variables (.env)

# Database
DATABASE_URL=...

# Redis
REDIS_URL=...

# LLM
LLM_API_KEY=...
LLM_MODEL_ID=...

# Vector DB
WEAVIATE_URL=...
ELASTICSEARCH_URL=...

# Storage
STORAGE_PROVIDER=...

BUILD & DEPLOYMENT

Binary

  • server - Main Go binary (compiled from cmd/desktop/main.go)
  • WeKnora - Desktop app version

Docker

  • Dockerfile - Backend container
  • frontend/Dockerfile - Frontend container
  • docker-compose.yml - Full stack

Kubernetes/Helm

  • helm/ directory - Helm charts

SUMMARY TABLE

Component Count Key Files
Models 7+ types/*.go
Services 10+ service/*.go
Repositories 10+ repository/*.go
Handlers 5+ handler/*.go
Agent Tools 13+ tools/wiki_.go, knowledge_.go
Retrieval Backends 6 retriever/*/repository.go
Frontend APIs 2 api/knowledge-base/, api/wiki/
Interfaces 5+ types/interfaces/*.go

QUICK FILE REFERENCE BY QUESTION

Q: Where is WikiConfig defined? A: internal/types/knowledgebase.go (line ~94-103)

Q: Where is wiki auto-ingest logic? A: internal/application/service/wiki_ingest.go and wiki_ingest_batch.go

Q: How are KB settings persisted? A: internal/handler/knowledgebase.go → service/knowledgebase.go → repository/knowledgebase.go

Q: Where do agents access KBs? A: internal/types/agent.go (AgentConfig.KnowledgeBases) → resolved in service layer

Q: Where is wiki page model? A: internal/types/wiki_page.go (WikiPage struct, ~line 43-84)

Q: How are wiki links maintained? A: internal/application/service/wiki_linkify.go and wiki_page.go::RebuildLinks

Q: Where is hybrid retrieval implemented? A: internal/application/service/retriever/keywords_vector_hybrid_indexer.go

Q: How does chunk embedding happen? A: Multiple backends in retriever/*/repository.go (Elasticsearch, Weaviate, etc.)

Q: Frontend API for KB updates? A: frontend/src/api/knowledge-base/index.ts::updateKnowledgeBase()

Q: Where is async task handling? A: Asynq queue, triggered in service layer, workers in wiki_ingest.go