Commit Graph

2 Commits

Author SHA1 Message Date
Claude 55bc5dcbe2 fix(chunking): post-review polish — error surfacing + doc accuracy
Three small follow-ups from the QA review of 13f57ca:

1. KBChunkingDebug error surfacing
   Previously a 200 OK response with { success: false, error: "..." }
   would have been swallowed under a generic "unexpected response shape"
   message. The strict-shape check now distinguishes empty response,
   success=false (surfaces resp.error directly), and missing data — so
   any future backend-side validation message reaches the user.

2. Token approximation in docs/CHUNKING.md
   Child Chunk Size default 384 ≈ 95 EN tokens (was rounded to 80).
   384 / 4 chars-per-token = 96; "~95" is the honest figure.

3. API surface in docs/CHUNKING.md
   The example only documented PUT /initialization/config/:kbId
   (camelCase, documentSplitting envelope). Added explicit notes that
   POST/PUT /knowledge-bases use snake_case under chunking_config, and
   that POST /chunker/preview also uses the snake_case form plus a text
   field. Readers picking the wrong endpoint won't be surprised by the
   case mismatch anymore.

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00
Claude 49adf0d73c docs(chunking): align overlap default + sharpen UI/source/repo docs
Three documentation passes around the adaptive chunking work:

UI
- Frontend ChunkOverlap default consolidated to 80 (was 100), matching
  chunker.DefaultChunkOverlap on the backend. Both DEFAULT_CHUNKING_PRESET
  and initFormData updated. The KB-load fallback also uses 80 when a
  loaded KB has no chunk_overlap stored.
- All four locales (en-US, zh-CN, ko-KR, ru-RU) get rewritten chunking
  setting descriptions: each now states the validated range, the default,
  and the situations where you'd deviate (FAQ vs narrative, embedder
  token limits, language-specific corpora).

Source code
- splitter.go: DefaultChunkSize / DefaultChunkOverlap constants get a
  longer block-comment explaining the per-language token math and the
  use-case sweet spots, plus the migration note on what the old
  inconsistent defaults were.
- KBChunkingSettings.vue: new comment block above ChunkingConfig
  documents the slider min/max for each setting, why those bounds
  exist, and the recommended TokenLimit values per embedding model.

Repo docs
- New docs/CHUNKING.md: end-to-end guide covering why chunking matters,
  the adaptive 3-tier architecture, per-setting reference with ranges
  and sweet spots, parent-child explanation, the token-limit table per
  embedder (OpenAI / Voyage / Cohere / BGE / MiniLM / Jina), 7 use-case
  presets, the debug panel workflow, the API surface, and known
  trade-offs (recursive strategy hidden from UI, no auto-reindex on
  strategy switch, OCR limitations).
- CHANGELOG.md gets a new [Unreleased] section consolidating all the
  adaptive-chunking work shipped on this branch: 5 features, 8
  improvements, 6 fixes, 1 docs entry. The entry references
  docs/CHUNKING.md for deeper explanation.

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00