Commit Graph

7 Commits

Author SHA1 Message Date
wizardchen f92302cc64 fix(knowledge): stitch chunks by text overlap, not by position
Reconstructing full document content from chunks relied on position math
(offset = content length - (EndAt - lastEndAt)) that assumes
len([]rune(Content)) == EndAt - StartAt. Two upstream behaviors break that
invariant and caused the merged "全文" view to drop or duplicate text:

- The parent-child chunker prepends a synthetic table header to split
  tables; that header is zero-width in position space (start == end), so
  Content is longer than EndAt - StartAt.
- Chunk Content can retain HTML entities (", > ...), inflating the
  rune count relative to the source span.

Replace position-based trimming with text-overlap matching: find where the
accumulated text's suffix reappears at the head of the next chunk and join
there. Positions are only used to size the search window. This naturally
skips the prepended header and is immune to entity length drift.

Extract the logic into searchutil.AppendWithOverlap / MergeTextChunks and
reuse it across the three backend stitching sites (wiki_ingest
reconstructContent, graph mergeChunkContents, chat_pipeline
mergeOverlappingChunks). Mirror the same algorithm in the frontend
doc-content merge. Also fix the markdown hr/heading styles (thin solid
divider, primary heading color).
2026-05-30 17:53:47 +08:00
wizardchen 59d2f6b54c feat(graph): enhance graph extraction prompt rendering and language context handling
- Added a new method to render graph extraction prompts with shared placeholders for language.
- Updated entity and relationship extraction methods to utilize the new rendering function, improving prompt customization.
- Enhanced question generation task to include language context, ensuring prompts are generated in the correct language.
- Improved error handling for empty prompt configurations in question generation, enhancing robustness.
2026-03-25 22:08:29 +08:00
ochan.kwon edcf7116cd feat: use language-neutral English prompts for multilingual LLM responses
Convert all LLM-facing prompts, templates, and tool output labels to
language-neutral English so the model responds in the user's input
language. Update frontend relevance/match-type mappings to match.

Output language strategy:
- User-facing: reply in user's input language
- Vector-indexed: match source document language
- LLM-internal: fixed English
2026-03-06 21:08:12 +08:00
wizardchen 135e33eca4 refactor: Clean up code by removing unnecessary blank lines and improving comments for better readability and maintainability 2025-11-28 16:10:37 +08:00
wizardchen 2f45bf7d86 refactor: Improve Elasticsearch query construction and error handling 2025-11-25 22:06:43 +08:00
wizardchen 64c44c9e27 refactor: Update tenant ID handling across the application 2025-11-25 16:30:54 +08:00
wizardchen 56eb2bce33 init commit 2025-08-05 15:08:07 +08:00