Commit Graph

83 Commits

Author SHA1 Message Date
wizardchen 20b01a6cd5 feat(embed): session sidebar grouping and embed theme alignment (#1657)
Group embed sessions in the sidebar by channel, replace session source filter
with grouping buckets, and align shared chat components with embed theme colors.
2026-06-12 23:10:39 +08:00
langcaiye d832dae818 fix(datasource): clarify edit does not sync 2026-06-12 22:49:37 +08:00
wizardchen ed9e780c8c feat(client): enhance knowledge file upload with process configuration support
- Updated the `CreateKnowledgeFromFile` function to accept an optional `processConfig` parameter for per-upload parse configuration overrides.
- Introduced the `KnowledgeProcessOverrides` type to define various parsing settings, including parser engine rules and chunking configurations.
- Modified the API client and related components to handle the new process configuration, improving flexibility in knowledge file uploads.
- Updated documentation and API specifications to reflect the new `process_config` parameter in the upload process.
- Added a new `UploadConfirmHost` component to facilitate user confirmation for uploads, integrating with the updated upload logic.
- Enhanced user experience by providing detailed configuration options during the upload process.
2026-06-09 20:38:24 +08:00
wizardchen 815c72c964 refactor(frontend): streamline knowledge base and agent handling with chat resources store
- Updated components to utilize the new `useChatResourcesStore` for managing knowledge bases and agents, improving data fetching and caching mechanisms.
- Refactored `AgentSelector`, `IMChannelPanel`, `Input-field`, and `menu` components to replace direct API calls with store methods, enhancing maintainability and performance.
- Adjusted knowledge base loading logic to ensure consistency across components and reduce redundant API requests.
- Improved session state handling in `settings` store to prevent overwriting user selections during state restoration.
2026-06-09 18:05:33 +08:00
langcaiye 3c2ee171cb fix: skip credential validation when editing data source 2026-06-04 15:29:22 +08:00
wizardchen f4af9cca97 feat(contextualGuides): add multiple contextual guide components for agent and knowledge base creation
- Introduced new components: AgentCreateContextualGuide, KbCreateContextualGuide, TenantModelsGuide, and SpotlightGuide to enhance user onboarding.
- Implemented dynamic step configurations for each guide, allowing tailored user experiences based on context.
- Enhanced the existing NewUserGuide component to utilize the new SpotlightGuide for improved visual guidance.
- Updated localization files to include new strings for contextual guides, ensuring a comprehensive user experience across languages.
- Refactored existing components to integrate with the new guide system, improving maintainability and user interaction.
2026-06-03 14:42:46 +08:00
wizardchen 48381fbaf5 feat: implement tenant default storage provider handling in Knowledge Base creation
This commit introduces functionality to utilize the tenant's default storage provider when creating a Knowledge Base. It includes updates to the frontend to load the default provider from settings and apply it during Knowledge Base initialization. Additionally, the backend has been enhanced to ensure that the storage provider is set correctly based on tenant configuration, improving consistency across the application. Tests have been added to verify the correct application of the default storage provider in various scenarios.
2026-05-29 13:47:32 +08:00
ochan.kwon 8b7d00b260 feat: expose KB ↔ vector store binding in list, editor, and detail UI
Backend
-------
ListKnowledgeBases now enriches each row with the resolved vector_store
metadata (name / source / engine_type / status) via a new
buildKBListResponse helper. Store views are batch-resolved once per
request through BatchResolveStoreView so an N-KB list costs one
vector-store service call rather than N — closing the N+1 limitation
flagged in #1372's known-limitations section. Cross-tenant shared KBs
continue to render via SharedStoreDisplay so the owning tenant's store
inventory cannot be correlated across rows; the underlying vector_store_id
UUID is stripped from those responses.

Resolver failures degrade gracefully: bound KBs render as unavailable
instead of breaking the list. Test coverage pins the env / bound /
shared distinction, the batch-call-count invariant, and the
graceful-failure path.

Frontend
--------
KB editor modal gains a new "Vector Store" section. Create mode shows a
dropdown that combines the system default (env store) and the tenant's
configured user stores, fetched once at mount via the existing
listVectorStores API. Edit mode shows the bound store read-only via a
new VectorStoreBadge component with an explicit immutability hint —
matching the backend's `<-:create` GORM tag and the service-layer
UpdateKnowledgeBaseRequest DTO that already omit the field.

KB list cards surface a small engine-type badge for own-tenant bound
KBs, and a warning badge when the bound store is unavailable. Env-bound
and shared KBs render no badge (visual noise control). KB detail header
shows the bound store via the same VectorStoreBadge component; shared
KBs fall through to the badge's internal "shared" branch with no name /
engine / id rendered.

The KB editor's create-time error handler translates the typed
ErrVectorStoreBindingInvalid (2200) and ErrVectorStoreUnavailable
(2201) into localized messages and jumps the user back to the
VectorStore section so they can pick a different store or fall back to
the system default.

The KB row type gains five optional fields (vector_store_id / name /
engine_type / source / status). i18n: 18 new keys added to en-US,
ko-KR, zh-CN; ru-RU receives English placeholders pending translation
(consistent with prior PRs in this locale).

Part of #993 (Phase 2: Per-KB VectorStore Binding).
Phase 2 roadmap item: PR 5 (KB binding UI + list-response enrichment).
Depends on #994, #1310, #1372, #1386 (all backend in the Phase 2 series).
2026-05-22 17:40:10 +08:00
Li Xianggang b3dbf86bf5 feat(obs): 支持华为云obs存储 2026-05-18 19:38:23 +08:00
wizardchen c6a2f6c8cc refactor(permissions): enhance role-based access controls across knowledge base and agent views
- Updated visibility conditions for various UI elements based on user roles, ensuring that only users with appropriate permissions (Admin, Contributor, or KB creator) can access certain actions.
- Introduced computed properties to manage permissions for sharing knowledge bases, managing data sources, and executing graph extraction.
- Adjusted the AgentList and KnowledgeBase components to reflect these changes, improving user experience by preventing unauthorized actions from being displayed.
- Ensured that the backend routes are aligned with the updated frontend role checks, enhancing security and clarity in user interactions.
2026-05-18 17:28:58 +08:00
wizardchen a9d37970ae fix(initialization): fall back to stored apiKey when test-connecting an existing model
Direct regression from the previous credential subresource refactor: the
edit-mode ModelEditorDialog stopped binding apiKey into formData (the
field now lives in <CredentialResource>), so "Test connection" sent
apiKey="" and every upstream call (DeepSeek / lkeap / OpenAI etc.) 401'd.

Fix in the backend so the frontend stays simple:

- ModelTestRequest gains an optional ModelID field.
- New fillSecretsFromStoredModel helper: when ModelID is set AND a secret
  field on the request is empty, copy it from the stored (decrypted) model.
  Non-empty request values still win, so users typing a new key to verify
  it before saving behave the same as before.
- The four test entrypoints (chat / embedding / rerank / asr) call the
  helper right after ShouldBindJSON.

Frontend: ModelEditorDialog.testConnection forwards modelId when isEdit
is true. The four API client signatures accept the optional field.

WebSearch and MCP are unaffected: WebSearch already has a TestProviderByID
ID-based path, MCP has no standalone raw test endpoint.
2026-05-17 15:27:52 +08:00
wizardchen 7643dd457e refactor(credentials): introduce /credentials subresource pattern
Replace the legacy "redacted placeholder + Clear* boolean" pattern with
dedicated per-resource credential subresources across MCP services,
Models, Web search providers, and Data sources.

Why
---
The previous design had three problems:

1. Main PUT body carried secret fields. The frontend echoed back a
   redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty
   defense in the service layer tried to detect "user did not change this"
   vs "user wants to clear this". A regression in that defense (or a new
   frontend forgetting it) silently overwrites the stored secret with the
   placeholder.

2. The "remove this credential" UX was a red checkbox under a pre-filled
   password input. Three intents (preserve / replace / clear) collapsed
   onto one field, and credential changes were bundled with unrelated
   config edits in the same submit. Users wiped working keys by mistake.

3. Secret presence was inferred from "did the response come back with a
   '***' placeholder", which couples the contract to a magic string.

Design
------
Each of the four resources now exposes:

  PUT    /{resource}/{id}/credentials          # write one or more fields
  DELETE /{resource}/{id}/credentials/{field}  # clear a single field

"Is this configured?" metadata travels on the main resource response as
a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET
endpoint. The frontend reads the configured boolean from the main GET
and never sees secret values at all.

Main PUT endpoints now ignore any secret fields in the request body and
log a deprecation warning if they appear, so legacy clients fail loudly
rather than overwriting silently.

Frontend
--------
- New reusable <CredentialResource> component renders a three-state card
  (unconfigured / configured / editing) and drives the dedicated
  endpoints. Used by MCP, Model, Web search; DataSource has a bespoke
  card with the same behaviour because its credentials are a single
  atomic per-connector map.
- Cancel from edit mode now restores state synchronously from the meta
  prop. The previous async refresh() was a no-op while state was still
  'editing', leaving the input frozen open.
- Remove is single-click + toast. The danger-themed button is the
  deterrent; a modal confirm adds friction without adding safety (the
  plaintext is irrecoverable client-side either way — recovery means
  re-typing).

DTOs (internal/handler/dto/) are deliberately separate from the GORM
models so "no secret in response" is a compile-time invariant: a future
contributor cannot leak a secret without explicitly adding the field to
the DTO, which is review-able in a single diff.

Storage is unchanged — credentials still live in the existing jsonb /
parameters columns. No schema migration.

Cleanup
-------
- types.MCPService / types.Model / types.WebSearchProviderEntity /
  types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret
  / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData().
- utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty.
  RedactedSecretPlaceholder constant is retained because VectorStore
  still uses the old pattern and is out of scope here.
- Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat
  redaction handling removed; i18n keys renamed secret -> credential.
2026-05-17 15:27:52 +08:00
ochan.kwon d869cc1ff6 fix: redact sensitive fields and prevent credential loss on edit
Closes #988

Unify the "write-only secrets" pattern across MCP services, Models,
WebSearch providers, and DataSources. Fix two related defects:

1. GET responses leaked plaintext credentials — GetMCPServiceByID
   while List masked, all Model / WebSearchProvider / DataSource GET
   responses unconditionally.
2. When the frontend echoed a masked placeholder back on save, the
   update path overwrote the stored secret with the mask — silently
   wiping the real credential. MCP surfaced this most visibly:
   ListMCPServices masked Token to "first4****last4" and
   McpServiceDialog round-tripped that masked string back into the DB.

- internal/types/secret.go introduces the shared sentinel
  RedactedSecretPlaceholder = "***" and two helpers:
  - IsRedactedOrEmpty(s): true for "" or the sentinel
  - PreserveIfRedacted(incoming, existing): merge predicate
- Each resource adds a RedactSensitiveData (mutates-in-place) or
  Redacted (returns-a-copy) method and a MergeUpdate contract:
  empty or placeholder preserves, any other value replaces, and a
  write-only Clear* bool flag (gorm:"-", json omitempty) wipes with
  precedence over preserve / replace. Clear* never persists.
- Every GET / List / Create / Update response path now redacts.
  VectorStore's pre-existing "***" literal is refactored onto the
  shared constant for consistency.

All four dialogs — McpServiceDialog, ModelEditorDialog,
WebSearchSettings, DataSourceEditorDialog — never pre-fill secret
inputs. The stored-state signal is collapsed into the input
placeholder: "•••••••• (Enter new value to replace)" when the server
reports a stored value, falling back to the connector-specific
placeholder otherwise. The input itself carries the "something is
there" cue, matching how password dots render.

An opt-in "Remove this credential" checkbox appears in edit mode when
a value is stored; checking it disables the input and triggers a
danger-themed confirmation dialog before submit. DataSource rolls the
clear up to a connector-level "Remove all credentials" flag since its
credentials are per-connector atomic bundles (e.g. Feishu app_id +
app_secret).

i18n updates across en-US, zh-CN, ko-KR, ru-RU:
secret.storedPlaceholder / clearHint / confirmClearTitle /
confirmClearBody.

- MCP update handler's manual map-to-struct projection was silently
  dropping clear_api_key / clear_token / custom_headers on partial
  updates. Pull them off the map explicitly.
- MCP configChanged detection now snapshots URL / StdioConfig /
  TransportType before the merge so connection recycling still
  triggers on real transport changes (not just whenever AuthConfig
  is present).
- MCP CustomHeaders: nil preserves existing, non-nil replaces. Was
  always overwriting, silently wiping headers on any auth-bearing
  update.
- GetMCPServiceByID now redacts (was plaintext while List masked).
- WebSearchProvider update preserves Name / Description when empty
  instead of clobbering them.
- WebSearchProvider TestProviderRaw rejects the "***" placeholder up
  front with an actionable message rather than forwarding it to the
  upstream provider API as a literal key.
- Model builtin hideSensitiveInfo now preserves non-secret metadata
  (SupportsVision, Provider, ExtraConfig, InterfaceType,
  ParameterSize, EmbeddingParameters) that were previously zeroed
  out for builtin model cards.
- Replace local equalStringSlices / equalStringMaps helpers with
  stdlib slices.Equal / maps.Equal (Go 1.21+).

- internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go:
  table-driven coverage of merge, redact, clear, and write-only flag
  invariants for every resource.
- internal/application/service/mcp_service_test.go: full service layer
  with a fake repo; covers ListMCPServices / GetMCPServiceByID
  redaction, UpdateMCPService partial-update honoring of Clear*,
  CustomHeaders preserve-on-nil / replace-on-non-nil, and
  clear-flag precedence over submitted values.

Additive only. No changes to request or response JSON shape beyond
the write-only Clear* flags (omitted by default via omitempty). No DB
schema changes. No SDK struct field removals. Existing clients that
don't know about the new flags continue to function unchanged.

- DataSource.Config jsonb at-rest encryption. Pre-existing residual
  risk; tracked as a follow-up.
- Renaming VectorStore.MaskSensitiveFields to match the new
  RedactSensitiveData / Redacted naming convention. Left alone to
  minimize diff; a follow-up rename PR can unify.
2026-05-17 11:37:53 +08:00
wizardchen 6019c4b661 fix(frontend): polish model settings UI and fix disabled brand color
- Fix global `--td-brand-color-disabled` to use correct muted green instead of bright green
- Unify `variant="text"` buttons (refresh, download, check dimension) in model settings to inherit standard TDesign styles
- Adjust WeKnora Cloud credential status layout and change reconfigure button to outline
- Fix font size of external doc links in model settings hint blocks
- Increase font weight of KB chunking debug trigger for better visibility
2026-05-07 21:51:22 +08:00
wizardchen c77cef5a69 feat(frontend): unify external doc-link style and outline add buttons
- Add global .doc-link utility in theme.css (brand color text + link icon, underline on hover).

- Convert all external doc/console links across settings to use .doc-link with a trailing <t-icon name="link" />, matching the Ollama browse-library style.

- Switch primary section-toolbar 'Add X' buttons (Model/WebSearch/VectorStore/MCP/DataSource) to outline variant for a lighter, more consistent look.

- Drop redundant per-component CSS for builtin-hint-link, weknoracloud-hint-link, engine-doc-link, engine-link, hint-link, section-doc-link, ds-doc-link-inline, model-link, etc.
2026-05-07 21:51:22 +08:00
wizardchen cc6ac6ac50 fix(frontend): unify radio button theme and disabled selection
- Add global t-radio-button outline styles in theme.css (brand fill when checked).

- Align ModelEditor, share KB, datasource, agent/MCP settings with t-radio-button.

- Remove redundant :deep overrides in AgentEditorModal, KB editor, FAQEntryManager, etc.

- Use brand disabled tone for disabled+checked radio buttons so selection stays visible.
2026-05-07 21:51:22 +08:00
wizardchen c577d0e474 fix(knowledge): polish chunking-settings debug trigger and strategy hint
The chunking-settings debug trigger and strategy picker had two UX
issues reported by real usage:

- The test button used theme=primary + variant=outline, a combo that
  appears nowhere else in the project. A green-bordered green-text
  pill in the corner of the section header read as a misplaced
  promotional element rather than a secondary action.
- The strategy select's placeholder was "选择策略(默认使用经典递归分块)"
  / "Select strategy (defaults to classic recursive splitting)". Naming
  the fallback strategy inside the placeholder made the empty select
  look as if the user had already picked "classic recursive" — even
  though the value is actually empty.

- Move the debug trigger from the section header into the strategy
  row's right column, stacked below the select. Users meet the test
  action exactly when they're deciding which strategy to use.
- Switch the trigger to a text-style button and override TDesign's
  default hover/focus/active surfaces so the button reads as an inline
  link (text colour shift only, no filled background block).
- Rename the trigger to "测试分块效果" / "Preview chunking" so it
  describes what will be inspected rather than the input source.
- Drop the reference to the fallback strategy inside the placeholder.
  "不填则按长度切分" / "splits by length if left empty" preserves the
  original information about empty-value behaviour without looking
  like a selected option.
- Remove the now-unused 80px right padding the section header reserved
  to keep the old inline trigger clear of the modal close button.
2026-05-07 11:08:29 +08:00
wizardchen 49cd392308 fix(frontend): rework chunking debug UX in KB editor
The chunking debug panel was an inline section that crowded the
strategy form, broke its own layout with negative margins, and crashed
on result.rejected being null. The trigger button required scrolling to
the form's bottom to reach.

- Move debug into a right-side t-drawer pinned above the modal stacking
  context (attach=body, z-index 3000) so it overlays cleanly.
- Promote the trigger to a sticky header sibling next to the section
  title, with right padding to clear the modal close button.
- Replace unicode arrows / + / − with tdesign-icons (Chevron, Play,
  ErrorCircle) and rebuild the result layout: pill-shaped breadcrumb
  tags, line-clamped previews with gradient fade on collapse, ordered
  chunk list, structured error block.
- Drop custom char counter (t-textarea already has one).
- Add four sample fixtures (markdown / faq / chapter / plain) loadable
  as preset chips so users have realistic content to test against.
- Guard `result.rejected` nullability that crashed the panel.
- Normalize the recursive tier label to legacy in the UI since both
  tiers behave identically — pairs with the backend cleanup.
- Rename strategy labels in zh-CN/en-US/ko-KR/ru-RU from internal jargon
  ("auto", "heading", "heuristic", "legacy") to descriptive names
  ("Automatic", "Heading-aware", "Structure-aware", "Length-based") and
  add presetLabel + sample i18n keys.
2026-05-06 20:23:21 +08:00
Claude 55bc5dcbe2 fix(chunking): post-review polish — error surfacing + doc accuracy
Three small follow-ups from the QA review of 13f57ca:

1. KBChunkingDebug error surfacing
   Previously a 200 OK response with { success: false, error: "..." }
   would have been swallowed under a generic "unexpected response shape"
   message. The strict-shape check now distinguishes empty response,
   success=false (surfaces resp.error directly), and missing data — so
   any future backend-side validation message reaches the user.

2. Token approximation in docs/CHUNKING.md
   Child Chunk Size default 384 ≈ 95 EN tokens (was rounded to 80).
   384 / 4 chars-per-token = 96; "~95" is the honest figure.

3. API surface in docs/CHUNKING.md
   The example only documented PUT /initialization/config/:kbId
   (camelCase, documentSplitting envelope). Added explicit notes that
   POST/PUT /knowledge-bases use snake_case under chunking_config, and
   that POST /chunker/preview also uses the snake_case form plus a text
   field. Readers picking the wrong endpoint won't be surprised by the
   case mismatch anymore.

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00
Claude 49adf0d73c docs(chunking): align overlap default + sharpen UI/source/repo docs
Three documentation passes around the adaptive chunking work:

UI
- Frontend ChunkOverlap default consolidated to 80 (was 100), matching
  chunker.DefaultChunkOverlap on the backend. Both DEFAULT_CHUNKING_PRESET
  and initFormData updated. The KB-load fallback also uses 80 when a
  loaded KB has no chunk_overlap stored.
- All four locales (en-US, zh-CN, ko-KR, ru-RU) get rewritten chunking
  setting descriptions: each now states the validated range, the default,
  and the situations where you'd deviate (FAQ vs narrative, embedder
  token limits, language-specific corpora).

Source code
- splitter.go: DefaultChunkSize / DefaultChunkOverlap constants get a
  longer block-comment explaining the per-language token math and the
  use-case sweet spots, plus the migration note on what the old
  inconsistent defaults were.
- KBChunkingSettings.vue: new comment block above ChunkingConfig
  documents the slider min/max for each setting, why those bounds
  exist, and the recommended TokenLimit values per embedding model.

Repo docs
- New docs/CHUNKING.md: end-to-end guide covering why chunking matters,
  the adaptive 3-tier architecture, per-setting reference with ranges
  and sweet spots, parent-child explanation, the token-limit table per
  embedder (OpenAI / Voyage / Cohere / BGE / MiniLM / Jina), 7 use-case
  presets, the debug panel workflow, the API surface, and known
  trade-offs (recursive strategy hidden from UI, no auto-reindex on
  strategy switch, OCR limitations).
- CHANGELOG.md gets a new [Unreleased] section consolidating all the
  adaptive-chunking work shipped on this branch: 5 features, 8
  improvements, 6 fixes, 1 docs entry. The entry references
  docs/CHUNKING.md for deeper explanation.

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00
Claude 2cc0322004 fix(frontend): chunking debug panel — visible loading + error states, prevent submit
User report: clicking "Run preview" made the debug section appear to
disappear. Three independent problems contributed; fix all of them.

1. t-button rendered without explicit type="button". TDesign defaults
   vary across versions; if the panel ever lives inside a parent
   <form> (the editor modal does not currently, but this is defense
   in depth), the click would submit the form and reload the page.
   Added type="button" plus @click.prevent.stop on the run button.

2. No loading visual. While the request was in flight, result was
   set back to null, so the result block disappeared with nothing
   in its place. The textarea was still on screen but the spacing
   shifted enough that users could read it as "section gone". Added
   a loading panel with t-loading spinner + status text.

3. Error display was too quiet. A tiny one-line error block could be
   missed entirely behind the modal's scrollable content. Made it
   visually prominent (bold prefix, full-width error background) so
   it can't be overlooked.

Bonus hardening:
- Stricter response-shape check (resp.success !== true || !resp.data
  catches malformed responses, not just non-200s).
- console.error with [KBChunkingDebug] prefix so the failure is
  debuggable from DevTools without enabling verbose logging.
- New i18n key chunking.debug.loading across all 4 locales.

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00
Claude 8388a50c23 fix(chunker): post-review fixes for preview endpoint robustness
Resolves issues from the review of be326aa..119f5e4. Each fix has a
regression test attached and overclaimed findings (parserEngineRules
defensive copy, runTier dead-code branch, init-vars architecture)
were intentionally not touched after re-evaluation.

Goroutine leak mitigation
- previewMaxChars dropped from 256k to 64k runes. The splitter does
  not accept a context.Context, so when previewTimeout fires the
  worker keeps running. Bounding input size keeps worst-case CPU
  per request well under a second on commodity hardware. Sized so
  10 concurrent timeouts don't pile up faster than they finish.
- Frontend MAX_CHARS lowered to match.
- Comment in handler explains the trade-off and points at the
  follow-up: real cancellation needs the splitter to take a ctx.

Performance
- ApproxTokenCountFromRuneLen variant lets the preview handler
  reuse a single rune-count per chunk for stats + size + token
  estimation. Eliminates the previous triple []rune allocation per
  chunk in the response loop.
- computeChunkSizeStats now takes []int (pre-computed rune lens)
  instead of []chunker.Chunk; sumSq computed in float64 to avoid
  the int*int overflow at l > ~46k.

Correctness / UX
- Preview panel sends strategy / token_limit / languages
  unconditionally, mirroring the buildSubmitData convention so the
  preview faithfully reflects what would happen on save.
- Empty-text returns a friendly 400 ("paste a sample…") instead of
  gin's cryptic 'Field validation failed on the required tag'.

Tests
- TestSplit_DelegatesToSplitWithDiagnostics renamed to
  TestSplit_AndDiagnostics_AgreeOnChunks (the post-audit refactor
  made the original name a misnomer; the test still asserts the
  right invariant under the new name).
- New TestSplitWithDiagnostics_ProfileSetForAuto and
  TestSplitWithDiagnostics_ProfileNilForExplicit lock in the
  profile-reuse contract that the preview endpoint depends on.
- New chunker_debug_test.go covers computeChunkSizeStats edge
  cases (empty / single / varying / no-variance underflow) plus
  PreviewChunking httptest scenarios (auto path, legacy strategy,
  empty-text rejection, oversize rejection, chunk truncation with
  full-set stats).

Doc cleanup
- runTier comment updated; the "stubbed in this scaffold" line was
  obsolete since the heading and heuristic splitters shipped.

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00
Claude beb7d005e9 fix(chunker): QA audit — pointer DTOs, profile reuse, stats correctness
Resolves the issues surfaced by the post-Phase-3 self-review.

High severity
- Strategy / TokenLimit / Languages can now be reset. The KB-config
  handler uses pointer DTOs so the absence of a field in the payload
  means "no change" while an explicit empty string / 0 / [] clears
  the value. The frontend now sends the fields unconditionally so
  the user can revert to defaults via the clear icon — previously
  these fields were write-once.
- DocProfile is no longer computed twice in the preview endpoint.
  Diagnostics carries the profile that drove tier selection (when
  auto-strategy ran); the handler only re-profiles for explicit
  strategies that bypass the profiler.
- Chunk-size stats (avg / min / max / stddev) are now computed over
  the FULL chunk set before the response is truncated to
  previewMaxChunks. Avoids systematically misleading metrics for
  large documents.
- SplitWithDiagnostics defaults SelectedTier to TierLegacy so an
  empty-text request never leaves the diag with a blank tier string.

Medium severity
- Split() reverts to its own loop instead of delegating to
  SplitWithDiagnostics. Diagnostics struct allocation is reserved
  for callers that actually want the trace (preview endpoint).
  Matters in SplitParentChild where Split runs once per parent.
- Preview endpoint runs the splitter on a goroutine wrapped in a
  5s context.WithTimeout. CJK input at the 256k-rune ceiling
  could otherwise tie up the worker for several seconds.

Low severity
- KBChunkingSettings emits arrays via spread copies so parent-side
  mutation cannot drift the form's reactive state.
- KBChunkingDebug font-family includes a monospace fallback so the
  chunk content stays monospaced even when the TDesign CSS variable
  is missing.

Documentation
- DocProfile JSON shape is now annotated as part of the public
  preview API; field renames would be breaking.

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00
Claude 6864a9f9e2 feat(frontend): chunking debug panel with live preview
Adds a collapsible "Test with sample text" panel underneath the
chunking settings in the KB editor. Users paste a Markdown / plain-text
snippet, hit "Run preview", and see exactly how the current chunking
configuration would split it — without re-uploading any document.

The panel calls POST /api/v1/chunker/preview and renders:
- Selected tier (heading / heuristic / recursive / legacy) as a tag,
  with a warning when the chain fell through to legacy fallback
- Rejected tiers + reasons (so users see why "auto" picked tier X)
- DocProfile stats grid: lines, chars, headings, page-breaks,
  chapter markers, detected languages
- Chunk-size summary: count / avg / stddev / min / max
- Per-chunk cards with seq, size (chars + ~tokens), position range,
  context header (when set), content preview (expand/collapse)

The panel reads its config from the live form state so edits to
strategy / chunk-size / overlap / etc. are reflected on the next
preview run — no save needed.

Pieces:
- types/chunker.ts: TS types mirroring the backend response shape
- api/chunker/index.ts: previewChunking(body) wrapper
- views/knowledge/settings/KBChunkingDebug.vue: the panel component
- KBChunkingSettings.vue: embeds the panel + exposes a debugConfig
  computed prop
- i18n keys for en-US / zh-CN / ko-KR / ru-RU (8 keys each)

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00
Claude 46d9c396c6 feat(frontend): expose chunking strategy in KB editor
Surfaces the adaptive 3-tier chunking controls in the existing
"Chunking" section of the KB editor modal. Users can now pick a
strategy from a dropdown without crafting JSON for the API:

- Automatic (recommended)  → backend profiler picks the tier
- Markdown-optimized       → forces heading-aware splitting
- Smart structure detection → forces heuristic boundary detection
- Classic                  → original recursive splitter (legacy)

The fifth backend value `recursive` is intentionally hidden from the
UI — it's functionally near `legacy` and would dilute the choice.

Advanced options (collapsible, disabled when strategy=legacy):
- TokenLimit (0–8192 approx tokens; 0 = char-based budget)
- Languages multi-select (DE/EN/ZH; empty = auto-detect)

Behaviour:
- New KBs default to strategy="auto"
- Existing KBs without the field render with empty strategy and keep
  classic behavior on save unless the user actively picks one
- Overlap warning when overlap >= chunkSize/2

Wired through:
- KBChunkingSettings.vue: strategy select + advanced collapsible +
  ChunkingConfig interface extended
- KnowledgeBaseEditorModal.vue: form-state defaults, kb-load mapping,
  buildSubmitData (chunking_config) + updateKBConfig (documentSplitting)
  pass strategy/token_limit/languages
- api/initialization/index.ts: types extended
- i18n: en-US / zh-CN / ko-KR / ru-RU all carry strategy + advanced
  keys (6 keys each)

https://claude.ai/code/session_01XADhx6mtu2ZYW3DE9Lun6k
2026-05-06 17:17:07 +08:00
Claude f140c12518 feat(frontend): unify font rendering and add font picker in settings
The UI used hardcoded "PingFang SC" in 50+ components, while the global
font stack in App.vue used a different fallback chain. Outside of mainland
China this caused visibly different fonts in the sidebar, breadcrumbs,
search and agent views compared to settings and document panes.

- Introduce --app-font-family / --app-font-family-mono CSS variables in
  theme.css and route TDesign's --td-font-family through them
- Add a useFont composable that persists the user's choice in localStorage
  and applies it via document.documentElement.style before mount
- Replace hardcoded font-family declarations across the app with the new
  variables (TencentSans logo and the SVG attribute list are preserved)
- Add UI font and monospace font selectors in General Settings with
  i18n strings for en-US, zh-CN, ko-KR and ru-RU

https://claude.ai/code/session_01S1FVPf4jRfWXkTSnpMmMp7
2026-05-05 00:48:40 +08:00
ChenRussell 42348eb40d feat(ks3): add KS3 object storage support
Add Kingsoft Cloud KS3 as a supported storage provider, including
  backend service implementation, storage config UI, and provider
  scheme recognition throughout the codebase.
2026-05-05 00:22:27 +08:00
Windfarer 20be935fe0 Merge branch 'main' into storage-allow-list 2026-04-29 19:35:13 +08:00
Windfarer 6601a483be feat: add STORAGE_ALLOW_LIST env var 2026-04-29 18:58:09 +08:00
wizardchen a8a3d9e694 feat: Enhance wiki extraction and configuration options
- Introduced extraction granularity settings in the wiki configuration, allowing users to control the level of detail in entity and concept extraction (focused, standard, exhaustive).
- Updated the UI to reflect these new options, including tooltips for better user guidance.
- Refactored the knowledge base editor to support the new extraction granularity feature, ensuring a seamless user experience.
- Improved the backend logic for wiki ingest tasks to utilize the new granularity settings, enhancing the accuracy of extracted content.
- Added tests to validate the new extraction granularity functionality and ensure robust performance across different configurations.
2026-04-22 21:18:32 +08:00
wizardchen 28deb8282f feat: Add comprehensive documentation for WeKnora knowledge base and wiki features
- Introduced multiple new documentation files, including a complete documentation index, quick reference guides, and comprehensive analyses of the knowledge base and wiki settings.
- The new files provide detailed insights into the architecture, data models, API endpoints, and implementation strategies for the knowledge base and wiki functionalities.
- Enhanced navigation and usability for developers and team members by summarizing key components, configurations, and data flows related to the knowledge base system.

These additions significantly improve the clarity and accessibility of documentation, supporting better onboarding and reference for current and future team members.
2026-04-22 21:18:10 +08:00
nullkey 16792713c5 feat(frontend): enable Yuque datasource option with optional base_url
Flip the Yuque tile to available, add a base_url field (optional) for users
on private Yuque deployments, and populate the permission doc/page links.
Introduces an optional flag on ConnectorDef.fields so the Test Connection
validator skips empty optional fields; required fields get a visible * mark.

Also adds an optional hintKey on ConnectorDef.fields for per-field guidance
text, a Yuque-specific prereq guide, and a base_url hint explaining when to
leave it empty vs fill it for Yuque Enterprise / self-hosted deployments.
2026-04-22 16:39:55 +08:00
wizardchen 86d26fa9bf feat(video): remove video upload support and update related terminology
- Removed support for video file uploads across the application, including updates to the knowledge base and document processing services.
- Updated internationalization files to reflect the removal of video-related terminology, changing labels and descriptions to focus solely on audio processing.
- Adjusted file type handling to exclude video formats, ensuring users receive accurate feedback regarding supported file types.
- Enhanced error handling to provide clear messages when users attempt to upload unsupported video files.

This update streamlines the application by focusing on audio processing capabilities and improving user experience through clearer messaging.
2026-04-14 22:59:14 +08:00
wizardchen 0fcab3a413 feat(video): update ASR terminology and file type labels for improved clarity
- Changed 'Audio Processing' to 'Audio & Video' in English, Korean, Russian, and Chinese translations to reflect expanded functionality.
- Updated ASR configuration titles and descriptions to include video processing capabilities, enhancing user understanding of the feature.
- Renamed file type labels from 'Audio Files' to 'Audio & Video' across multiple languages for consistency and clarity.
- Enhanced document upload handling in the KnowledgeBase to filter unsupported file types, providing user feedback for filtered images, videos, and audio files.

This update improves the internationalization of the application and clarifies the capabilities of the ASR feature.
2026-04-14 22:59:14 +08:00
wizardchen 1fc90f5772 refactor(weknoracloud): enhance internationalization and UI feedback for credential management
- Updated the WeKnoraCloud settings and related components to utilize i18n for improved localization.
- Replaced hardcoded strings with translation keys for better maintainability and user experience.
- Enhanced user feedback messages for credential status, including configuration, expiration, and unconfiguration states.
- Improved UI elements to guide users in managing WeKnoraCloud credentials effectively.

This update streamlines the user interface and ensures consistent messaging across different languages.
2026-04-14 17:36:01 +08:00
wizardchen 0eff63b308 feat(weknoracloud): implement credential management and status checks
- Introduced a new WeKnoraCloudSettings component for managing APPID and APPSECRET.
- Added functionality to save WeKnoraCloud credentials without automatically creating models.
- Implemented credential status checks across various components to provide user feedback on configuration status.
- Updated API endpoints to handle credential saving and status retrieval.
- Enhanced UI elements to guide users in configuring WeKnoraCloud settings.

This update improves the integration of WeKnoraCloud by allowing users to manage their credentials more effectively and receive real-time feedback on their configuration status.
2026-04-14 17:36:01 +08:00
bingxiang.cheng 847660e4fc feat(storage): 添加对阿里云OSS的支持,更新相关文档和界面文本 2026-04-13 15:37:33 +08:00
nullkey 11b30fdd61 feat(frontend): tree view resource selector with cascading selection
- Render resources as expandable tree with parent-child indentation
- Cascade check/uncheck to all descendants
- Add has_children to Resource API type
- Add 'untitled' i18n key for en-US, zh-CN, ko-KR, ru-RU
2026-04-08 16:31:25 +08:00
wizardchen c93f4e94ae feat(asr): add ASR model connection check and update related components
- Implemented a new function to check ASR model connectivity via the /v1/audio/transcriptions endpoint.
- Updated ModelEditorDialog.vue to integrate ASR model checks and adjust UI elements accordingly.
- Enhanced localization files to reflect changes in ASR terminology and descriptions.
- Added ASR configuration options to the knowledge base and updated related migration scripts.
- Refactored model handling to support ASR as a distinct model type in the application.
2026-04-02 21:27:27 +08:00
wizardchen d2d1d6d59a feat(datasource): enhance DataSourceSettings and DataSourceSyncLogs functionality
- Implemented polling mechanism in DataSourceSettings for real-time updates on data source synchronization status.
- Improved loading state management and user feedback during data source operations.
- Added load more functionality in DataSourceSyncLogs for better log retrieval experience.
- Updated UI elements for consistency, including icon changes and styling adjustments for better visual clarity.
2026-03-30 20:46:07 +08:00
wizardchen 6a5a46cfc3 feat(datasource): add icons and improve data source feedback
- Introduced new icons for Feishu, Notion, and Yuque data sources to enhance visual representation.
- Updated localization files to include success and failure messages for data source creation and synchronization.
- Enhanced DataSourceEditorDialog and DataSourceSettings components with improved user feedback during data source operations.
- Added DataSourceTypeIcon component for consistent icon display across data source settings.
2026-03-30 20:46:07 +08:00
wizardchen 1a340a8374 refactor(datasource): remove WeCom document integration and related components
- Deleted WeCom document connector and callback handling code to streamline the datasource management.
- Removed associated localization entries and UI elements related to WeCom integration.
- Updated data source management logic to eliminate references to WeCom, enhancing maintainability and clarity.
- Adjusted error handling and logging to reflect the removal of WeCom-specific functionality.
2026-03-30 20:46:07 +08:00
wizardchen 36dac0120a fix(datasource): improve error handling and user feedback in DataSource components
- Updated error handling in DataSourceSettings and DataSourceEditorDialog to provide more informative messages.
- Enhanced the UI to display error details more clearly, improving user experience during connection tests and resource loading.
- Adjusted HTTP response status checks in request utility for better accuracy in handling success responses.
2026-03-30 20:46:07 +08:00
wizardchen 8021647929 feat(datasource): enhance DataSourceSettings and DataSourceSyncLogs UI
- Improved DataSourceSettings component with better loading handling and UI updates for data source cards.
- Refactored layout and styling for data source cards, including enhanced title and status display.
- Updated DataSourceSyncLogs component with improved summary statistics and log display styles.
- Enhanced responsiveness and visual consistency across components with updated padding, margins, and font sizes.
2026-03-30 20:46:07 +08:00
wizardchen d5ec5dc0ea feat(i18n): update localization files for new UI elements and actions
- Added translations for "Refresh logs" in English, Korean, Russian, and Chinese.
- Enhanced the knowledge base editor modal to include a badge for data source count.
- Improved data source settings UI with a confirmation dialog for deletion and a refresh button in sync logs.
- Updated the data source sync logs component to fetch logs on visibility change and added a refresh option in the header.
2026-03-30 20:46:07 +08:00
wizardchen 11cabd5952 feat(datasource): add WeCom document integration and callback setup
- Implemented WeCom document connector to sync smart documents and WeDrive files into WeKnora.
- Added detailed documentation for WeCom integration, including setup instructions for callback URL and API access.
- Enhanced frontend localization to support WeCom-specific messages and instructions.
- Updated data source management UI to include WeCom configuration options and callback URL handling.
- Improved error handling and validation for WeCom API interactions.
2026-03-30 20:46:07 +08:00
wizardchen adabf8e089 feat(datasource): implement data source management functionality
- Added a new API for managing data sources, including CRUD operations and connection validation.
- Introduced UI components for data source settings, sync logs, and editor dialog, enhancing user experience in managing external data sources.
- Updated localization files to support new data source features in multiple languages.
- Implemented cron expression humanization for better scheduling visibility.
- Enhanced knowledge base editor to include data source management options.
2026-03-30 20:46:07 +08:00
wizardchen 3e3ffa4b0b feat(i18n): add JSON file type support across multiple locales
- Introduced support for JSON file types in English, Korean, Russian, and Chinese locale files.
- Updated the file type acceptance criteria in the document upload component to include JSON.
- Enhanced the knowledge base parser settings to recognize JSON as a valid file type.
- Implemented JSON conversion functionality in the document parser, allowing for JSON content to be processed and converted to markdown.
- Added comprehensive tests for JSON to markdown conversion to ensure functionality and reliability.
2026-03-25 22:08:29 +08:00
wizardchen a8185dcb43 feat: add computed model status for LLM availability in GraphSettings 2026-03-13 19:42:07 +08:00
wizardchen 5996d07aa6 refactor: update LLM configuration to model ID in API and frontend
- Replaced LLMConfig with model_id in TextRelationExtractionRequest and FabriTextRequest interfaces.
- Updated related API handlers to retrieve chat models based on model_id.
- Adjusted frontend components to pass model_id instead of LLMConfig, enhancing clarity and simplifying the configuration process.

These changes streamline the model configuration process across the application, improving maintainability and usability.
2026-03-13 19:34:35 +08:00