Commit Graph

3 Commits

Author SHA1 Message Date
ochan.kwon d869cc1ff6 fix: redact sensitive fields and prevent credential loss on edit
Closes #988

Unify the "write-only secrets" pattern across MCP services, Models,
WebSearch providers, and DataSources. Fix two related defects:

1. GET responses leaked plaintext credentials — GetMCPServiceByID
   while List masked, all Model / WebSearchProvider / DataSource GET
   responses unconditionally.
2. When the frontend echoed a masked placeholder back on save, the
   update path overwrote the stored secret with the mask — silently
   wiping the real credential. MCP surfaced this most visibly:
   ListMCPServices masked Token to "first4****last4" and
   McpServiceDialog round-tripped that masked string back into the DB.

- internal/types/secret.go introduces the shared sentinel
  RedactedSecretPlaceholder = "***" and two helpers:
  - IsRedactedOrEmpty(s): true for "" or the sentinel
  - PreserveIfRedacted(incoming, existing): merge predicate
- Each resource adds a RedactSensitiveData (mutates-in-place) or
  Redacted (returns-a-copy) method and a MergeUpdate contract:
  empty or placeholder preserves, any other value replaces, and a
  write-only Clear* bool flag (gorm:"-", json omitempty) wipes with
  precedence over preserve / replace. Clear* never persists.
- Every GET / List / Create / Update response path now redacts.
  VectorStore's pre-existing "***" literal is refactored onto the
  shared constant for consistency.

All four dialogs — McpServiceDialog, ModelEditorDialog,
WebSearchSettings, DataSourceEditorDialog — never pre-fill secret
inputs. The stored-state signal is collapsed into the input
placeholder: "•••••••• (Enter new value to replace)" when the server
reports a stored value, falling back to the connector-specific
placeholder otherwise. The input itself carries the "something is
there" cue, matching how password dots render.

An opt-in "Remove this credential" checkbox appears in edit mode when
a value is stored; checking it disables the input and triggers a
danger-themed confirmation dialog before submit. DataSource rolls the
clear up to a connector-level "Remove all credentials" flag since its
credentials are per-connector atomic bundles (e.g. Feishu app_id +
app_secret).

i18n updates across en-US, zh-CN, ko-KR, ru-RU:
secret.storedPlaceholder / clearHint / confirmClearTitle /
confirmClearBody.

- MCP update handler's manual map-to-struct projection was silently
  dropping clear_api_key / clear_token / custom_headers on partial
  updates. Pull them off the map explicitly.
- MCP configChanged detection now snapshots URL / StdioConfig /
  TransportType before the merge so connection recycling still
  triggers on real transport changes (not just whenever AuthConfig
  is present).
- MCP CustomHeaders: nil preserves existing, non-nil replaces. Was
  always overwriting, silently wiping headers on any auth-bearing
  update.
- GetMCPServiceByID now redacts (was plaintext while List masked).
- WebSearchProvider update preserves Name / Description when empty
  instead of clobbering them.
- WebSearchProvider TestProviderRaw rejects the "***" placeholder up
  front with an actionable message rather than forwarding it to the
  upstream provider API as a literal key.
- Model builtin hideSensitiveInfo now preserves non-secret metadata
  (SupportsVision, Provider, ExtraConfig, InterfaceType,
  ParameterSize, EmbeddingParameters) that were previously zeroed
  out for builtin model cards.
- Replace local equalStringSlices / equalStringMaps helpers with
  stdlib slices.Equal / maps.Equal (Go 1.21+).

- internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go:
  table-driven coverage of merge, redact, clear, and write-only flag
  invariants for every resource.
- internal/application/service/mcp_service_test.go: full service layer
  with a fake repo; covers ListMCPServices / GetMCPServiceByID
  redaction, UpdateMCPService partial-update honoring of Clear*,
  CustomHeaders preserve-on-nil / replace-on-non-nil, and
  clear-flag precedence over submitted values.

Additive only. No changes to request or response JSON shape beyond
the write-only Clear* flags (omitted by default via omitempty). No DB
schema changes. No SDK struct field removals. Existing clients that
don't know about the new flags continue to function unchanged.

- DataSource.Config jsonb at-rest encryption. Pre-existing residual
  risk; tracked as a follow-up.
- Renaming VectorStore.MaskSensitiveFields to match the new
  RedactSensitiveData / Redacted naming convention. Left alone to
  minimize diff; a follow-up rename PR can unify.
2026-05-17 11:37:53 +08:00
nullkey c753c67608 docs(api): add and fix swag annotations on handlers
将 swag 注解覆盖率从 ~88% 提升到 100%,并修复审计中发现的若干既有
注解 bug。

新增注解(24 个):
- im.go: 4 (UpdateIMChannel/DeleteIMChannel/ToggleIMChannel/IMCallback)
  其中 IMCallback 同时注册 GET 和 POST,两个方法都加了 @Router 声明。
- web_search.go: 1 (GetProviders)
- web_search_provider.go: 6
- wechat_qrcode.go: 2
- weknoracloud.go: 2
- knowledge.go: 2 (MoveKnowledge/GetKnowledgeMoveProgress)
- knowledgebase.go: 1 (ListMoveTargets)
- organization.go: 3 (ListSharedAgents/ListOrgAgentShares/RemoveAgentShare)
- mcp_service.go: 2 (SetMCPToolApproval/ResolveToolApproval)
- chunker_debug.go: 1 (PreviewChunking)

修复既有注解 bug(12 处):
- initialization.go: 10 个 @Router 路径与真实路由不符
  (e.g. /initialization/kb/{kbId}/config → /initialization/config/{kbId}
   /initialization/fabri/tag GET → /initialization/extract/fabri-tag POST)
- session/stream.go: ContinueStream 路径
  /sessions/{session_id}/continue → /sessions/continue-stream/{session_id}
- mcp_service.go: ResolveToolApproval 请求体 {approve: bool}
  → {decision: "approve"|"reject", reason?, modified_args?}

收敛响应 schema 到已有命名 struct(之前用 map[string]interface{}):
- knowledge.go MoveKnowledge → handler.MoveKnowledgeResponse
- knowledge.go GetKnowledgeMoveProgress → types.KnowledgeMoveProgress
- web_search_provider.go GetProvider/UpdateProvider → types.WebSearchProviderEntity
- initialization.go InitializeByKB → handler.InitializationRequest
- initialization.go 三个 model-test endpoint → handler.ModelTestRequest

修正 organization.go 中 errors.AppError 应使用 apperrors 别名(该文件
stdlib errors 未别名化)。

不补 /files 和 /api/v1/files/presigned 内联闭包路由(文件服务,非业务 REST)。

Refs #890 #1049 #1168
2026-05-12 13:16:58 +08:00
wizardchen a5ca9a2784 feat(web-search): refactor web search provider
- Updated relevant files to include provider registration, implementation, and metadata.
- Enhanced frontend components to support provider management and configuration.
- Added localization for new provider settings and messages.
- Implemented backend repository methods for CRUD operations on web search providers.
2026-03-31 20:22:34 +08:00