fix: redact sensitive fields and prevent credential loss on edit

Closes #988

Unify the "write-only secrets" pattern across MCP services, Models,
WebSearch providers, and DataSources. Fix two related defects:

1. GET responses leaked plaintext credentials — GetMCPServiceByID
   while List masked, all Model / WebSearchProvider / DataSource GET
   responses unconditionally.
2. When the frontend echoed a masked placeholder back on save, the
   update path overwrote the stored secret with the mask — silently
   wiping the real credential. MCP surfaced this most visibly:
   ListMCPServices masked Token to "first4****last4" and
   McpServiceDialog round-tripped that masked string back into the DB.

- internal/types/secret.go introduces the shared sentinel
  RedactedSecretPlaceholder = "***" and two helpers:
  - IsRedactedOrEmpty(s): true for "" or the sentinel
  - PreserveIfRedacted(incoming, existing): merge predicate
- Each resource adds a RedactSensitiveData (mutates-in-place) or
  Redacted (returns-a-copy) method and a MergeUpdate contract:
  empty or placeholder preserves, any other value replaces, and a
  write-only Clear* bool flag (gorm:"-", json omitempty) wipes with
  precedence over preserve / replace. Clear* never persists.
- Every GET / List / Create / Update response path now redacts.
  VectorStore's pre-existing "***" literal is refactored onto the
  shared constant for consistency.

All four dialogs — McpServiceDialog, ModelEditorDialog,
WebSearchSettings, DataSourceEditorDialog — never pre-fill secret
inputs. The stored-state signal is collapsed into the input
placeholder: "•••••••• (Enter new value to replace)" when the server
reports a stored value, falling back to the connector-specific
placeholder otherwise. The input itself carries the "something is
there" cue, matching how password dots render.

An opt-in "Remove this credential" checkbox appears in edit mode when
a value is stored; checking it disables the input and triggers a
danger-themed confirmation dialog before submit. DataSource rolls the
clear up to a connector-level "Remove all credentials" flag since its
credentials are per-connector atomic bundles (e.g. Feishu app_id +
app_secret).

i18n updates across en-US, zh-CN, ko-KR, ru-RU:
secret.storedPlaceholder / clearHint / confirmClearTitle /
confirmClearBody.

- MCP update handler's manual map-to-struct projection was silently
  dropping clear_api_key / clear_token / custom_headers on partial
  updates. Pull them off the map explicitly.
- MCP configChanged detection now snapshots URL / StdioConfig /
  TransportType before the merge so connection recycling still
  triggers on real transport changes (not just whenever AuthConfig
  is present).
- MCP CustomHeaders: nil preserves existing, non-nil replaces. Was
  always overwriting, silently wiping headers on any auth-bearing
  update.
- GetMCPServiceByID now redacts (was plaintext while List masked).
- WebSearchProvider update preserves Name / Description when empty
  instead of clobbering them.
- WebSearchProvider TestProviderRaw rejects the "***" placeholder up
  front with an actionable message rather than forwarding it to the
  upstream provider API as a literal key.
- Model builtin hideSensitiveInfo now preserves non-secret metadata
  (SupportsVision, Provider, ExtraConfig, InterfaceType,
  ParameterSize, EmbeddingParameters) that were previously zeroed
  out for builtin model cards.
- Replace local equalStringSlices / equalStringMaps helpers with
  stdlib slices.Equal / maps.Equal (Go 1.21+).

- internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go:
  table-driven coverage of merge, redact, clear, and write-only flag
  invariants for every resource.
- internal/application/service/mcp_service_test.go: full service layer
  with a fake repo; covers ListMCPServices / GetMCPServiceByID
  redaction, UpdateMCPService partial-update honoring of Clear*,
  CustomHeaders preserve-on-nil / replace-on-non-nil, and
  clear-flag precedence over submitted values.

Additive only. No changes to request or response JSON shape beyond
the write-only Clear* flags (omitted by default via omitempty). No DB
schema changes. No SDK struct field removals. Existing clients that
don't know about the new flags continue to function unchanged.

- DataSource.Config jsonb at-rest encryption. Pre-existing residual
  risk; tracked as a follow-up.
- Renaming VectorStore.MaskSensitiveFields to match the new
  RedactSensitiveData / Redacted naming convention. Left alone to
  minimize diff; a follow-up rename PR can unify.
This commit is contained in:
ochan.kwon
2026-04-18 21:48:50 +09:00
committed by lyingbug
parent a3411899cf
commit d869cc1ff6
32 changed files with 2039 additions and 130 deletions
@@ -1,6 +1,6 @@
<script setup lang="ts">
import { ref, computed, watch } from 'vue'
import { MessagePlugin } from 'tdesign-vue-next'
import { MessagePlugin, DialogPlugin } from 'tdesign-vue-next'
import { useI18n } from 'vue-i18n'
import {
createDataSource,
@@ -28,6 +28,38 @@ const isEdit = computed(() => !!props.dataSource)
const step = ref(0)
const submitting = ref(false)
// Fixed placeholder returned by the server for redacted credential strings.
// Must match internal/types/secret.go → RedactedSecretPlaceholder.
const REDACTED_PLACEHOLDER = '***'
// Per-field: is this specific credential key stored server-side? Drives the
// per-input placeholder swap to the "••••••• (Enter new value to replace)"
// hint when the server marked it present.
function hasFieldCredential(key: string): boolean {
return props.dataSource?.config?.credentials?.[key] === REDACTED_PLACEHOLDER
}
// Connector-level rollup: true when any credential key is stored. Used to
// decide whether to render the "Remove all credentials" checkbox.
const hasExistingCredentials = computed(() => {
const creds = props.dataSource?.config?.credentials
if (!creds) return false
return Object.values(creds).some(v => v === REDACTED_PLACEHOLDER)
})
// Placeholder for a given credential input: stored → shared hint,
// otherwise the connector-specific default (e.g. "ntn_xxxx" for Notion).
function credentialPlaceholder(field: { key: string; placeholder: string }): string {
return hasFieldCredential(field.key)
? t('secret.storedPlaceholder')
: field.placeholder
}
// Explicit "remove all credentials" flag. Because credentials form an atomic
// set per connector (e.g. an OAuth pair) we only support all-or-nothing
// clearing, mirroring the backend DataSourceConfig.ClearCredentials contract.
const clearCredentials = ref(false)
// Form data
const form = ref({
name: '',
@@ -213,10 +245,22 @@ watch(visible, (v) => {
selectedResourceIds.value = []
if (isEdit.value && props.dataSource) {
// Reset clear intent on every open so an aborted deletion doesn't
// carry over to the next edit session.
clearCredentials.value = false
// Credentials are intentionally NOT pre-filled from the server response
// — even though the server returns '***' for stored secrets, the form
// holds an empty map. This keeps the "non-empty means user typed it"
// invariant that the save path relies on, and the badge reflects the
// stored state via hasExistingCredentials instead.
form.value = {
name: props.dataSource.name,
type: props.dataSource.type,
config: props.dataSource.config || { credentials: {}, resource_ids: [], settings: {} },
config: {
credentials: {},
resource_ids: props.dataSource.config?.resource_ids || [],
settings: props.dataSource.config?.settings || {},
},
sync_schedule: props.dataSource.sync_schedule,
sync_mode: props.dataSource.sync_mode,
conflict_strategy: props.dataSource.conflict_strategy,
@@ -225,6 +269,7 @@ watch(visible, (v) => {
selectedResourceIds.value = form.value.config?.resource_ids || []
tempDsId.value = props.dataSource.id
} else {
clearCredentials.value = false
form.value = {
name: '',
type: '',
@@ -405,8 +450,45 @@ function prevStep() {
step.value--
}
// Prompt the user before irrevocable credential removal.
function confirmClearIfNeeded(): Promise<boolean> {
if (!clearCredentials.value) return Promise.resolve(true)
return new Promise((resolve) => {
const d = DialogPlugin.confirm({
header: t('secret.confirmClearTitle'),
body: t('secret.confirmClearBody'),
confirmBtn: { content: t('common.confirm'), theme: 'danger' },
cancelBtn: t('common.cancel'),
onConfirm: () => { d.hide(); resolve(true) },
onCancel: () => { d.hide(); resolve(false) },
onClose: () => { d.hide(); resolve(false) },
})
})
}
// Build the config payload for Create / Update requests. In edit mode this
// applies write-only secret semantics:
// - clearCredentials checked → send clear_credentials: true (backend wipes)
// - user typed credential values → send them as-is
// - user left fields blank → send empty credentials map (backend preserves
// every existing credential key individually)
function buildConfigPayload(): Record<string, unknown> {
const cfg: Record<string, unknown> = {
credentials: clearCredentials.value ? {} : { ...form.value.config.credentials },
resource_ids: form.value.config.resource_ids,
settings: form.value.config.settings,
}
if (clearCredentials.value) {
cfg.clear_credentials = true
}
return cfg
}
// --- Final submit ---
async function handleSubmit() {
const ok = await confirmClearIfNeeded()
if (!ok) return
form.value.config.resource_ids = selectedResourceIds.value
submitting.value = true
try {
@@ -415,12 +497,14 @@ async function handleSubmit() {
if (tempDsId.value) {
await updateDataSource(tempDsId.value, {
...form.value,
config: buildConfigPayload(),
knowledge_base_id: props.kbId,
status: 'active',
} as any)
} else {
const res = await createDataSource({
...form.value,
config: buildConfigPayload(),
knowledge_base_id: props.kbId,
status: 'active',
} as any)
@@ -575,6 +659,14 @@ const stepTitles = computed(() => [
</a>
</div>
<!--
Credential fields. When the server returned '***' for a field the
per-input placeholder swaps to the shared "••••••• (Enter new value
to replace)" hint so the input carries the "something is stored"
signal without a separate badge. The "Remove all credentials"
checkbox below handles the destructive clear for the whole set
(DataSource credentials are per-connector atomic bundles).
-->
<div v-for="field in currentDef?.fields || []" :key="field.key" class="form-item">
<label class="form-label">
{{ t(field.labelKey) }}
@@ -582,12 +674,19 @@ const stepTitles = computed(() => [
</label>
<t-input
v-model="form.config.credentials[field.key]"
:placeholder="field.placeholder"
:disabled="clearCredentials"
:placeholder="credentialPlaceholder(field)"
:type="field.secret ? 'password' : 'text'"
/>
<div v-if="field.hintKey" class="form-hint">{{ t(field.hintKey) }}</div>
</div>
<div v-if="isEdit && hasExistingCredentials" class="form-item">
<t-checkbox v-model="clearCredentials" class="clear-credential">
{{ t('secret.clearHint') }}
</t-checkbox>
</div>
<div class="form-actions">
<t-button variant="outline" :loading="testing" @click="testConnection">
{{ t('datasource.testConnection') }}
@@ -886,6 +985,12 @@ const stepTitles = computed(() => [
.form-item { margin-bottom: 16px; }
.form-label { display: block; font-size: 13px; font-weight: 500; margin-bottom: 6px; color: var(--td-text-color-primary); }
.required-mark { color: var(--td-error-color); margin-left: 2px; }
/* Destructive-action checkbox — red label, matches the other 3 dialogs. */
.clear-credential :deep(.t-checkbox__label) {
color: var(--td-error-color);
font-size: 13px;
}
.form-tip { font-size: 12px; color: var(--td-text-color-placeholder); margin: 4px 0 12px; }
.form-hint { font-size: 12px; color: var(--td-text-color-placeholder); margin-top: 6px; line-height: 1.5; }
.form-actions { display: flex; align-items: center; gap: 8px; margin-top: 12px; }