refactor(credentials): introduce /credentials subresource pattern

Replace the legacy "redacted placeholder + Clear* boolean" pattern with
dedicated per-resource credential subresources across MCP services,
Models, Web search providers, and Data sources.

Why
---
The previous design had three problems:

1. Main PUT body carried secret fields. The frontend echoed back a
   redacted "***" placeholder, and a fragile MergeUpdate / IsRedactedOrEmpty
   defense in the service layer tried to detect "user did not change this"
   vs "user wants to clear this". A regression in that defense (or a new
   frontend forgetting it) silently overwrites the stored secret with the
   placeholder.

2. The "remove this credential" UX was a red checkbox under a pre-filled
   password input. Three intents (preserve / replace / clear) collapsed
   onto one field, and credential changes were bundled with unrelated
   config edits in the same submit. Users wiped working keys by mistake.

3. Secret presence was inferred from "did the response come back with a
   '***' placeholder", which couples the contract to a magic string.

Design
------
Each of the four resources now exposes:

  PUT    /{resource}/{id}/credentials          # write one or more fields
  DELETE /{resource}/{id}/credentials/{field}  # clear a single field

"Is this configured?" metadata travels on the main resource response as
a typed map (dto.MCPServiceResponse.Credentials etc.) — no separate GET
endpoint. The frontend reads the configured boolean from the main GET
and never sees secret values at all.

Main PUT endpoints now ignore any secret fields in the request body and
log a deprecation warning if they appear, so legacy clients fail loudly
rather than overwriting silently.

Frontend
--------
- New reusable <CredentialResource> component renders a three-state card
  (unconfigured / configured / editing) and drives the dedicated
  endpoints. Used by MCP, Model, Web search; DataSource has a bespoke
  card with the same behaviour because its credentials are a single
  atomic per-connector map.
- Cancel from edit mode now restores state synchronously from the meta
  prop. The previous async refresh() was a no-op while state was still
  'editing', leaving the input frozen open.
- Remove is single-click + toast. The danger-themed button is the
  deterrent; a modal confirm adds friction without adding safety (the
  plaintext is irrecoverable client-side either way — recovery means
  re-typing).

DTOs (internal/handler/dto/) are deliberately separate from the GORM
models so "no secret in response" is a compile-time invariant: a future
contributor cannot leak a secret without explicitly adding the field to
the DTO, which is review-able in a single diff.

Storage is unchanged — credentials still live in the existing jsonb /
parameters columns. No schema migration.

Cleanup
-------
- types.MCPService / types.Model / types.WebSearchProviderEntity /
  types.DataSourceConfig: drop ClearAPIKey / ClearToken / ClearAppSecret
  / ClearCredentials boolean fields, MergeUpdate(), RedactSensitiveData().
- utils/types/secret.go: drop PreserveIfRedacted / IsRedactedOrEmpty.
  RedactedSecretPlaceholder constant is retained because VectorStore
  still uses the old pattern and is out of scope here.
- Frontend hasExistingApiKey / clearApiKey / convertToLegacyFormat
  redaction handling removed; i18n keys renamed secret -> credential.
This commit is contained in:
wizardchen
2026-05-17 15:09:30 +08:00
committed by lyingbug
parent f30e9ccd0f
commit 7643dd457e
53 changed files with 3271 additions and 2347 deletions
@@ -155,31 +155,27 @@ func (s *DataSourceService) UpdateDataSource(ctx context.Context, ds *types.Data
return nil, datasource.ErrDataSourceInvalid
}
// Apply write-only secret merge semantics to the Config jsonb:
// empty / redacted credential strings preserve existing values,
// ClearCredentials wipes the whole map, other values replace. This
// blocks the NELO-1299-style round-trip bug where a frontend that
// echoed back the redacted placeholder would overwrite real secrets.
//
// We also remember the parsed configs for the post-merge "changed?"
// comparison — a raw-bytes compare against ds.Config is unreliable
// because Go map iteration order makes the re-serialized JSON's key
// ordering nondeterministic, causing string(merged) != string(existing)
// to spuriously trip every time.
// Credentials NEVER flow through this endpoint — they live behind the
// /credentials subresource. Force-preserve the stored credentials map
// regardless of what the body says. Log a warning if a stale caller
// passes one so we can spot them and migrate later. Non-credential
// fields of Config (Type / ResourceIDs / Settings) flow through.
var mergedCfg, existingParsedCfg *types.DataSourceConfig
if len(ds.Config) > 0 {
incomingCfg, parseIncErr := ds.ParseConfig()
existingCfg, parseExErr := existing.ParseConfig()
if parseIncErr == nil && parseExErr == nil && incomingCfg != nil {
baseExisting := types.DataSourceConfig{}
if existingCfg != nil {
baseExisting = *existingCfg
}
if incomingCfg.ClearCredentials {
logger.Infof(ctx, "DataSource credentials cleared by user: id=%s",
if incomingCfg.HasCredentials() {
logger.Warnf(ctx,
"deprecated: credentials in PUT /datasource/%s body are ignored; use PUT /credentials instead",
secutils.SanitizeForLog(ds.ID))
}
merged := incomingCfg.MergeUpdate(baseExisting)
merged := *incomingCfg
if existingCfg != nil {
merged.Credentials = existingCfg.Credentials
} else {
merged.Credentials = nil
}
if blob, err := merged.ToJSON(); err == nil {
ds.Config = blob
}
@@ -188,22 +184,16 @@ func (s *DataSourceService) UpdateDataSource(ctx context.Context, ds *types.Data
}
}
// Validate new configuration if changed. Compare parsed structures to
// avoid re-running (potentially expensive) OAuth / live-connection
// validators on every update that only re-orders JSON keys.
//
// Skip validation entirely when the user is explicitly revoking
// credentials (ClearCredentials=true). Live validators (e.g. Notion's
// /me API) require a working secret by definition; trying to validate
// after a wipe would always fail and leave the row un-cleared, defeating
// the whole point of the "Remove all credentials" affordance.
// Validate new configuration if non-credential fields changed. Skip
// when there are no stored credentials yet (validators would fail with
// no token to call the live API) and when the parsed config is
// structurally identical.
configActuallyChanged := true
if mergedCfg != nil && existingParsedCfg != nil {
configActuallyChanged = !reflect.DeepEqual(*mergedCfg, *existingParsedCfg)
}
clearingCredentials := mergedCfg != nil && !mergedCfg.HasCredentials() &&
existingParsedCfg != nil && existingParsedCfg.HasCredentials()
if !clearingCredentials && (ds.Type != existing.Type || configActuallyChanged) {
hasCreds := mergedCfg != nil && mergedCfg.HasCredentials()
if hasCreds && (ds.Type != existing.Type || configActuallyChanged) {
if err := s.validateDataSourceConfig(ctx, ds); err != nil {
return nil, err
}
@@ -223,6 +213,78 @@ func (s *DataSourceService) UpdateDataSource(ctx context.Context, ds *types.Data
return ds, nil
}
// UpdateDataSourceCredentials replaces the connector credential map. This is
// a single atomic write; the previous credential set is discarded entirely
// (callers cannot patch individual keys because half-configured connector
// auth is meaningless). After persisting, the live connection is validated
// so the caller learns immediately if the new credentials are wrong.
func (s *DataSourceService) UpdateDataSourceCredentials(
ctx context.Context, id string, credentials map[string]interface{},
) (*types.DataSource, error) {
if id == "" {
return nil, datasource.ErrDataSourceInvalid
}
existing, err := s.dsRepo.FindByID(ctx, id)
if err != nil {
return nil, err
}
parsed, err := existing.ParseConfig()
if err != nil {
return nil, err
}
if parsed == nil {
parsed = &types.DataSourceConfig{Type: existing.Type}
}
parsed.Credentials = credentials
blob, err := parsed.ToJSON()
if err != nil {
return nil, err
}
existing.Config = blob
// Run live validation now that the credentials are in place — surfaces
// "wrong token" feedback immediately to the user instead of waiting for
// the next scheduled sync.
if err := s.validateDataSourceConfig(ctx, existing); err != nil {
return nil, err
}
if err := s.dsRepo.Update(ctx, existing); err != nil {
return nil, err
}
logger.Infof(ctx, "DataSource credentials updated: id=%s", secutils.SanitizeForLog(id))
return existing, nil
}
// ClearDataSourceCredentials wipes the connector credential map without
// touching any other config field. Idempotent.
func (s *DataSourceService) ClearDataSourceCredentials(ctx context.Context, id string) error {
if id == "" {
return datasource.ErrDataSourceInvalid
}
existing, err := s.dsRepo.FindByID(ctx, id)
if err != nil {
return err
}
parsed, err := existing.ParseConfig()
if err != nil {
return err
}
if parsed == nil || !parsed.HasCredentials() {
return nil
}
parsed.Credentials = nil
blob, err := parsed.ToJSON()
if err != nil {
return err
}
existing.Config = blob
if err := s.dsRepo.Update(ctx, existing); err != nil {
return err
}
logger.Infof(ctx, "DataSource credentials cleared by user: id=%s", secutils.SanitizeForLog(id))
return nil
}
// DeleteDataSource deletes a data source (soft delete)
func (s *DataSourceService) DeleteDataSource(ctx context.Context, id string) error {
// Verify data source exists