fix: redact sensitive fields and prevent credential loss on edit

Closes #988

Unify the "write-only secrets" pattern across MCP services, Models,
WebSearch providers, and DataSources. Fix two related defects:

1. GET responses leaked plaintext credentials — GetMCPServiceByID
   while List masked, all Model / WebSearchProvider / DataSource GET
   responses unconditionally.
2. When the frontend echoed a masked placeholder back on save, the
   update path overwrote the stored secret with the mask — silently
   wiping the real credential. MCP surfaced this most visibly:
   ListMCPServices masked Token to "first4****last4" and
   McpServiceDialog round-tripped that masked string back into the DB.

- internal/types/secret.go introduces the shared sentinel
  RedactedSecretPlaceholder = "***" and two helpers:
  - IsRedactedOrEmpty(s): true for "" or the sentinel
  - PreserveIfRedacted(incoming, existing): merge predicate
- Each resource adds a RedactSensitiveData (mutates-in-place) or
  Redacted (returns-a-copy) method and a MergeUpdate contract:
  empty or placeholder preserves, any other value replaces, and a
  write-only Clear* bool flag (gorm:"-", json omitempty) wipes with
  precedence over preserve / replace. Clear* never persists.
- Every GET / List / Create / Update response path now redacts.
  VectorStore's pre-existing "***" literal is refactored onto the
  shared constant for consistency.

All four dialogs — McpServiceDialog, ModelEditorDialog,
WebSearchSettings, DataSourceEditorDialog — never pre-fill secret
inputs. The stored-state signal is collapsed into the input
placeholder: "•••••••• (Enter new value to replace)" when the server
reports a stored value, falling back to the connector-specific
placeholder otherwise. The input itself carries the "something is
there" cue, matching how password dots render.

An opt-in "Remove this credential" checkbox appears in edit mode when
a value is stored; checking it disables the input and triggers a
danger-themed confirmation dialog before submit. DataSource rolls the
clear up to a connector-level "Remove all credentials" flag since its
credentials are per-connector atomic bundles (e.g. Feishu app_id +
app_secret).

i18n updates across en-US, zh-CN, ko-KR, ru-RU:
secret.storedPlaceholder / clearHint / confirmClearTitle /
confirmClearBody.

- MCP update handler's manual map-to-struct projection was silently
  dropping clear_api_key / clear_token / custom_headers on partial
  updates. Pull them off the map explicitly.
- MCP configChanged detection now snapshots URL / StdioConfig /
  TransportType before the merge so connection recycling still
  triggers on real transport changes (not just whenever AuthConfig
  is present).
- MCP CustomHeaders: nil preserves existing, non-nil replaces. Was
  always overwriting, silently wiping headers on any auth-bearing
  update.
- GetMCPServiceByID now redacts (was plaintext while List masked).
- WebSearchProvider update preserves Name / Description when empty
  instead of clobbering them.
- WebSearchProvider TestProviderRaw rejects the "***" placeholder up
  front with an actionable message rather than forwarding it to the
  upstream provider API as a literal key.
- Model builtin hideSensitiveInfo now preserves non-secret metadata
  (SupportsVision, Provider, ExtraConfig, InterfaceType,
  ParameterSize, EmbeddingParameters) that were previously zeroed
  out for builtin model cards.
- Replace local equalStringSlices / equalStringMaps helpers with
  stdlib slices.Equal / maps.Equal (Go 1.21+).

- internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go:
  table-driven coverage of merge, redact, clear, and write-only flag
  invariants for every resource.
- internal/application/service/mcp_service_test.go: full service layer
  with a fake repo; covers ListMCPServices / GetMCPServiceByID
  redaction, UpdateMCPService partial-update honoring of Clear*,
  CustomHeaders preserve-on-nil / replace-on-non-nil, and
  clear-flag precedence over submitted values.

Additive only. No changes to request or response JSON shape beyond
the write-only Clear* flags (omitted by default via omitempty). No DB
schema changes. No SDK struct field removals. Existing clients that
don't know about the new flags continue to function unchanged.

- DataSource.Config jsonb at-rest encryption. Pre-existing residual
  risk; tracked as a follow-up.
- Renaming VectorStore.MaskSensitiveFields to match the new
  RedactSensitiveData / Redacted naming convention. Left alone to
  minimize diff; a follow-up rename PR can unify.
This commit is contained in:
ochan.kwon
2026-04-18 21:48:50 +09:00
committed by lyingbug
parent a3411899cf
commit d869cc1ff6
32 changed files with 2039 additions and 130 deletions
+6
View File
@@ -114,6 +114,7 @@ func (h *DataSourceHandler) CreateDataSource(c *gin.Context) {
return
}
ds.RedactSensitiveData()
c.JSON(http.StatusCreated, ds)
}
@@ -142,6 +143,7 @@ func (h *DataSourceHandler) GetDataSource(c *gin.Context) {
return
}
ds.RedactSensitiveData()
c.JSON(http.StatusOK, ds)
}
@@ -179,6 +181,9 @@ func (h *DataSourceHandler) ListDataSources(c *gin.Context) {
if dataSources == nil {
dataSources = make([]*types.DataSource, 0)
}
for _, ds := range dataSources {
ds.RedactSensitiveData()
}
c.JSON(http.StatusOK, dataSources)
}
@@ -224,6 +229,7 @@ func (h *DataSourceHandler) UpdateDataSource(c *gin.Context) {
return
}
ds.RedactSensitiveData()
c.JSON(http.StatusOK, ds)
}
+30
View File
@@ -81,6 +81,9 @@ func (h *MCPServiceHandler) CreateMCPService(c *gin.Context) {
return
}
// Redact before echoing so the Create response honors the same
// "secrets never leave the server" invariant as List/Get/Update.
service.RedactSensitiveData()
c.JSON(http.StatusOK, gin.H{
"success": true,
"data": service,
@@ -278,6 +281,29 @@ func (h *MCPServiceHandler) UpdateMCPService(c *gin.Context) {
if token, ok := authConfig["token"].(string); ok {
service.AuthConfig.Token = token
}
// Write-only clear flags — must be explicitly pulled off the map
// since this handler intentionally uses a manual map-to-struct
// projection to support partial updates (see "Use map to handle
// partial updates" comment above). A struct-based bind would have
// picked these up automatically.
if clearAPIKey, ok := authConfig["clear_api_key"].(bool); ok {
service.AuthConfig.ClearAPIKey = clearAPIKey
}
if clearToken, ok := authConfig["clear_token"].(bool); ok {
service.AuthConfig.ClearToken = clearToken
}
// CustomHeaders — nil preserves, non-nil replaces (see
// MCPAuthConfig.MergeUpdate). Only copy when the key is present so
// the preserve-on-nil branch is reachable.
if customHeaders, ok := authConfig["custom_headers"].(map[string]interface{}); ok {
headers := make(map[string]string, len(customHeaders))
for k, v := range customHeaders {
if s, ok := v.(string); ok {
headers[k] = s
}
}
service.AuthConfig.CustomHeaders = headers
}
}
if advancedConfig, ok := updateData["advanced_config"].(map[string]interface{}); ok {
service.AdvancedConfig = &types.MCPAdvancedConfig{}
@@ -299,6 +325,10 @@ func (h *MCPServiceHandler) UpdateMCPService(c *gin.Context) {
}
logger.Infof(ctx, "MCP service updated successfully: %s", secutils.SanitizeForLog(serviceID))
// Redact before echoing so the Update response honors the same
// "secrets never leave the server" invariant as List/Get/Create, and so
// write-only Clear* flags from the request body are never echoed back.
service.RedactSensitiveData()
c.JSON(http.StatusOK, gin.H{
"success": true,
"data": service,
+61 -31
View File
@@ -30,34 +30,54 @@ func NewModelHandler(service interfaces.ModelService) *ModelHandler {
return &ModelHandler{service: service}
}
// hideSensitiveInfo hides sensitive information (APIKey, BaseURL) for builtin models
// Returns a copy of the model with sensitive fields cleared if it's a builtin model
// hideSensitiveInfo prepares a Model for inclusion in a response body.
//
// For builtin models, APIKey and BaseURL are stripped entirely and only the
// non-credential fields (embedding dimensions, parameter size) survive on a
// fresh copy — builtin models are shared across tenants and must never leak
// their underlying provider configuration.
//
// For tenant-owned models the sensitive fields (APIKey, AppSecret) are
// replaced by the shared RedactedSecretPlaceholder via RedactSensitiveData,
// on a copy so the in-memory representation stays usable by subsequent
// code paths. Empty values stay empty so the frontend can render "set vs
// not set" without guesswork.
func hideSensitiveInfo(model *types.Model) *types.Model {
if !model.IsBuiltin {
return model
}
// Create a copy with sensitive information hidden
return &types.Model{
ID: model.ID,
TenantID: model.TenantID,
Name: model.Name,
Type: model.Type,
Source: model.Source,
Description: model.Description,
Parameters: types.ModelParameters{
// Hide APIKey and BaseURL for builtin models
BaseURL: "",
APIKey: "",
// Keep other parameters like embedding dimensions
EmbeddingParameters: model.Parameters.EmbeddingParameters,
ParameterSize: model.Parameters.ParameterSize,
},
IsBuiltin: model.IsBuiltin,
Status: model.Status,
CreatedAt: model.CreatedAt,
UpdatedAt: model.UpdatedAt,
if model.IsBuiltin {
return &types.Model{
ID: model.ID,
TenantID: model.TenantID,
Name: model.Name,
Type: model.Type,
Source: model.Source,
Description: model.Description,
Parameters: types.ModelParameters{
// Hide APIKey and BaseURL for builtin models.
BaseURL: "",
APIKey: "",
// Preserve non-credential fields so the frontend can still
// render correct capability metadata (vision support, Ollama
// parameter size, interface type, provider identifier, and
// any embedding dimensions). These were previously zeroed
// out, which caused builtin chat cards to look like their
// multimodal / provider metadata was unset.
EmbeddingParameters: model.Parameters.EmbeddingParameters,
ParameterSize: model.Parameters.ParameterSize,
InterfaceType: model.Parameters.InterfaceType,
Provider: model.Parameters.Provider,
ExtraConfig: model.Parameters.ExtraConfig,
SupportsVision: model.Parameters.SupportsVision,
},
IsBuiltin: model.IsBuiltin,
Status: model.Status,
CreatedAt: model.CreatedAt,
UpdatedAt: model.UpdatedAt,
}
}
// Tenant-owned model: redact secrets on a copy, preserving everything else.
cp := *model
cp.RedactSensitiveData()
return &cp
}
// CreateModelRequest defines the structure for model creation requests
@@ -310,12 +330,22 @@ func (h *ModelHandler) UpdateModel(c *gin.Context) {
return
}
}
// Preserve backend-managed fields not sent by frontend
req.Parameters.ParameterSize = model.Parameters.ParameterSize
if req.Parameters.ExtraConfig == nil {
req.Parameters.ExtraConfig = model.Parameters.ExtraConfig
// Apply write-only secret merge semantics before overwriting the stored
// parameters: empty or redacted APIKey/AppSecret preserves the stored
// value, the respective ClearXxx flag wipes it, any other value replaces.
merged := req.Parameters.MergeUpdate(model.Parameters)
// Preserve backend-managed fields not sent by frontend.
merged.ParameterSize = model.Parameters.ParameterSize
if merged.ExtraConfig == nil {
merged.ExtraConfig = model.Parameters.ExtraConfig
}
model.Parameters = req.Parameters
if req.Parameters.ClearAPIKey {
logger.Infof(ctx, "Model API key cleared by user: id=%s", id)
}
if req.Parameters.ClearAppSecret {
logger.Infof(ctx, "Model app_secret cleared by user: id=%s", id)
}
model.Parameters = merged
model.Source = req.Source
model.Type = req.Type
+42 -4
View File
@@ -108,6 +108,7 @@ func (h *WebSearchProviderHandler) CreateProvider(c *gin.Context) {
return
}
provider.RedactSensitiveData()
c.JSON(http.StatusCreated, gin.H{
"success": true,
"data": provider,
@@ -131,6 +132,9 @@ func (h *WebSearchProviderHandler) ListProviders(c *gin.Context) {
return
}
for _, p := range providers {
p.RedactSensitiveData()
}
c.JSON(http.StatusOK, gin.H{
"success": true,
"data": providers,
@@ -166,6 +170,7 @@ func (h *WebSearchProviderHandler) GetProvider(c *gin.Context) {
return
}
provider.RedactSensitiveData()
c.JSON(http.StatusOK, gin.H{
"success": true,
"data": provider,
@@ -212,14 +217,37 @@ func (h *WebSearchProviderHandler) UpdateProvider(c *gin.Context) {
return
}
// Merge incoming parameters into the existing ones using write-only
// secret semantics: empty / redacted APIKey preserves the stored value,
// ClearAPIKey explicitly wipes it, any other value replaces. Non-secret
// fields flow through from the request directly.
mergedParams := req.Parameters.MergeUpdate(existing.Parameters)
if req.Parameters.ClearAPIKey {
logger.Infof(ctx, "WebSearchProvider API key cleared by user: id=%s",
secutils.SanitizeForLog(id))
}
// Preserve existing values for top-level metadata fields when the
// request omits them (empty string from the JSON decoder). Without this,
// a partial update that only flips IsDefault would clobber Name and
// Description on the stored record.
mergedName := req.Name
if mergedName == "" {
mergedName = existing.Name
}
mergedDescription := req.Description
if mergedDescription == "" {
mergedDescription = existing.Description
}
// Build updated entity, keeping immutable fields from existing
provider := &types.WebSearchProviderEntity{
ID: id,
TenantID: tenantID,
Name: secutils.SanitizeForLog(req.Name),
Name: secutils.SanitizeForLog(mergedName),
Provider: existing.Provider, // Provider type is immutable after creation
Description: secutils.SanitizeForLog(req.Description),
Parameters: req.Parameters,
Description: secutils.SanitizeForLog(mergedDescription),
Parameters: mergedParams,
IsDefault: req.IsDefault,
}
@@ -232,6 +260,7 @@ func (h *WebSearchProviderHandler) UpdateProvider(c *gin.Context) {
// Re-fetch to get the full stored state
updated, _ := h.repo.GetByID(ctx, tenantID, id)
if updated != nil {
updated.RedactSensitiveData()
c.JSON(http.StatusOK, gin.H{"success": true, "data": updated})
} else {
c.JSON(http.StatusOK, gin.H{"success": true})
@@ -371,8 +400,17 @@ func (h *WebSearchProviderHandler) TestProviderRaw(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"success": true})
}
// doTestSearch creates a temporary provider and runs a simple test query
// doTestSearch creates a temporary provider and runs a simple test query.
//
// The provider would otherwise try to authenticate against the upstream API
// with the redacted placeholder (which is guaranteed to fail with a
// confusing error). Reject it up front with an actionable message so the
// user knows they should type a real key or test against the saved config
// via /test instead.
func (h *WebSearchProviderHandler) doTestSearch(ctx context.Context, providerType string, params types.WebSearchProviderParameters) error {
if params.APIKey == types.RedactedSecretPlaceholder {
return fmt.Errorf("api_key looks like the redacted placeholder; enter your real key or call POST /api/v1/web-search-providers/{id}/test to exercise the saved credential")
}
logger.Infof(ctx, "[WebSearch][Test] testing provider type=%s", providerType)
searchProvider, err := h.registry.CreateProvider(providerType, params)
if err != nil {