fix: redact sensitive fields and prevent credential loss on edit
Closes #988 Unify the "write-only secrets" pattern across MCP services, Models, WebSearch providers, and DataSources. Fix two related defects: 1. GET responses leaked plaintext credentials — GetMCPServiceByID while List masked, all Model / WebSearchProvider / DataSource GET responses unconditionally. 2. When the frontend echoed a masked placeholder back on save, the update path overwrote the stored secret with the mask — silently wiping the real credential. MCP surfaced this most visibly: ListMCPServices masked Token to "first4****last4" and McpServiceDialog round-tripped that masked string back into the DB. - internal/types/secret.go introduces the shared sentinel RedactedSecretPlaceholder = "***" and two helpers: - IsRedactedOrEmpty(s): true for "" or the sentinel - PreserveIfRedacted(incoming, existing): merge predicate - Each resource adds a RedactSensitiveData (mutates-in-place) or Redacted (returns-a-copy) method and a MergeUpdate contract: empty or placeholder preserves, any other value replaces, and a write-only Clear* bool flag (gorm:"-", json omitempty) wipes with precedence over preserve / replace. Clear* never persists. - Every GET / List / Create / Update response path now redacts. VectorStore's pre-existing "***" literal is refactored onto the shared constant for consistency. All four dialogs — McpServiceDialog, ModelEditorDialog, WebSearchSettings, DataSourceEditorDialog — never pre-fill secret inputs. The stored-state signal is collapsed into the input placeholder: "•••••••• (Enter new value to replace)" when the server reports a stored value, falling back to the connector-specific placeholder otherwise. The input itself carries the "something is there" cue, matching how password dots render. An opt-in "Remove this credential" checkbox appears in edit mode when a value is stored; checking it disables the input and triggers a danger-themed confirmation dialog before submit. DataSource rolls the clear up to a connector-level "Remove all credentials" flag since its credentials are per-connector atomic bundles (e.g. Feishu app_id + app_secret). i18n updates across en-US, zh-CN, ko-KR, ru-RU: secret.storedPlaceholder / clearHint / confirmClearTitle / confirmClearBody. - MCP update handler's manual map-to-struct projection was silently dropping clear_api_key / clear_token / custom_headers on partial updates. Pull them off the map explicitly. - MCP configChanged detection now snapshots URL / StdioConfig / TransportType before the merge so connection recycling still triggers on real transport changes (not just whenever AuthConfig is present). - MCP CustomHeaders: nil preserves existing, non-nil replaces. Was always overwriting, silently wiping headers on any auth-bearing update. - GetMCPServiceByID now redacts (was plaintext while List masked). - WebSearchProvider update preserves Name / Description when empty instead of clobbering them. - WebSearchProvider TestProviderRaw rejects the "***" placeholder up front with an actionable message rather than forwarding it to the upstream provider API as a literal key. - Model builtin hideSensitiveInfo now preserves non-secret metadata (SupportsVision, Provider, ExtraConfig, InterfaceType, ParameterSize, EmbeddingParameters) that were previously zeroed out for builtin model cards. - Replace local equalStringSlices / equalStringMaps helpers with stdlib slices.Equal / maps.Equal (Go 1.21+). - internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go: table-driven coverage of merge, redact, clear, and write-only flag invariants for every resource. - internal/application/service/mcp_service_test.go: full service layer with a fake repo; covers ListMCPServices / GetMCPServiceByID redaction, UpdateMCPService partial-update honoring of Clear*, CustomHeaders preserve-on-nil / replace-on-non-nil, and clear-flag precedence over submitted values. Additive only. No changes to request or response JSON shape beyond the write-only Clear* flags (omitted by default via omitempty). No DB schema changes. No SDK struct field removals. Existing clients that don't know about the new flags continue to function unchanged. - DataSource.Config jsonb at-rest encryption. Pre-existing residual risk; tracked as a follow-up. - Renaming VectorStore.MaskSensitiveFields to match the new RedactSensitiveData / Redacted naming convention. Left alone to minimize diff; a follow-up rename PR can unify.
This commit is contained in:
@@ -114,6 +114,7 @@ func (h *DataSourceHandler) CreateDataSource(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
ds.RedactSensitiveData()
|
||||
c.JSON(http.StatusCreated, ds)
|
||||
}
|
||||
|
||||
@@ -142,6 +143,7 @@ func (h *DataSourceHandler) GetDataSource(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
ds.RedactSensitiveData()
|
||||
c.JSON(http.StatusOK, ds)
|
||||
}
|
||||
|
||||
@@ -179,6 +181,9 @@ func (h *DataSourceHandler) ListDataSources(c *gin.Context) {
|
||||
if dataSources == nil {
|
||||
dataSources = make([]*types.DataSource, 0)
|
||||
}
|
||||
for _, ds := range dataSources {
|
||||
ds.RedactSensitiveData()
|
||||
}
|
||||
c.JSON(http.StatusOK, dataSources)
|
||||
}
|
||||
|
||||
@@ -224,6 +229,7 @@ func (h *DataSourceHandler) UpdateDataSource(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
ds.RedactSensitiveData()
|
||||
c.JSON(http.StatusOK, ds)
|
||||
}
|
||||
|
||||
|
||||
@@ -81,6 +81,9 @@ func (h *MCPServiceHandler) CreateMCPService(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
// Redact before echoing so the Create response honors the same
|
||||
// "secrets never leave the server" invariant as List/Get/Update.
|
||||
service.RedactSensitiveData()
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"success": true,
|
||||
"data": service,
|
||||
@@ -278,6 +281,29 @@ func (h *MCPServiceHandler) UpdateMCPService(c *gin.Context) {
|
||||
if token, ok := authConfig["token"].(string); ok {
|
||||
service.AuthConfig.Token = token
|
||||
}
|
||||
// Write-only clear flags — must be explicitly pulled off the map
|
||||
// since this handler intentionally uses a manual map-to-struct
|
||||
// projection to support partial updates (see "Use map to handle
|
||||
// partial updates" comment above). A struct-based bind would have
|
||||
// picked these up automatically.
|
||||
if clearAPIKey, ok := authConfig["clear_api_key"].(bool); ok {
|
||||
service.AuthConfig.ClearAPIKey = clearAPIKey
|
||||
}
|
||||
if clearToken, ok := authConfig["clear_token"].(bool); ok {
|
||||
service.AuthConfig.ClearToken = clearToken
|
||||
}
|
||||
// CustomHeaders — nil preserves, non-nil replaces (see
|
||||
// MCPAuthConfig.MergeUpdate). Only copy when the key is present so
|
||||
// the preserve-on-nil branch is reachable.
|
||||
if customHeaders, ok := authConfig["custom_headers"].(map[string]interface{}); ok {
|
||||
headers := make(map[string]string, len(customHeaders))
|
||||
for k, v := range customHeaders {
|
||||
if s, ok := v.(string); ok {
|
||||
headers[k] = s
|
||||
}
|
||||
}
|
||||
service.AuthConfig.CustomHeaders = headers
|
||||
}
|
||||
}
|
||||
if advancedConfig, ok := updateData["advanced_config"].(map[string]interface{}); ok {
|
||||
service.AdvancedConfig = &types.MCPAdvancedConfig{}
|
||||
@@ -299,6 +325,10 @@ func (h *MCPServiceHandler) UpdateMCPService(c *gin.Context) {
|
||||
}
|
||||
|
||||
logger.Infof(ctx, "MCP service updated successfully: %s", secutils.SanitizeForLog(serviceID))
|
||||
// Redact before echoing so the Update response honors the same
|
||||
// "secrets never leave the server" invariant as List/Get/Create, and so
|
||||
// write-only Clear* flags from the request body are never echoed back.
|
||||
service.RedactSensitiveData()
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"success": true,
|
||||
"data": service,
|
||||
|
||||
+61
-31
@@ -30,34 +30,54 @@ func NewModelHandler(service interfaces.ModelService) *ModelHandler {
|
||||
return &ModelHandler{service: service}
|
||||
}
|
||||
|
||||
// hideSensitiveInfo hides sensitive information (APIKey, BaseURL) for builtin models
|
||||
// Returns a copy of the model with sensitive fields cleared if it's a builtin model
|
||||
// hideSensitiveInfo prepares a Model for inclusion in a response body.
|
||||
//
|
||||
// For builtin models, APIKey and BaseURL are stripped entirely and only the
|
||||
// non-credential fields (embedding dimensions, parameter size) survive on a
|
||||
// fresh copy — builtin models are shared across tenants and must never leak
|
||||
// their underlying provider configuration.
|
||||
//
|
||||
// For tenant-owned models the sensitive fields (APIKey, AppSecret) are
|
||||
// replaced by the shared RedactedSecretPlaceholder via RedactSensitiveData,
|
||||
// on a copy so the in-memory representation stays usable by subsequent
|
||||
// code paths. Empty values stay empty so the frontend can render "set vs
|
||||
// not set" without guesswork.
|
||||
func hideSensitiveInfo(model *types.Model) *types.Model {
|
||||
if !model.IsBuiltin {
|
||||
return model
|
||||
}
|
||||
|
||||
// Create a copy with sensitive information hidden
|
||||
return &types.Model{
|
||||
ID: model.ID,
|
||||
TenantID: model.TenantID,
|
||||
Name: model.Name,
|
||||
Type: model.Type,
|
||||
Source: model.Source,
|
||||
Description: model.Description,
|
||||
Parameters: types.ModelParameters{
|
||||
// Hide APIKey and BaseURL for builtin models
|
||||
BaseURL: "",
|
||||
APIKey: "",
|
||||
// Keep other parameters like embedding dimensions
|
||||
EmbeddingParameters: model.Parameters.EmbeddingParameters,
|
||||
ParameterSize: model.Parameters.ParameterSize,
|
||||
},
|
||||
IsBuiltin: model.IsBuiltin,
|
||||
Status: model.Status,
|
||||
CreatedAt: model.CreatedAt,
|
||||
UpdatedAt: model.UpdatedAt,
|
||||
if model.IsBuiltin {
|
||||
return &types.Model{
|
||||
ID: model.ID,
|
||||
TenantID: model.TenantID,
|
||||
Name: model.Name,
|
||||
Type: model.Type,
|
||||
Source: model.Source,
|
||||
Description: model.Description,
|
||||
Parameters: types.ModelParameters{
|
||||
// Hide APIKey and BaseURL for builtin models.
|
||||
BaseURL: "",
|
||||
APIKey: "",
|
||||
// Preserve non-credential fields so the frontend can still
|
||||
// render correct capability metadata (vision support, Ollama
|
||||
// parameter size, interface type, provider identifier, and
|
||||
// any embedding dimensions). These were previously zeroed
|
||||
// out, which caused builtin chat cards to look like their
|
||||
// multimodal / provider metadata was unset.
|
||||
EmbeddingParameters: model.Parameters.EmbeddingParameters,
|
||||
ParameterSize: model.Parameters.ParameterSize,
|
||||
InterfaceType: model.Parameters.InterfaceType,
|
||||
Provider: model.Parameters.Provider,
|
||||
ExtraConfig: model.Parameters.ExtraConfig,
|
||||
SupportsVision: model.Parameters.SupportsVision,
|
||||
},
|
||||
IsBuiltin: model.IsBuiltin,
|
||||
Status: model.Status,
|
||||
CreatedAt: model.CreatedAt,
|
||||
UpdatedAt: model.UpdatedAt,
|
||||
}
|
||||
}
|
||||
// Tenant-owned model: redact secrets on a copy, preserving everything else.
|
||||
cp := *model
|
||||
cp.RedactSensitiveData()
|
||||
return &cp
|
||||
}
|
||||
|
||||
// CreateModelRequest defines the structure for model creation requests
|
||||
@@ -310,12 +330,22 @@ func (h *ModelHandler) UpdateModel(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
}
|
||||
// Preserve backend-managed fields not sent by frontend
|
||||
req.Parameters.ParameterSize = model.Parameters.ParameterSize
|
||||
if req.Parameters.ExtraConfig == nil {
|
||||
req.Parameters.ExtraConfig = model.Parameters.ExtraConfig
|
||||
// Apply write-only secret merge semantics before overwriting the stored
|
||||
// parameters: empty or redacted APIKey/AppSecret preserves the stored
|
||||
// value, the respective ClearXxx flag wipes it, any other value replaces.
|
||||
merged := req.Parameters.MergeUpdate(model.Parameters)
|
||||
// Preserve backend-managed fields not sent by frontend.
|
||||
merged.ParameterSize = model.Parameters.ParameterSize
|
||||
if merged.ExtraConfig == nil {
|
||||
merged.ExtraConfig = model.Parameters.ExtraConfig
|
||||
}
|
||||
model.Parameters = req.Parameters
|
||||
if req.Parameters.ClearAPIKey {
|
||||
logger.Infof(ctx, "Model API key cleared by user: id=%s", id)
|
||||
}
|
||||
if req.Parameters.ClearAppSecret {
|
||||
logger.Infof(ctx, "Model app_secret cleared by user: id=%s", id)
|
||||
}
|
||||
model.Parameters = merged
|
||||
|
||||
model.Source = req.Source
|
||||
model.Type = req.Type
|
||||
|
||||
@@ -108,6 +108,7 @@ func (h *WebSearchProviderHandler) CreateProvider(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
provider.RedactSensitiveData()
|
||||
c.JSON(http.StatusCreated, gin.H{
|
||||
"success": true,
|
||||
"data": provider,
|
||||
@@ -131,6 +132,9 @@ func (h *WebSearchProviderHandler) ListProviders(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
for _, p := range providers {
|
||||
p.RedactSensitiveData()
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"success": true,
|
||||
"data": providers,
|
||||
@@ -166,6 +170,7 @@ func (h *WebSearchProviderHandler) GetProvider(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
provider.RedactSensitiveData()
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"success": true,
|
||||
"data": provider,
|
||||
@@ -212,14 +217,37 @@ func (h *WebSearchProviderHandler) UpdateProvider(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
// Merge incoming parameters into the existing ones using write-only
|
||||
// secret semantics: empty / redacted APIKey preserves the stored value,
|
||||
// ClearAPIKey explicitly wipes it, any other value replaces. Non-secret
|
||||
// fields flow through from the request directly.
|
||||
mergedParams := req.Parameters.MergeUpdate(existing.Parameters)
|
||||
if req.Parameters.ClearAPIKey {
|
||||
logger.Infof(ctx, "WebSearchProvider API key cleared by user: id=%s",
|
||||
secutils.SanitizeForLog(id))
|
||||
}
|
||||
|
||||
// Preserve existing values for top-level metadata fields when the
|
||||
// request omits them (empty string from the JSON decoder). Without this,
|
||||
// a partial update that only flips IsDefault would clobber Name and
|
||||
// Description on the stored record.
|
||||
mergedName := req.Name
|
||||
if mergedName == "" {
|
||||
mergedName = existing.Name
|
||||
}
|
||||
mergedDescription := req.Description
|
||||
if mergedDescription == "" {
|
||||
mergedDescription = existing.Description
|
||||
}
|
||||
|
||||
// Build updated entity, keeping immutable fields from existing
|
||||
provider := &types.WebSearchProviderEntity{
|
||||
ID: id,
|
||||
TenantID: tenantID,
|
||||
Name: secutils.SanitizeForLog(req.Name),
|
||||
Name: secutils.SanitizeForLog(mergedName),
|
||||
Provider: existing.Provider, // Provider type is immutable after creation
|
||||
Description: secutils.SanitizeForLog(req.Description),
|
||||
Parameters: req.Parameters,
|
||||
Description: secutils.SanitizeForLog(mergedDescription),
|
||||
Parameters: mergedParams,
|
||||
IsDefault: req.IsDefault,
|
||||
}
|
||||
|
||||
@@ -232,6 +260,7 @@ func (h *WebSearchProviderHandler) UpdateProvider(c *gin.Context) {
|
||||
// Re-fetch to get the full stored state
|
||||
updated, _ := h.repo.GetByID(ctx, tenantID, id)
|
||||
if updated != nil {
|
||||
updated.RedactSensitiveData()
|
||||
c.JSON(http.StatusOK, gin.H{"success": true, "data": updated})
|
||||
} else {
|
||||
c.JSON(http.StatusOK, gin.H{"success": true})
|
||||
@@ -371,8 +400,17 @@ func (h *WebSearchProviderHandler) TestProviderRaw(c *gin.Context) {
|
||||
c.JSON(http.StatusOK, gin.H{"success": true})
|
||||
}
|
||||
|
||||
// doTestSearch creates a temporary provider and runs a simple test query
|
||||
// doTestSearch creates a temporary provider and runs a simple test query.
|
||||
//
|
||||
// The provider would otherwise try to authenticate against the upstream API
|
||||
// with the redacted placeholder (which is guaranteed to fail with a
|
||||
// confusing error). Reject it up front with an actionable message so the
|
||||
// user knows they should type a real key or test against the saved config
|
||||
// via /test instead.
|
||||
func (h *WebSearchProviderHandler) doTestSearch(ctx context.Context, providerType string, params types.WebSearchProviderParameters) error {
|
||||
if params.APIKey == types.RedactedSecretPlaceholder {
|
||||
return fmt.Errorf("api_key looks like the redacted placeholder; enter your real key or call POST /api/v1/web-search-providers/{id}/test to exercise the saved credential")
|
||||
}
|
||||
logger.Infof(ctx, "[WebSearch][Test] testing provider type=%s", providerType)
|
||||
searchProvider, err := h.registry.CreateProvider(providerType, params)
|
||||
if err != nil {
|
||||
|
||||
Reference in New Issue
Block a user