fix: redact sensitive fields and prevent credential loss on edit

Closes #988

Unify the "write-only secrets" pattern across MCP services, Models,
WebSearch providers, and DataSources. Fix two related defects:

1. GET responses leaked plaintext credentials — GetMCPServiceByID
   while List masked, all Model / WebSearchProvider / DataSource GET
   responses unconditionally.
2. When the frontend echoed a masked placeholder back on save, the
   update path overwrote the stored secret with the mask — silently
   wiping the real credential. MCP surfaced this most visibly:
   ListMCPServices masked Token to "first4****last4" and
   McpServiceDialog round-tripped that masked string back into the DB.

- internal/types/secret.go introduces the shared sentinel
  RedactedSecretPlaceholder = "***" and two helpers:
  - IsRedactedOrEmpty(s): true for "" or the sentinel
  - PreserveIfRedacted(incoming, existing): merge predicate
- Each resource adds a RedactSensitiveData (mutates-in-place) or
  Redacted (returns-a-copy) method and a MergeUpdate contract:
  empty or placeholder preserves, any other value replaces, and a
  write-only Clear* bool flag (gorm:"-", json omitempty) wipes with
  precedence over preserve / replace. Clear* never persists.
- Every GET / List / Create / Update response path now redacts.
  VectorStore's pre-existing "***" literal is refactored onto the
  shared constant for consistency.

All four dialogs — McpServiceDialog, ModelEditorDialog,
WebSearchSettings, DataSourceEditorDialog — never pre-fill secret
inputs. The stored-state signal is collapsed into the input
placeholder: "•••••••• (Enter new value to replace)" when the server
reports a stored value, falling back to the connector-specific
placeholder otherwise. The input itself carries the "something is
there" cue, matching how password dots render.

An opt-in "Remove this credential" checkbox appears in edit mode when
a value is stored; checking it disables the input and triggers a
danger-themed confirmation dialog before submit. DataSource rolls the
clear up to a connector-level "Remove all credentials" flag since its
credentials are per-connector atomic bundles (e.g. Feishu app_id +
app_secret).

i18n updates across en-US, zh-CN, ko-KR, ru-RU:
secret.storedPlaceholder / clearHint / confirmClearTitle /
confirmClearBody.

- MCP update handler's manual map-to-struct projection was silently
  dropping clear_api_key / clear_token / custom_headers on partial
  updates. Pull them off the map explicitly.
- MCP configChanged detection now snapshots URL / StdioConfig /
  TransportType before the merge so connection recycling still
  triggers on real transport changes (not just whenever AuthConfig
  is present).
- MCP CustomHeaders: nil preserves existing, non-nil replaces. Was
  always overwriting, silently wiping headers on any auth-bearing
  update.
- GetMCPServiceByID now redacts (was plaintext while List masked).
- WebSearchProvider update preserves Name / Description when empty
  instead of clobbering them.
- WebSearchProvider TestProviderRaw rejects the "***" placeholder up
  front with an actionable message rather than forwarding it to the
  upstream provider API as a literal key.
- Model builtin hideSensitiveInfo now preserves non-secret metadata
  (SupportsVision, Provider, ExtraConfig, InterfaceType,
  ParameterSize, EmbeddingParameters) that were previously zeroed
  out for builtin model cards.
- Replace local equalStringSlices / equalStringMaps helpers with
  stdlib slices.Equal / maps.Equal (Go 1.21+).

- internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go:
  table-driven coverage of merge, redact, clear, and write-only flag
  invariants for every resource.
- internal/application/service/mcp_service_test.go: full service layer
  with a fake repo; covers ListMCPServices / GetMCPServiceByID
  redaction, UpdateMCPService partial-update honoring of Clear*,
  CustomHeaders preserve-on-nil / replace-on-non-nil, and
  clear-flag precedence over submitted values.

Additive only. No changes to request or response JSON shape beyond
the write-only Clear* flags (omitted by default via omitempty). No DB
schema changes. No SDK struct field removals. Existing clients that
don't know about the new flags continue to function unchanged.

- DataSource.Config jsonb at-rest encryption. Pre-existing residual
  risk; tracked as a follow-up.
- Renaming VectorStore.MaskSensitiveFields to match the new
  RedactSensitiveData / Redacted naming convention. Left alone to
  minimize diff; a follow-up rename PR can unify.
This commit is contained in:
ochan.kwon
2026-04-18 21:48:50 +09:00
committed by lyingbug
parent a3411899cf
commit d869cc1ff6
32 changed files with 2039 additions and 130 deletions
+101 -1
View File
@@ -194,7 +194,15 @@ func (s *SyncLog) BeforeCreate(tx *gorm.DB) error {
}
// DataSourceConfig represents the unencrypted configuration structure
// Each connector type will have its own specific fields
// Each connector type will have its own specific fields.
//
// ClearCredentials is a write-only flag used in Update requests to
// explicitly remove the stored credential map. Because credentials are
// per-connector atomic sets (e.g. an OAuth token pair, a GitHub PAT, a
// Confluence API token), individual-field removal is intentionally not
// supported — revoking one field in isolation would leave the connector in
// a broken half-configured state. The flag is never persisted to storage
// and never surfaces in responses (omitempty + zero value).
type DataSourceConfig struct {
// Common fields applicable to most connectors
Type string `json:"type"`
@@ -207,6 +215,98 @@ type DataSourceConfig struct {
// Connector-specific configuration
Settings map[string]interface{} `json:"settings"`
// Write-only flag — request-only, never persisted or returned.
ClearCredentials bool `json:"clear_credentials,omitempty"`
}
// HasCredentials reports whether the credentials map carries any value at
// all. Used by the Update path to detect the "Remove all credentials"
// transition (existing.HasCredentials() && !merged.HasCredentials()) and
// skip live-connector validation that would otherwise reject the wipe.
func (d DataSourceConfig) HasCredentials() bool {
return len(d.Credentials) > 0
}
// Redacted returns a copy of the config with every string value in
// Credentials replaced by RedactedSecretPlaceholder so it can be returned to
// the client without exposing stored secrets. Empty string values stay empty
// so the frontend can still distinguish "set (hidden)" from "not set" per
// key. Non-string credential values (rare but possible for numeric IDs or
// structured settings) pass through unchanged.
//
// Named "Redacted" rather than "RedactSensitiveData" because it returns a
// copy instead of mutating in place, matching the naming convention of
// ConnectionConfig.MaskSensitiveFields in types/vectorstore.go.
func (d DataSourceConfig) Redacted() DataSourceConfig {
cp := d
// ClearCredentials is a request-only signal; never echo it in responses.
cp.ClearCredentials = false
if d.Credentials == nil {
return cp
}
redacted := make(map[string]interface{}, len(d.Credentials))
for k, v := range d.Credentials {
if s, ok := v.(string); ok && s != "" {
redacted[k] = RedactedSecretPlaceholder
} else {
redacted[k] = v
}
}
cp.Credentials = redacted
return cp
}
// MergeUpdate applies write-only secret merge semantics for Update requests.
//
// - ClearCredentials=true → wipe the entire credentials map
// - Otherwise, each string credential key is merged individually using
// PreserveIfRedacted: empty or the redacted placeholder preserves the
// existing value, any other value replaces. Non-string credential
// values from incoming always overwrite (they aren't redacted either).
// - Non-secret fields (Type, ResourceIDs, Settings) flow from incoming
// directly.
// - The write-only ClearCredentials flag is cleared on the merged result.
func (d DataSourceConfig) MergeUpdate(existing DataSourceConfig) DataSourceConfig {
merged := d
merged.ClearCredentials = false
if d.ClearCredentials {
merged.Credentials = nil
return merged
}
mergedCreds := make(map[string]interface{}, len(existing.Credentials)+len(d.Credentials))
for k, v := range existing.Credentials {
mergedCreds[k] = v
}
for k, v := range d.Credentials {
if s, ok := v.(string); ok {
if IsRedactedOrEmpty(s) {
continue // preserve existing value for this key
}
}
mergedCreds[k] = v
}
merged.Credentials = mergedCreds
return merged
}
// RedactSensitiveData parses the Config jsonb, redacts string values in the
// Credentials map, and writes the result back. No-op when Config is empty or
// fails to parse (the caller continues to return the DataSource with
// whatever Config representation it has — better than dropping the response).
func (d *DataSource) RedactSensitiveData() {
if len(d.Config) == 0 {
return
}
parsed, err := d.ParseConfig()
if err != nil || parsed == nil {
return
}
redacted := parsed.Redacted()
if blob, err := redacted.ToJSON(); err == nil {
d.Config = blob
}
}
// Resource represents a syncable resource (document, folder, space) from external system
+206
View File
@@ -0,0 +1,206 @@
package types
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestDataSourceConfig_Redacted(t *testing.T) {
t.Run("redacts all string values in Credentials", func(t *testing.T) {
cfg := DataSourceConfig{
Type: "github",
Credentials: map[string]interface{}{
"access_token": "real-pat",
"installation_id": "real-id",
},
ResourceIDs: []string{"repo-1"},
}
out := cfg.Redacted()
assert.Equal(t, RedactedSecretPlaceholder, out.Credentials["access_token"])
assert.Equal(t, RedactedSecretPlaceholder, out.Credentials["installation_id"])
assert.Equal(t, "github", out.Type, "non-credential fields preserved")
assert.Equal(t, []string{"repo-1"}, out.ResourceIDs)
})
t.Run("leaves empty string credentials empty", func(t *testing.T) {
cfg := DataSourceConfig{
Credentials: map[string]interface{}{"token": ""},
}
out := cfg.Redacted()
assert.Equal(t, "", out.Credentials["token"],
"empty must stay empty to signal 'not set' to the frontend")
})
t.Run("non-string credential values pass through unchanged", func(t *testing.T) {
cfg := DataSourceConfig{
Credentials: map[string]interface{}{
"token": "secret",
"installation_id": float64(12345),
"metadata": map[string]interface{}{"nested": "value"},
},
}
out := cfg.Redacted()
assert.Equal(t, RedactedSecretPlaceholder, out.Credentials["token"])
assert.Equal(t, float64(12345), out.Credentials["installation_id"])
assert.NotNil(t, out.Credentials["metadata"])
})
t.Run("does not mutate original", func(t *testing.T) {
cfg := DataSourceConfig{
Credentials: map[string]interface{}{"token": "secret"},
}
_ = cfg.Redacted()
assert.Equal(t, "secret", cfg.Credentials["token"])
})
t.Run("nil credentials map stays nil", func(t *testing.T) {
cfg := DataSourceConfig{Type: "github"}
out := cfg.Redacted()
assert.Nil(t, out.Credentials)
})
t.Run("ClearCredentials flag stripped from response", func(t *testing.T) {
cfg := DataSourceConfig{ClearCredentials: true, Type: "github"}
out := cfg.Redacted()
assert.False(t, out.ClearCredentials, "write-only flag must not echo back")
})
}
func TestDataSourceConfig_MergeUpdate(t *testing.T) {
existing := DataSourceConfig{
Type: "github",
Credentials: map[string]interface{}{
"access_token": "stored-pat",
"installation_id": "stored-id",
},
ResourceIDs: []string{"old-repo"},
}
t.Run("empty credential value preserves existing", func(t *testing.T) {
in := DataSourceConfig{
Type: "github",
Credentials: map[string]interface{}{
"access_token": "",
},
}
out := in.MergeUpdate(existing)
assert.Equal(t, "stored-pat", out.Credentials["access_token"])
assert.Equal(t, "stored-id", out.Credentials["installation_id"],
"keys not present in incoming carry over from existing")
})
t.Run("redacted placeholder preserves existing", func(t *testing.T) {
in := DataSourceConfig{
Credentials: map[string]interface{}{
"access_token": RedactedSecretPlaceholder,
},
}
out := in.MergeUpdate(existing)
assert.Equal(t, "stored-pat", out.Credentials["access_token"])
})
t.Run("real value replaces existing", func(t *testing.T) {
in := DataSourceConfig{
Credentials: map[string]interface{}{
"access_token": "new-pat",
},
}
out := in.MergeUpdate(existing)
assert.Equal(t, "new-pat", out.Credentials["access_token"])
assert.Equal(t, "stored-id", out.Credentials["installation_id"],
"other keys preserved from existing")
})
t.Run("mixed preserve and replace", func(t *testing.T) {
in := DataSourceConfig{
Credentials: map[string]interface{}{
"access_token": RedactedSecretPlaceholder, // preserve
"installation_id": "new-id", // replace
},
}
out := in.MergeUpdate(existing)
assert.Equal(t, "stored-pat", out.Credentials["access_token"])
assert.Equal(t, "new-id", out.Credentials["installation_id"])
})
t.Run("ClearCredentials wipes entire map", func(t *testing.T) {
in := DataSourceConfig{ClearCredentials: true}
out := in.MergeUpdate(existing)
assert.Nil(t, out.Credentials,
"ClearCredentials removes all credential keys atomically")
})
t.Run("ClearCredentials takes precedence over submitted values", func(t *testing.T) {
in := DataSourceConfig{
ClearCredentials: true,
Credentials: map[string]interface{}{
"access_token": "will-be-ignored",
},
}
out := in.MergeUpdate(existing)
assert.Nil(t, out.Credentials)
})
t.Run("ClearCredentials flag never persists on merged result", func(t *testing.T) {
in := DataSourceConfig{ClearCredentials: true}
out := in.MergeUpdate(existing)
assert.False(t, out.ClearCredentials, "write-only flag must not leak to storage")
})
t.Run("non-string incoming values always overwrite", func(t *testing.T) {
in := DataSourceConfig{
Credentials: map[string]interface{}{
"installation_id": float64(999),
},
}
out := in.MergeUpdate(existing)
assert.Equal(t, float64(999), out.Credentials["installation_id"],
"non-string values are never treated as redacted/empty")
})
t.Run("non-credential fields flow from incoming", func(t *testing.T) {
in := DataSourceConfig{
Type: "notion",
ResourceIDs: []string{"new-repo"},
}
out := in.MergeUpdate(existing)
assert.Equal(t, "notion", out.Type)
assert.Equal(t, []string{"new-repo"}, out.ResourceIDs)
})
}
func TestDataSource_RedactSensitiveData(t *testing.T) {
t.Run("redacts string credentials in Config jsonb", func(t *testing.T) {
cfg := DataSourceConfig{
Type: "github",
Credentials: map[string]interface{}{
"access_token": "real-pat",
},
}
blob, err := json.Marshal(cfg)
require.NoError(t, err)
ds := &DataSource{Config: JSON(blob)}
ds.RedactSensitiveData()
parsed, err := ds.ParseConfig()
require.NoError(t, err)
assert.Equal(t, RedactedSecretPlaceholder, parsed.Credentials["access_token"])
})
t.Run("no-op on empty config", func(t *testing.T) {
ds := &DataSource{Config: nil}
ds.RedactSensitiveData() // must not panic
assert.Nil(t, ds.Config)
})
t.Run("no-op on malformed config", func(t *testing.T) {
ds := &DataSource{Config: JSON(`not valid json`)}
ds.RedactSensitiveData() // must not panic
// Config remains unchanged — better than dropping the response.
assert.Equal(t, JSON(`not valid json`), ds.Config)
})
}
+73 -17
View File
@@ -41,11 +41,60 @@ type MCPService struct {
// MCPHeaders represents HTTP headers as a map
type MCPHeaders map[string]string
// MCPAuthConfig represents authentication configuration for MCP service
// MCPAuthConfig represents authentication configuration for MCP service.
//
// ClearAPIKey and ClearToken are write-only flags used in Update requests to
// explicitly remove a stored credential. They are not persisted (gorm:"-") and
// never appear in responses (omitempty + zero value). Without these flags an
// empty APIKey/Token submitted in an Update request is interpreted as "no
// change", so this is the only way to intentionally clear a stored secret.
type MCPAuthConfig struct {
APIKey string `json:"api_key,omitempty"`
Token string `json:"token,omitempty"`
CustomHeaders map[string]string `json:"custom_headers,omitempty"`
// Write-only clear flags.
ClearAPIKey bool `json:"clear_api_key,omitempty" gorm:"-"`
ClearToken bool `json:"clear_token,omitempty" gorm:"-"`
}
// MergeUpdate applies write-only secret merge semantics for Update requests.
// The receiver is the incoming request DTO; existing is the currently stored
// value (may be nil for new services).
//
// Behavior:
// - ClearAPIKey / ClearToken take precedence — wipe the corresponding field.
// - Otherwise APIKey / Token use PreserveIfRedacted: empty or the redacted
// placeholder preserves existing, any other value replaces.
// - CustomHeaders: nil (absent in request) preserves existing; any non-nil
// value (including an empty map) replaces. Callers that want to clear
// headers explicitly should send {} rather than omitting the field.
// - Write-only clear flags are cleared on the merged result so they never
// leak to storage.
func (c *MCPAuthConfig) MergeUpdate(existing *MCPAuthConfig) *MCPAuthConfig {
merged := &MCPAuthConfig{}
if existing != nil {
*merged = *existing
merged.ClearAPIKey = false
merged.ClearToken = false
}
if c == nil {
return merged
}
if c.ClearAPIKey {
merged.APIKey = ""
} else {
merged.APIKey = PreserveIfRedacted(c.APIKey, merged.APIKey)
}
if c.ClearToken {
merged.Token = ""
} else {
merged.Token = PreserveIfRedacted(c.Token, merged.Token)
}
if c.CustomHeaders != nil {
merged.CustomHeaders = c.CustomHeaders
}
return merged
}
// MCPAdvancedConfig represents advanced configuration for MCP service
@@ -228,16 +277,30 @@ func GetDefaultAdvancedConfig() *MCPAdvancedConfig {
}
}
// MaskSensitiveData masks sensitive information in the MCP service for display
func (m *MCPService) MaskSensitiveData() {
if m.AuthConfig != nil {
if m.AuthConfig.APIKey != "" {
m.AuthConfig.APIKey = maskString(m.AuthConfig.APIKey)
}
if m.AuthConfig.Token != "" {
m.AuthConfig.Token = maskString(m.AuthConfig.Token)
}
// RedactSensitiveData replaces secret values in the MCP service with
// RedactedSecretPlaceholder while preserving the "set vs not set" distinction
// (empty values stay empty). Mutates the receiver in place; callers that need
// to retain the originals should pass a copy.
//
// This replaces the previous MaskSensitiveData implementation which returned
// a "first4****last4" slice of the original secret, leaking 8 characters of
// entropy per value. The new implementation returns a fixed placeholder with
// no information about the underlying secret.
func (m *MCPService) RedactSensitiveData() {
if m.AuthConfig == nil {
return
}
if m.AuthConfig.APIKey != "" {
m.AuthConfig.APIKey = RedactedSecretPlaceholder
}
if m.AuthConfig.Token != "" {
m.AuthConfig.Token = RedactedSecretPlaceholder
}
// Write-only flags must never reach the client. omitempty alone is
// insufficient because a true value serializes; explicitly clear them so
// any response path that runs RedactSensitiveData is safe by construction.
m.AuthConfig.ClearAPIKey = false
m.AuthConfig.ClearToken = false
}
// HideSensitiveInfo returns a copy of the MCP service with sensitive fields cleared for builtin services
@@ -255,10 +318,3 @@ func (m *MCPService) HideSensitiveInfo() *MCPService {
return &copy
}
// maskString masks a string, showing only first 4 and last 4 characters
func maskString(s string) string {
if len(s) <= 8 {
return "****"
}
return s[:4] + "****" + s[len(s)-4:]
}
+52
View File
@@ -0,0 +1,52 @@
package types
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestMCPService_RedactSensitiveData(t *testing.T) {
t.Run("redacts both api_key and token when set", func(t *testing.T) {
m := &MCPService{
AuthConfig: &MCPAuthConfig{
APIKey: "real-api",
Token: "real-token",
},
}
m.RedactSensitiveData()
assert.Equal(t, RedactedSecretPlaceholder, m.AuthConfig.APIKey)
assert.Equal(t, RedactedSecretPlaceholder, m.AuthConfig.Token)
})
t.Run("leaves empty secrets empty", func(t *testing.T) {
m := &MCPService{AuthConfig: &MCPAuthConfig{}}
m.RedactSensitiveData()
assert.Empty(t, m.AuthConfig.APIKey, "empty must stay empty to signal 'not set' to the frontend")
assert.Empty(t, m.AuthConfig.Token)
})
t.Run("noop on nil AuthConfig", func(t *testing.T) {
m := &MCPService{AuthConfig: nil}
assert.NotPanics(t, func() { m.RedactSensitiveData() })
assert.Nil(t, m.AuthConfig)
})
t.Run("zeroes write-only clear flags so they never echo back", func(t *testing.T) {
// Regression for response-side leak: a true Clear* bool serializes
// despite json:"...,omitempty" because true is not a zero value.
// Any response path that runs RedactSensitiveData must produce JSON
// without the clear_api_key / clear_token keys.
m := &MCPService{
AuthConfig: &MCPAuthConfig{
APIKey: "real-api",
Token: "real-token",
ClearAPIKey: true,
ClearToken: true,
},
}
m.RedactSensitiveData()
assert.False(t, m.AuthConfig.ClearAPIKey, "ClearAPIKey must be reset before response")
assert.False(t, m.AuthConfig.ClearToken, "ClearToken must be reset before response")
})
}
+51
View File
@@ -77,6 +77,57 @@ type ModelParameters struct {
// WeKnoraCloud 厂商专用凭证
AppID string `yaml:"app_id,omitempty" json:"app_id,omitempty"`
AppSecret string `yaml:"app_secret,omitempty" json:"app_secret,omitempty"` // AES-256 加密存储,实际承载上游 API Key
// Write-only flags: set to true in Update requests to explicitly remove
// the corresponding credential. Never persisted (gorm:"-"), never
// returned in responses (omitempty + zero value).
ClearAPIKey bool `yaml:"-" json:"clear_api_key,omitempty" gorm:"-"`
ClearAppSecret bool `yaml:"-" json:"clear_app_secret,omitempty" gorm:"-"`
}
// RedactSensitiveData replaces secret values (APIKey, AppSecret) in
// ModelParameters with RedactedSecretPlaceholder when set. Empty values stay
// empty so the frontend can distinguish "set (hidden)" from "not set".
// Mutates the receiver in place.
func (m *Model) RedactSensitiveData() {
if m.Parameters.APIKey != "" {
m.Parameters.APIKey = RedactedSecretPlaceholder
}
if m.Parameters.AppSecret != "" {
m.Parameters.AppSecret = RedactedSecretPlaceholder
}
// Write-only flags must never reach the client. omitempty alone is
// insufficient because a true value serializes; explicitly clear them so
// any response path that runs RedactSensitiveData is safe by construction.
m.Parameters.ClearAPIKey = false
m.Parameters.ClearAppSecret = false
}
// MergeUpdate applies write-only secret merge semantics for Update requests:
//
// - ClearAPIKey / ClearAppSecret take precedence — when true, the
// corresponding field becomes empty
// - Otherwise APIKey and AppSecret use PreserveIfRedacted: empty or the
// redacted placeholder preserves the existing value, any other value
// replaces
// - Non-secret fields flow from incoming directly
// - Write-only clear flags are cleared on the merged result so they never
// leak to storage
func (p ModelParameters) MergeUpdate(existing ModelParameters) ModelParameters {
merged := p
if p.ClearAPIKey {
merged.APIKey = ""
} else {
merged.APIKey = PreserveIfRedacted(p.APIKey, existing.APIKey)
}
if p.ClearAppSecret {
merged.AppSecret = ""
} else {
merged.AppSecret = PreserveIfRedacted(p.AppSecret, existing.AppSecret)
}
merged.ClearAPIKey = false
merged.ClearAppSecret = false
return merged
}
// Model represents the AI model
+139
View File
@@ -0,0 +1,139 @@
package types
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestModel_RedactSensitiveData(t *testing.T) {
t.Run("redacts both APIKey and AppSecret when set", func(t *testing.T) {
m := &Model{
Parameters: ModelParameters{
APIKey: "real-api",
AppSecret: "real-secret",
BaseURL: "https://example.com",
},
}
m.RedactSensitiveData()
assert.Equal(t, RedactedSecretPlaceholder, m.Parameters.APIKey)
assert.Equal(t, RedactedSecretPlaceholder, m.Parameters.AppSecret)
assert.Equal(t, "https://example.com", m.Parameters.BaseURL,
"non-secret fields must be preserved")
})
t.Run("leaves empty secrets empty", func(t *testing.T) {
m := &Model{
Parameters: ModelParameters{BaseURL: "https://example.com"},
}
m.RedactSensitiveData()
assert.Empty(t, m.Parameters.APIKey)
assert.Empty(t, m.Parameters.AppSecret)
})
t.Run("redacts only set secret when one is empty", func(t *testing.T) {
m := &Model{
Parameters: ModelParameters{APIKey: "real-api"}, // AppSecret empty
}
m.RedactSensitiveData()
assert.Equal(t, RedactedSecretPlaceholder, m.Parameters.APIKey)
assert.Empty(t, m.Parameters.AppSecret)
})
t.Run("zeroes write-only clear flags so they never echo back", func(t *testing.T) {
// Regression for response-side leak: a true Clear* bool serializes
// despite json:"...,omitempty" because true is not a zero value.
// Any response path that runs RedactSensitiveData must produce JSON
// without the clear_* keys.
m := &Model{
Parameters: ModelParameters{
APIKey: "real-api",
ClearAPIKey: true,
ClearAppSecret: true,
},
}
m.RedactSensitiveData()
assert.False(t, m.Parameters.ClearAPIKey, "ClearAPIKey must be reset before response")
assert.False(t, m.Parameters.ClearAppSecret, "ClearAppSecret must be reset before response")
})
}
func TestModelParameters_MergeUpdate(t *testing.T) {
existing := ModelParameters{
APIKey: "stored-api",
AppSecret: "stored-secret",
BaseURL: "https://stored.example.com",
Provider: "openai",
}
t.Run("empty secrets preserve existing", func(t *testing.T) {
in := ModelParameters{BaseURL: "https://new.example.com"}
out := in.MergeUpdate(existing)
assert.Equal(t, "stored-api", out.APIKey)
assert.Equal(t, "stored-secret", out.AppSecret)
assert.Equal(t, "https://new.example.com", out.BaseURL,
"non-secret fields flow from incoming")
})
t.Run("redacted placeholders preserve existing", func(t *testing.T) {
in := ModelParameters{
APIKey: RedactedSecretPlaceholder,
AppSecret: RedactedSecretPlaceholder,
}
out := in.MergeUpdate(existing)
assert.Equal(t, "stored-api", out.APIKey)
assert.Equal(t, "stored-secret", out.AppSecret)
})
t.Run("real values replace existing", func(t *testing.T) {
in := ModelParameters{APIKey: "new-api", AppSecret: "new-secret"}
out := in.MergeUpdate(existing)
assert.Equal(t, "new-api", out.APIKey)
assert.Equal(t, "new-secret", out.AppSecret)
})
t.Run("mixed preserve and replace", func(t *testing.T) {
in := ModelParameters{
APIKey: RedactedSecretPlaceholder, // preserve
AppSecret: "new-secret", // replace
}
out := in.MergeUpdate(existing)
assert.Equal(t, "stored-api", out.APIKey)
assert.Equal(t, "new-secret", out.AppSecret)
})
t.Run("ClearAPIKey wipes only APIKey", func(t *testing.T) {
in := ModelParameters{ClearAPIKey: true}
out := in.MergeUpdate(existing)
assert.Empty(t, out.APIKey)
assert.Equal(t, "stored-secret", out.AppSecret,
"ClearAPIKey must not affect AppSecret")
})
t.Run("ClearAppSecret wipes only AppSecret", func(t *testing.T) {
in := ModelParameters{ClearAppSecret: true}
out := in.MergeUpdate(existing)
assert.Equal(t, "stored-api", out.APIKey)
assert.Empty(t, out.AppSecret)
})
t.Run("both clear flags wipe both secrets", func(t *testing.T) {
in := ModelParameters{ClearAPIKey: true, ClearAppSecret: true}
out := in.MergeUpdate(existing)
assert.Empty(t, out.APIKey)
assert.Empty(t, out.AppSecret)
})
t.Run("clear takes precedence over submitted value", func(t *testing.T) {
in := ModelParameters{APIKey: "will-be-ignored", ClearAPIKey: true}
out := in.MergeUpdate(existing)
assert.Empty(t, out.APIKey)
})
t.Run("clear flags never persist on merged result", func(t *testing.T) {
in := ModelParameters{ClearAPIKey: true, ClearAppSecret: true}
out := in.MergeUpdate(existing)
assert.False(t, out.ClearAPIKey, "write-only flag must not leak to storage")
assert.False(t, out.ClearAppSecret, "write-only flag must not leak to storage")
})
}
+56
View File
@@ -0,0 +1,56 @@
// Package types — shared secret redaction helpers for the "write-only secrets"
// pattern applied across MCP services, Models, WebSearch providers,
// DataSources, and VectorStores.
//
// The contract is:
//
// - API responses replace sensitive values with RedactedSecretPlaceholder
// when the value is set, and leave it as the empty string when it is not.
// Clients can therefore distinguish "set (hidden)" from "not set" without
// ever seeing the secret itself.
//
// - API Update requests treat an absent field, the empty string, or the
// RedactedSecretPlaceholder as "no change requested" (preserve), and any
// other value as an explicit replacement.
//
// - Explicit removal of a stored secret is expressed with a dedicated
// write-only boolean flag on the request DTO (e.g. ClearAPIKey) rather
// than by sending an empty string, because the empty string is already
// reserved for "preserve".
package types
// RedactedSecretPlaceholder is the fixed value returned in API responses
// whenever a sensitive field is set but withheld from the client. The empty
// string ("") is reserved for the orthogonal "not set" state, which lets the
// frontend distinguish the two states without an extra boolean field.
//
// The value matches the placeholder already used by
// ConnectionConfig.MaskSensitiveFields on VectorStore responses; this package
// promotes it to a shared constant so every resource agrees on the same
// sentinel.
const RedactedSecretPlaceholder = "***"
// IsRedactedOrEmpty reports whether s should be treated as "no change
// requested" in an Update* request. It returns true for:
//
// - "" — the field was absent from the client payload or explicitly cleared
// on the form without using the dedicated Clear* flag
// - RedactedSecretPlaceholder — the client echoed back the value it received
// in a GET response (for example, a legacy frontend that pre-fills the
// edit form with the redacted value)
//
// Any other value is an explicit replacement and must be persisted.
func IsRedactedOrEmpty(s string) bool {
return s == "" || s == RedactedSecretPlaceholder
}
// PreserveIfRedacted returns existing when incoming is empty or the redacted
// placeholder, otherwise returns incoming. Call this from every Update*
// service that accepts secret fields in its request DTO to keep the preserve
// semantics identical across resources.
func PreserveIfRedacted(incoming, existing string) string {
if IsRedactedOrEmpty(incoming) {
return existing
}
return incoming
}
+51
View File
@@ -0,0 +1,51 @@
package types
import "testing"
func TestIsRedactedOrEmpty(t *testing.T) {
tests := []struct {
name string
in string
want bool
}{
{"empty string is redacted-or-empty", "", true},
{"fixed placeholder is redacted-or-empty", RedactedSecretPlaceholder, true},
{"short real secret is not redacted", "ab", false},
{"real secret longer than placeholder is not redacted", "real-bearer-token", false},
{"secret containing placeholder substring is not redacted", "prefix***suffix", false},
{"single asterisk is not redacted", "*", false},
{"double asterisk is not redacted", "**", false},
{"four asterisks is not redacted (distinct from 3)", "****", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := IsRedactedOrEmpty(tt.in); got != tt.want {
t.Errorf("IsRedactedOrEmpty(%q) = %v, want %v", tt.in, got, tt.want)
}
})
}
}
func TestPreserveIfRedacted(t *testing.T) {
tests := []struct {
name string
incoming string
existing string
want string
}{
{"empty incoming preserves existing", "", "stored-secret", "stored-secret"},
{"placeholder incoming preserves existing", RedactedSecretPlaceholder, "stored-secret", "stored-secret"},
{"real incoming replaces existing", "new-secret", "stored-secret", "new-secret"},
{"real incoming replaces empty existing", "new-secret", "", "new-secret"},
{"empty incoming with empty existing stays empty", "", "", ""},
{"placeholder incoming with empty existing stays empty", RedactedSecretPlaceholder, "", ""},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := PreserveIfRedacted(tt.incoming, tt.existing); got != tt.want {
t.Errorf("PreserveIfRedacted(%q, %q) = %q, want %q",
tt.incoming, tt.existing, got, tt.want)
}
})
}
}
+5 -3
View File
@@ -194,14 +194,16 @@ func (c ConnectionConfig) GetEndpoint() string {
return ""
}
// MaskSensitiveFields returns a copy with Password and APIKey masked.
// MaskSensitiveFields returns a copy with Password and APIKey replaced by the
// shared RedactedSecretPlaceholder. Empty values stay empty so the frontend
// can distinguish "set (hidden)" from "not set" without an extra flag.
func (c ConnectionConfig) MaskSensitiveFields() ConnectionConfig {
masked := c
if masked.Password != "" {
masked.Password = "***"
masked.Password = RedactedSecretPlaceholder
}
if masked.APIKey != "" {
masked.APIKey = "***"
masked.APIKey = RedactedSecretPlaceholder
}
return masked
}
+4 -4
View File
@@ -214,8 +214,8 @@ func TestNewVectorStoreResponse(t *testing.T) {
t.Run("masks sensitive fields", func(t *testing.T) {
resp := NewVectorStoreResponse(store, "user", false)
assert.Equal(t, "***", resp.ConnectionConfig.Password)
assert.Equal(t, "***", resp.ConnectionConfig.APIKey)
assert.Equal(t, RedactedSecretPlaceholder, resp.ConnectionConfig.Password)
assert.Equal(t, RedactedSecretPlaceholder, resp.ConnectionConfig.APIKey)
assert.Equal(t, "http://es:9200", resp.ConnectionConfig.Addr) // non-sensitive preserved
})
@@ -574,8 +574,8 @@ func TestConnectionConfig_MaskSensitiveFields(t *testing.T) {
APIKey: "sk-api-key",
}
masked := c.MaskSensitiveFields()
assert.Equal(t, "***", masked.Password)
assert.Equal(t, "***", masked.APIKey)
assert.Equal(t, RedactedSecretPlaceholder, masked.Password)
assert.Equal(t, RedactedSecretPlaceholder, masked.APIKey)
assert.Equal(t, "http://es:9200", masked.Addr)
assert.Equal(t, "elastic", masked.Username)
})
+42
View File
@@ -67,6 +67,12 @@ func (e *WebSearchProviderEntity) BeforeCreate(tx *gorm.DB) (err error) {
// API keys are encrypted at rest using AES-GCM.
// BaseURL is intentionally NOT included — each provider type uses a hardcoded
// official API endpoint to prevent SSRF attacks.
//
// ClearAPIKey is a write-only flag used in Update requests to explicitly
// remove the stored API key. It is never persisted and never returned in
// responses. Without this flag an empty APIKey in an Update request is
// interpreted as "no change", so this is the only way to intentionally clear
// a stored key without deleting the entire provider record.
type WebSearchProviderParameters struct {
// API key for the search provider (encrypted in DB)
APIKey string `yaml:"api_key" json:"api_key,omitempty"`
@@ -80,6 +86,42 @@ type WebSearchProviderParameters struct {
ProxyURL string `yaml:"proxy_url" json:"proxy_url,omitempty"`
// Provider-specific extra configuration for future extensibility
ExtraConfig map[string]string `yaml:"extra_config" json:"extra_config,omitempty"`
// Write-only clear flag — never persisted, never returned.
ClearAPIKey bool `yaml:"-" json:"clear_api_key,omitempty" gorm:"-"`
}
// RedactSensitiveData replaces APIKey with RedactedSecretPlaceholder when a
// value is set. Empty values stay empty so the frontend can distinguish
// "set (hidden)" from "not set". Mutates the receiver in place.
func (e *WebSearchProviderEntity) RedactSensitiveData() {
if e.Parameters.APIKey != "" {
e.Parameters.APIKey = RedactedSecretPlaceholder
}
// Write-only flag must never reach the client. omitempty alone is
// insufficient because a true value serializes; explicitly clear it so
// any response path that runs RedactSensitiveData is safe by construction.
e.Parameters.ClearAPIKey = false
}
// MergeUpdate applies write-only secret merge semantics for Update requests:
//
// - ClearAPIKey takes precedence — when true, APIKey becomes empty
// - Otherwise APIKey uses PreserveIfRedacted: empty or the redacted
// placeholder preserves existing, any other value replaces
// - Non-secret fields (EngineID, ProxyURL, ExtraConfig) are replaced
// directly from incoming
// - The write-only ClearAPIKey flag is cleared on the merged result so it
// never leaks to storage
func (p WebSearchProviderParameters) MergeUpdate(existing WebSearchProviderParameters) WebSearchProviderParameters {
merged := p
if p.ClearAPIKey {
merged.APIKey = ""
} else {
merged.APIKey = PreserveIfRedacted(p.APIKey, existing.APIKey)
}
merged.ClearAPIKey = false
return merged
}
// Value implements the driver.Valuer interface.
@@ -0,0 +1,87 @@
package types
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestWebSearchProviderEntity_RedactSensitiveData(t *testing.T) {
t.Run("redacts set api key", func(t *testing.T) {
e := &WebSearchProviderEntity{
Parameters: WebSearchProviderParameters{APIKey: "real-key", EngineID: "cse-id"},
}
e.RedactSensitiveData()
assert.Equal(t, RedactedSecretPlaceholder, e.Parameters.APIKey)
assert.Equal(t, "cse-id", e.Parameters.EngineID, "non-secret fields preserved")
})
t.Run("leaves empty api key empty", func(t *testing.T) {
e := &WebSearchProviderEntity{
Parameters: WebSearchProviderParameters{APIKey: "", EngineID: "cse-id"},
}
e.RedactSensitiveData()
assert.Empty(t, e.Parameters.APIKey, "empty must stay empty to signal 'not set' to the frontend")
})
t.Run("zeroes write-only clear flag so it never echoes back", func(t *testing.T) {
// Regression for response-side leak: a true ClearAPIKey serializes
// despite json:"...,omitempty" because true is not a zero value.
e := &WebSearchProviderEntity{
Parameters: WebSearchProviderParameters{APIKey: "real-key", ClearAPIKey: true},
}
e.RedactSensitiveData()
assert.False(t, e.Parameters.ClearAPIKey, "ClearAPIKey must be reset before response")
})
}
func TestWebSearchProviderParameters_MergeUpdate(t *testing.T) {
existing := WebSearchProviderParameters{
APIKey: "stored-key",
EngineID: "stored-engine",
ProxyURL: "http://stored-proxy",
}
t.Run("empty api key preserves existing", func(t *testing.T) {
in := WebSearchProviderParameters{APIKey: "", EngineID: "new-engine"}
out := in.MergeUpdate(existing)
assert.Equal(t, "stored-key", out.APIKey)
assert.Equal(t, "new-engine", out.EngineID, "non-secret fields flow from incoming")
})
t.Run("redacted placeholder preserves existing", func(t *testing.T) {
in := WebSearchProviderParameters{APIKey: RedactedSecretPlaceholder}
out := in.MergeUpdate(existing)
assert.Equal(t, "stored-key", out.APIKey)
})
t.Run("real value replaces existing", func(t *testing.T) {
in := WebSearchProviderParameters{APIKey: "new-key"}
out := in.MergeUpdate(existing)
assert.Equal(t, "new-key", out.APIKey)
})
t.Run("ClearAPIKey wipes stored value", func(t *testing.T) {
in := WebSearchProviderParameters{ClearAPIKey: true}
out := in.MergeUpdate(existing)
assert.Empty(t, out.APIKey)
})
t.Run("ClearAPIKey takes precedence over submitted value", func(t *testing.T) {
in := WebSearchProviderParameters{APIKey: "will-be-ignored", ClearAPIKey: true}
out := in.MergeUpdate(existing)
assert.Empty(t, out.APIKey)
})
t.Run("ClearAPIKey flag never persists on merged result", func(t *testing.T) {
in := WebSearchProviderParameters{ClearAPIKey: true}
out := in.MergeUpdate(existing)
assert.False(t, out.ClearAPIKey, "write-only flag must not leak to storage")
})
t.Run("merge against empty existing", func(t *testing.T) {
in := WebSearchProviderParameters{APIKey: RedactedSecretPlaceholder}
out := in.MergeUpdate(WebSearchProviderParameters{})
assert.Empty(t, out.APIKey, "placeholder against empty preserves empty")
})
}