Files
Enterprise-Knowledge-Base/internal/application/service/mcp_service_test.go
T
ochan.kwon d869cc1ff6 fix: redact sensitive fields and prevent credential loss on edit
Closes #988

Unify the "write-only secrets" pattern across MCP services, Models,
WebSearch providers, and DataSources. Fix two related defects:

1. GET responses leaked plaintext credentials — GetMCPServiceByID
   while List masked, all Model / WebSearchProvider / DataSource GET
   responses unconditionally.
2. When the frontend echoed a masked placeholder back on save, the
   update path overwrote the stored secret with the mask — silently
   wiping the real credential. MCP surfaced this most visibly:
   ListMCPServices masked Token to "first4****last4" and
   McpServiceDialog round-tripped that masked string back into the DB.

- internal/types/secret.go introduces the shared sentinel
  RedactedSecretPlaceholder = "***" and two helpers:
  - IsRedactedOrEmpty(s): true for "" or the sentinel
  - PreserveIfRedacted(incoming, existing): merge predicate
- Each resource adds a RedactSensitiveData (mutates-in-place) or
  Redacted (returns-a-copy) method and a MergeUpdate contract:
  empty or placeholder preserves, any other value replaces, and a
  write-only Clear* bool flag (gorm:"-", json omitempty) wipes with
  precedence over preserve / replace. Clear* never persists.
- Every GET / List / Create / Update response path now redacts.
  VectorStore's pre-existing "***" literal is refactored onto the
  shared constant for consistency.

All four dialogs — McpServiceDialog, ModelEditorDialog,
WebSearchSettings, DataSourceEditorDialog — never pre-fill secret
inputs. The stored-state signal is collapsed into the input
placeholder: "•••••••• (Enter new value to replace)" when the server
reports a stored value, falling back to the connector-specific
placeholder otherwise. The input itself carries the "something is
there" cue, matching how password dots render.

An opt-in "Remove this credential" checkbox appears in edit mode when
a value is stored; checking it disables the input and triggers a
danger-themed confirmation dialog before submit. DataSource rolls the
clear up to a connector-level "Remove all credentials" flag since its
credentials are per-connector atomic bundles (e.g. Feishu app_id +
app_secret).

i18n updates across en-US, zh-CN, ko-KR, ru-RU:
secret.storedPlaceholder / clearHint / confirmClearTitle /
confirmClearBody.

- MCP update handler's manual map-to-struct projection was silently
  dropping clear_api_key / clear_token / custom_headers on partial
  updates. Pull them off the map explicitly.
- MCP configChanged detection now snapshots URL / StdioConfig /
  TransportType before the merge so connection recycling still
  triggers on real transport changes (not just whenever AuthConfig
  is present).
- MCP CustomHeaders: nil preserves existing, non-nil replaces. Was
  always overwriting, silently wiping headers on any auth-bearing
  update.
- GetMCPServiceByID now redacts (was plaintext while List masked).
- WebSearchProvider update preserves Name / Description when empty
  instead of clobbering them.
- WebSearchProvider TestProviderRaw rejects the "***" placeholder up
  front with an actionable message rather than forwarding it to the
  upstream provider API as a literal key.
- Model builtin hideSensitiveInfo now preserves non-secret metadata
  (SupportsVision, Provider, ExtraConfig, InterfaceType,
  ParameterSize, EmbeddingParameters) that were previously zeroed
  out for builtin model cards.
- Replace local equalStringSlices / equalStringMaps helpers with
  stdlib slices.Equal / maps.Equal (Go 1.21+).

- internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go:
  table-driven coverage of merge, redact, clear, and write-only flag
  invariants for every resource.
- internal/application/service/mcp_service_test.go: full service layer
  with a fake repo; covers ListMCPServices / GetMCPServiceByID
  redaction, UpdateMCPService partial-update honoring of Clear*,
  CustomHeaders preserve-on-nil / replace-on-non-nil, and
  clear-flag precedence over submitted values.

Additive only. No changes to request or response JSON shape beyond
the write-only Clear* flags (omitted by default via omitempty). No DB
schema changes. No SDK struct field removals. Existing clients that
don't know about the new flags continue to function unchanged.

- DataSource.Config jsonb at-rest encryption. Pre-existing residual
  risk; tracked as a follow-up.
- Renaming VectorStore.MaskSensitiveFields to match the new
  RedactSensitiveData / Redacted naming convention. Left alone to
  minimize diff; a follow-up rename PR can unify.
2026-05-17 11:37:53 +08:00

396 lines
12 KiB
Go

package service
import (
"context"
"testing"
"github.com/Tencent/WeKnora/internal/mcp"
"github.com/Tencent/WeKnora/internal/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// fakeMCPRepo is a minimal in-memory implementation of
// interfaces.MCPServiceRepository for testing the service-layer merge logic
// without depending on the database.
type fakeMCPRepo struct {
store map[string]*types.MCPService
}
func newFakeMCPRepo() *fakeMCPRepo {
return &fakeMCPRepo{store: make(map[string]*types.MCPService)}
}
func (r *fakeMCPRepo) Create(_ context.Context, s *types.MCPService) error {
r.store[s.ID] = s
return nil
}
func (r *fakeMCPRepo) GetByID(_ context.Context, _ uint64, id string) (*types.MCPService, error) {
s, ok := r.store[id]
if !ok {
return nil, nil
}
// return a deep-ish copy so service-layer mutations to the returned value
// do not leak back into the fake store, mirroring how GORM produces a
// fresh struct per query
cp := *s
if s.AuthConfig != nil {
ac := *s.AuthConfig
cp.AuthConfig = &ac
}
if s.AdvancedConfig != nil {
adv := *s.AdvancedConfig
cp.AdvancedConfig = &adv
}
return &cp, nil
}
func cloneService(s *types.MCPService) *types.MCPService {
cp := *s
if s.AuthConfig != nil {
ac := *s.AuthConfig
cp.AuthConfig = &ac
}
if s.AdvancedConfig != nil {
adv := *s.AdvancedConfig
cp.AdvancedConfig = &adv
}
return &cp
}
func (r *fakeMCPRepo) List(_ context.Context, _ uint64) ([]*types.MCPService, error) {
out := make([]*types.MCPService, 0, len(r.store))
for _, s := range r.store {
out = append(out, cloneService(s))
}
return out, nil
}
func (r *fakeMCPRepo) ListEnabled(ctx context.Context, tenantID uint64) ([]*types.MCPService, error) {
return r.List(ctx, tenantID)
}
func (r *fakeMCPRepo) ListByIDs(_ context.Context, _ uint64, ids []string) ([]*types.MCPService, error) {
out := make([]*types.MCPService, 0, len(ids))
for _, id := range ids {
if s, ok := r.store[id]; ok {
out = append(out, cloneService(s))
}
}
return out, nil
}
func (r *fakeMCPRepo) Update(_ context.Context, s *types.MCPService) error {
r.store[s.ID] = s
return nil
}
func (r *fakeMCPRepo) Delete(_ context.Context, _ uint64, id string) error {
delete(r.store, id)
return nil
}
// seedService plants a service with the given auth credentials into the fake
// repo and returns the service ID. Convenience helper used by every test.
func seedService(t *testing.T, repo *fakeMCPRepo, apiKey, token string) string {
t.Helper()
s := &types.MCPService{
ID: "svc-test",
TenantID: 1,
Name: "test",
Enabled: true,
TransportType: types.MCPTransportSSE,
AuthConfig: &types.MCPAuthConfig{
APIKey: apiKey,
Token: token,
},
}
require.NoError(t, repo.Create(context.Background(), s))
return s.ID
}
// newTestService wires up a mcpServiceService with a fresh fake repo and a
// real (empty) MCPManager. CloseClient on an empty manager is a no-op, so
// tests can safely exercise the configChanged code paths.
func newTestService() (*mcpServiceService, *fakeMCPRepo) {
repo := newFakeMCPRepo()
svc := &mcpServiceService{
mcpServiceRepo: repo,
mcpManager: mcp.NewMCPManager(),
}
return svc, repo
}
// buildFullUpdate returns an UpdateMCPService input that exercises the
// "full update" branch (service.Name is non-empty), which is the branch that
// actually merges AuthConfig.
func buildFullUpdate(id string, auth *types.MCPAuthConfig) *types.MCPService {
return &types.MCPService{
ID: id,
TenantID: 1,
Name: "test", // non-empty → full update branch
Enabled: true,
TransportType: types.MCPTransportSSE,
AuthConfig: auth,
}
}
func TestUpdateMCPService_PreservesSecretsOnEmptyOrRedacted(t *testing.T) {
ctx := context.Background()
t.Run("nil AuthConfig preserves existing", func(t *testing.T) {
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
upd := buildFullUpdate(id, nil)
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, "stored-api", got.AuthConfig.APIKey)
assert.Equal(t, "stored-token", got.AuthConfig.Token)
})
t.Run("empty string preserves existing", func(t *testing.T) {
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
upd := buildFullUpdate(id, &types.MCPAuthConfig{APIKey: "", Token: ""})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, "stored-api", got.AuthConfig.APIKey)
assert.Equal(t, "stored-token", got.AuthConfig.Token)
})
t.Run("redacted placeholder preserves existing (NELO-1299 round-trip)", func(t *testing.T) {
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
upd := buildFullUpdate(id, &types.MCPAuthConfig{
APIKey: types.RedactedSecretPlaceholder,
Token: types.RedactedSecretPlaceholder,
})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, "stored-api", got.AuthConfig.APIKey)
assert.Equal(t, "stored-token", got.AuthConfig.Token)
})
}
func TestUpdateMCPService_ReplacesOnExplicitValue(t *testing.T) {
ctx := context.Background()
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
upd := buildFullUpdate(id, &types.MCPAuthConfig{
APIKey: "new-api",
Token: "new-token",
})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, "new-api", got.AuthConfig.APIKey)
assert.Equal(t, "new-token", got.AuthConfig.Token)
}
func TestUpdateMCPService_MixedFields(t *testing.T) {
ctx := context.Background()
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
// Replace token only, leave api_key as redacted placeholder → preserve
upd := buildFullUpdate(id, &types.MCPAuthConfig{
APIKey: types.RedactedSecretPlaceholder,
Token: "new-token",
})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, "stored-api", got.AuthConfig.APIKey, "api_key should be preserved")
assert.Equal(t, "new-token", got.AuthConfig.Token, "token should be replaced")
}
func TestUpdateMCPService_ClearFlags(t *testing.T) {
ctx := context.Background()
t.Run("ClearToken removes stored token, preserves api_key", func(t *testing.T) {
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
upd := buildFullUpdate(id, &types.MCPAuthConfig{
APIKey: types.RedactedSecretPlaceholder,
ClearToken: true,
})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, "stored-api", got.AuthConfig.APIKey)
assert.Empty(t, got.AuthConfig.Token)
})
t.Run("ClearAPIKey removes stored api_key, preserves token", func(t *testing.T) {
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
upd := buildFullUpdate(id, &types.MCPAuthConfig{
ClearAPIKey: true,
Token: types.RedactedSecretPlaceholder,
})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Empty(t, got.AuthConfig.APIKey)
assert.Equal(t, "stored-token", got.AuthConfig.Token)
})
t.Run("ClearToken takes precedence over submitted value", func(t *testing.T) {
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
upd := buildFullUpdate(id, &types.MCPAuthConfig{
Token: "will-be-ignored",
ClearToken: true,
})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Empty(t, got.AuthConfig.Token)
})
t.Run("clear flags are not persisted", func(t *testing.T) {
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
upd := buildFullUpdate(id, &types.MCPAuthConfig{ClearToken: true})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
require.NotNil(t, got.AuthConfig)
assert.False(t, got.AuthConfig.ClearToken, "ClearToken must not be persisted")
assert.False(t, got.AuthConfig.ClearAPIKey, "ClearAPIKey must not be persisted")
})
t.Run("clear on already-empty field is a no-op", func(t *testing.T) {
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "") // token not set
upd := buildFullUpdate(id, &types.MCPAuthConfig{ClearToken: true})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, "stored-api", got.AuthConfig.APIKey)
assert.Empty(t, got.AuthConfig.Token)
})
}
func TestListMCPServices_RedactsSecrets(t *testing.T) {
ctx := context.Background()
svc, repo := newTestService()
seedService(t, repo, "real-api", "real-token")
got, err := svc.ListMCPServices(ctx, 1)
require.NoError(t, err)
require.Len(t, got, 1)
assert.Equal(t, types.RedactedSecretPlaceholder, got[0].AuthConfig.APIKey)
assert.Equal(t, types.RedactedSecretPlaceholder, got[0].AuthConfig.Token)
// And the underlying store is untouched
stored := repo.store["svc-test"]
assert.Equal(t, "real-api", stored.AuthConfig.APIKey)
assert.Equal(t, "real-token", stored.AuthConfig.Token)
}
func TestGetMCPServiceByID_RedactsSecrets(t *testing.T) {
ctx := context.Background()
svc, repo := newTestService()
id := seedService(t, repo, "real-api", "real-token")
got, err := svc.GetMCPServiceByID(ctx, 1, id)
require.NoError(t, err)
require.NotNil(t, got.AuthConfig)
assert.Equal(t, types.RedactedSecretPlaceholder, got.AuthConfig.APIKey)
assert.Equal(t, types.RedactedSecretPlaceholder, got.AuthConfig.Token)
// underlying store unchanged
stored := repo.store[id]
assert.Equal(t, "real-api", stored.AuthConfig.APIKey)
assert.Equal(t, "real-token", stored.AuthConfig.Token)
}
func TestGetMCPServiceByID_EmptySecretsStayEmpty(t *testing.T) {
ctx := context.Background()
svc, repo := newTestService()
id := seedService(t, repo, "", "") // both empty
got, err := svc.GetMCPServiceByID(ctx, 1, id)
require.NoError(t, err)
require.NotNil(t, got.AuthConfig)
assert.Empty(t, got.AuthConfig.APIKey, "empty secret must stay empty, not be redacted")
assert.Empty(t, got.AuthConfig.Token)
}
// Partial update: service.Name == "" is interpreted as "only touch enabled".
// A clear flag sent in such a body was previously dropped; the refactor now
// merges AuthConfig unconditionally so single-field clears round-trip.
func TestUpdateMCPService_PartialUpdateHonorsClearFlag(t *testing.T) {
ctx := context.Background()
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
// Partial update body: no name, enabled flipped, ClearToken set.
upd := &types.MCPService{
ID: id,
TenantID: 1,
Enabled: true, // keep enabled, but simulate a "flag only" request
AuthConfig: &types.MCPAuthConfig{
ClearToken: true,
},
}
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, "stored-api", got.AuthConfig.APIKey, "APIKey preserved in partial update")
assert.Empty(t, got.AuthConfig.Token, "ClearToken must clear Token even in partial update")
}
// CustomHeaders: nil means "no change" (preserve existing). Sending a
// non-nil (including empty) map replaces. Previously the merge always
// overwrote, silently wiping headers on any auth_config-bearing request.
func TestUpdateMCPService_CustomHeadersPreserveOnNil(t *testing.T) {
ctx := context.Background()
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
// Seed with custom headers.
repo.store[id].AuthConfig.CustomHeaders = map[string]string{"X-Tenant": "acme"}
// Update with AuthConfig but nil CustomHeaders → preserve.
upd := buildFullUpdate(id, &types.MCPAuthConfig{
Token: "new-token",
})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, "acme", got.AuthConfig.CustomHeaders["X-Tenant"],
"nil CustomHeaders in request must preserve existing headers")
}
func TestUpdateMCPService_CustomHeadersReplaceOnNonNil(t *testing.T) {
ctx := context.Background()
svc, repo := newTestService()
id := seedService(t, repo, "stored-api", "stored-token")
repo.store[id].AuthConfig.CustomHeaders = map[string]string{"X-Tenant": "acme"}
upd := buildFullUpdate(id, &types.MCPAuthConfig{
APIKey: types.RedactedSecretPlaceholder,
Token: types.RedactedSecretPlaceholder,
CustomHeaders: map[string]string{"X-Replaced": "yes"},
})
require.NoError(t, svc.UpdateMCPService(ctx, upd))
got := repo.store[id]
assert.Equal(t, map[string]string{"X-Replaced": "yes"}, got.AuthConfig.CustomHeaders,
"non-nil CustomHeaders must replace the stored map")
}