d869cc1ff6
Closes #988 Unify the "write-only secrets" pattern across MCP services, Models, WebSearch providers, and DataSources. Fix two related defects: 1. GET responses leaked plaintext credentials — GetMCPServiceByID while List masked, all Model / WebSearchProvider / DataSource GET responses unconditionally. 2. When the frontend echoed a masked placeholder back on save, the update path overwrote the stored secret with the mask — silently wiping the real credential. MCP surfaced this most visibly: ListMCPServices masked Token to "first4****last4" and McpServiceDialog round-tripped that masked string back into the DB. - internal/types/secret.go introduces the shared sentinel RedactedSecretPlaceholder = "***" and two helpers: - IsRedactedOrEmpty(s): true for "" or the sentinel - PreserveIfRedacted(incoming, existing): merge predicate - Each resource adds a RedactSensitiveData (mutates-in-place) or Redacted (returns-a-copy) method and a MergeUpdate contract: empty or placeholder preserves, any other value replaces, and a write-only Clear* bool flag (gorm:"-", json omitempty) wipes with precedence over preserve / replace. Clear* never persists. - Every GET / List / Create / Update response path now redacts. VectorStore's pre-existing "***" literal is refactored onto the shared constant for consistency. All four dialogs — McpServiceDialog, ModelEditorDialog, WebSearchSettings, DataSourceEditorDialog — never pre-fill secret inputs. The stored-state signal is collapsed into the input placeholder: "•••••••• (Enter new value to replace)" when the server reports a stored value, falling back to the connector-specific placeholder otherwise. The input itself carries the "something is there" cue, matching how password dots render. An opt-in "Remove this credential" checkbox appears in edit mode when a value is stored; checking it disables the input and triggers a danger-themed confirmation dialog before submit. DataSource rolls the clear up to a connector-level "Remove all credentials" flag since its credentials are per-connector atomic bundles (e.g. Feishu app_id + app_secret). i18n updates across en-US, zh-CN, ko-KR, ru-RU: secret.storedPlaceholder / clearHint / confirmClearTitle / confirmClearBody. - MCP update handler's manual map-to-struct projection was silently dropping clear_api_key / clear_token / custom_headers on partial updates. Pull them off the map explicitly. - MCP configChanged detection now snapshots URL / StdioConfig / TransportType before the merge so connection recycling still triggers on real transport changes (not just whenever AuthConfig is present). - MCP CustomHeaders: nil preserves existing, non-nil replaces. Was always overwriting, silently wiping headers on any auth-bearing update. - GetMCPServiceByID now redacts (was plaintext while List masked). - WebSearchProvider update preserves Name / Description when empty instead of clobbering them. - WebSearchProvider TestProviderRaw rejects the "***" placeholder up front with an actionable message rather than forwarding it to the upstream provider API as a literal key. - Model builtin hideSensitiveInfo now preserves non-secret metadata (SupportsVision, Provider, ExtraConfig, InterfaceType, ParameterSize, EmbeddingParameters) that were previously zeroed out for builtin model cards. - Replace local equalStringSlices / equalStringMaps helpers with stdlib slices.Equal / maps.Equal (Go 1.21+). - internal/types/{secret,mcp,model,web_search_provider,datasource}_test.go: table-driven coverage of merge, redact, clear, and write-only flag invariants for every resource. - internal/application/service/mcp_service_test.go: full service layer with a fake repo; covers ListMCPServices / GetMCPServiceByID redaction, UpdateMCPService partial-update honoring of Clear*, CustomHeaders preserve-on-nil / replace-on-non-nil, and clear-flag precedence over submitted values. Additive only. No changes to request or response JSON shape beyond the write-only Clear* flags (omitted by default via omitempty). No DB schema changes. No SDK struct field removals. Existing clients that don't know about the new flags continue to function unchanged. - DataSource.Config jsonb at-rest encryption. Pre-existing residual risk; tracked as a follow-up. - Renaming VectorStore.MaskSensitiveFields to match the new RedactSensitiveData / Redacted naming convention. Left alone to minimize diff; a follow-up rename PR can unify.
WeKnora HTTP Client
This package provides a client library for interacting with WeKnora services, supporting all HTTP-based interface calls, making it easier for other modules to integrate with WeKnora services without having to write HTTP request code directly.
Main Features
The client includes the following main functional modules:
- Session Management: Create, retrieve, update, and delete sessions
- Knowledge Base Management: Create, retrieve, update, and delete knowledge bases
- Knowledge Management: Add, retrieve, and delete knowledge content
- Tenant Management: CRUD operations for tenants
- Knowledge Q&A: Supports regular Q&A and streaming Q&A
- Chunk Management: Query, update, and delete knowledge chunks
- Message Management: Retrieve and delete session messages
- Model Management: Create, retrieve, update, and delete models
- Evaluation Function: Start evaluation tasks and get evaluation results
Usage
Creating Client Instance
import (
"context"
"github.com/Tencent/WeKnora/client"
"time"
)
// Create client instance
apiClient := client.NewClient(
"http://api.example.com",
client.WithToken("your-auth-token"),
client.WithTimeout(30*time.Second),
)
Tenant Configuration
You can set a default tenant with WithTenantID; the client will automatically send the X-Tenant-ID header:
tenantID := uint64(10000)
apiClient := client.NewClient(
"http://api.example.com",
client.WithToken("your-auth-token"),
client.WithTenantID(tenantID),
)
If a single request needs a different tenant, set TenantID in the request context. The value can be a uint64, *uint64, or a numeric string, and it will take precedence over the client default:
ctx := context.WithValue(context.Background(), "TenantID", uint64(10000))
// Pass ctx into any client method to switch to tenant 10000 for that request
Example: Create Knowledge Base and Upload File
// Create knowledge base
kb := &client.KnowledgeBase{
Name: "Test Knowledge Base",
Description: "This is a test knowledge base",
ChunkingConfig: client.ChunkingConfig{
ChunkSize: 500,
ChunkOverlap: 50,
Separators: []string{"\n\n", "\n", ". ", "? ", "! "},
},
ImageProcessingConfig: client.ImageProcessingConfig{
ModelID: "image_model_id",
},
EmbeddingModelID: "embedding_model_id",
SummaryModelID: "summary_model_id",
}
kb, err := apiClient.CreateKnowledgeBase(context.Background(), kb)
if err != nil {
// Handle error
}
// Upload knowledge file with metadata
metadata := map[string]string{
"source": "local",
"type": "document",
}
knowledge, err := apiClient.CreateKnowledgeFromFile(context.Background(), kb.ID, "path/to/file.pdf", metadata)
if err != nil {
// Handle error
}
Example: Create Session and Chat
// Create session
sessionRequest := &client.CreateSessionRequest{
KnowledgeBaseID: knowledgeBaseID,
SessionStrategy: &client.SessionStrategy{
MaxRounds: 10,
EnableRewrite: true,
FallbackStrategy: "fixed_answer",
FallbackResponse: "Sorry, I cannot answer this question",
EmbeddingTopK: 5,
KeywordThreshold: 0.5,
VectorThreshold: 0.7,
RerankModelID: "rerank_model_id",
RerankTopK: 3,
RerankThreshold: 0.8,
SummaryModelID: "summary_model_id",
},
}
session, err := apiClient.CreateSession(context.Background(), sessionRequest)
if err != nil {
// Handle error
}
// Regular Q&A
answer, err := apiClient.KnowledgeQA(context.Background(), session.ID, &client.KnowledgeQARequest{
Query: "What is artificial intelligence?",
})
if err != nil {
// Handle error
}
// Streaming Q&A
err = apiClient.KnowledgeQAStream(context.Background(), session.ID, "What is machine learning?", func(response *client.StreamResponse) error {
// Handle each response chunk
fmt.Print(response.Content)
return nil
})
if err != nil {
// Handle error
}
Example: Managing Models
// Create model
modelRequest := &client.CreateModelRequest{
Name: "Test Model",
Type: client.ModelTypeChat,
Source: client.ModelSourceInternal,
Description: "This is a test model",
Parameters: client.ModelParameters{
"temperature": 0.7,
"top_p": 0.9,
},
IsDefault: true,
}
model, err := apiClient.CreateModel(context.Background(), modelRequest)
if err != nil {
// Handle error
}
// List all models
models, err := apiClient.ListModels(context.Background())
if err != nil {
// Handle error
}
Example: Managing Knowledge Chunks
// List knowledge chunks
chunks, total, err := apiClient.ListKnowledgeChunks(context.Background(), knowledgeID, 1, 10)
if err != nil {
// Handle error
}
// Update chunk
updateRequest := &client.UpdateChunkRequest{
Content: "Updated chunk content",
IsEnabled: true,
}
updatedChunk, err := apiClient.UpdateChunk(context.Background(), knowledgeID, chunkID, updateRequest)
if err != nil {
// Handle error
}
Example: Getting Session Messages
// Get recent messages
messages, err := apiClient.GetRecentMessages(context.Background(), sessionID, 10)
if err != nil {
// Handle error
}
// Get messages before a specific time
beforeTime := time.Now().Add(-24 * time.Hour)
olderMessages, err := apiClient.GetMessagesBefore(context.Background(), sessionID, beforeTime, 10)
if err != nil {
// Handle error
}
Complete Example
Please refer to the ExampleUsage function in the example.go file, which demonstrates the complete usage flow of the client.