acf053faf3
背景:发布管理把"逐条核验"的仪式强加给每一份文档。实测一份文档要 9~11 次交互,
且有两处顺序陷阱(冻结后批准 → 建批 422;生成即固化 → 批准被永久钉住)。
P0-1 建批时冻结:
- CreateReleaseBatch 在未指定版本时,自己把当前实时计划冻结成一份**已确认**版本并绑定
(同一事务;同摘要复用最近版本,planned 或 confirmed 都可复用,因此同一份计划不堆版本)。
- 语义不变:批一旦建立,清单/摘要/验收轮次/目标库绑定全部不可变、回滚按它走;变的只是
"谁在什么时候按固化键"——从操作员的手动两步变成建批这一下。
- 效果:去掉「生成新版本」「确认冻结」两步;消灭"批准晚于冻结 → 422"这类顺序陷阱;
前端不再传版本 id,也不再有"所选知识不在冻结清单里"的阻断。
- 计划→清单项的映射抽成 ChangeSetItemsFromPlan 供两处共用,否则两处算出的摘要不同,
"同一计划复用同一版本"永远不命中。
P0-2 验收通过自动批准:
- 验收三项结论达发布门槛(knowledge_exists/chunks_present=passed,retrieval_hit 与门槛同
口径)时,由 worker 写入 reviewer_ref='system:auto' 的批准记录,替代"每份文档点一次批准"。
- 策略 TEST_AUTO_REVIEW_POLICY = off | internal(默认,仅对内)| all;值不认识拒绝启动。
只影响审核这一步:生产授权、双闸与发布前快照门禁不受影响。界面上只读展示当前策略。
- 安全边界(写死在审核 SQL 里):系统只允许写"第一条"审核——人工一旦表过态,机器不得用一条
更晚的 approved 把人工结论翻回去;人工随时可再审。集成测试去掉该约束即 FAIL(已实测)。
- 自动批准自身的任何失败都不影响验收结论,只上报。
顺带(P1-1 前半,已被 P0-1 取代的部分已删除):
- 写操作失败把服务端 {"error": ...} 的原因带出来,并对已知原因给出下一步(不再只给 HTTP 状态码)。
- 当前的"不在已冻结清单里"标注与就地修复按钮随 P0-1 一起移除(该失败模式已不存在)。
测试:bind_time_freeze_integration_test.go(绑定已确认版本/同计划复用/冻结后不可变/
动作不符仍 NotEligible)、auto_review_integration_test.go(系统只写第一条、人工可再审)、
validate_test.go(对内自动/对外不自动/策略关闭/验收未过不批/自身失败不影响验收)、
auto_review_test.go、config_test 及前端能力解析测试。
实测耗时:本机 Go 全量 26 包通过、前端 178 条通过。
316 lines
15 KiB
Go
316 lines
15 KiB
Go
package appconfig
|
|
|
|
import (
|
|
"errors"
|
|
"fmt"
|
|
"path/filepath"
|
|
"strconv"
|
|
"strings"
|
|
"time"
|
|
|
|
"knowledge-governance/internal/governance"
|
|
)
|
|
|
|
type Config struct {
|
|
DatabaseURL string
|
|
AppAddr string
|
|
WorkerID string
|
|
PollInterval time.Duration
|
|
LeaseDuration time.Duration
|
|
RetryDelay time.Duration
|
|
SourceRefreshSchedulerInterval time.Duration
|
|
ReleaseWorkerEnabled bool
|
|
ReleaseConfigFile string
|
|
ProductionReleaseRequestsEnabled bool
|
|
ProductionBackupRestoreVerified bool
|
|
ProductionReleaseAuthorizerRef string
|
|
ProductionReleaseAuthTokenFile string
|
|
// Production WeKnora endpoint: the release target itself stays declared in
|
|
// the release runtime config (RELEASE_CONFIG_FILE); these settings only let
|
|
// the API probe that endpoint so the console can show its real state.
|
|
ProductionWeKnoraBaseURL string
|
|
ProductionWeKnoraAPIKeyFile string
|
|
ProductionWeKnoraAllowDockerHTTP bool
|
|
PPScanEnabled bool
|
|
PPScanAllowedRoot string
|
|
PPScanLedgerPath string
|
|
PPScanManifestPath string
|
|
TestWeKnoraEnabled bool
|
|
TestWeKnoraAllowDockerHTTP bool
|
|
TestWeKnoraBaseURL string
|
|
MinerUBaseURL string
|
|
TestWeKnoraKBID string
|
|
InternalRoutingKnowledgeBaseID string
|
|
ExternalRoutingKnowledgeBaseID string
|
|
TestWeKnoraAPIKeyFile string
|
|
GovernanceSpoolRoot string
|
|
IngestionSpoolRoot string
|
|
ArtifactFetchMaxBytes int64
|
|
WorkerLivenessStaleAfter time.Duration
|
|
TestWeKnoraHTTPTimeout time.Duration
|
|
TestWeKnoraPollInitial time.Duration
|
|
TestWeKnoraPollMax time.Duration
|
|
TestWeKnoraStalledAfter time.Duration
|
|
TestWeKnoraMaxUploadBytes int64
|
|
TestAcceptancePolicyVersion string
|
|
// TestAutoReviewPolicy:验收通过后是否自动批准(off|internal|all;默认 internal)。
|
|
TestAutoReviewPolicy string
|
|
// 目标库所在主机(整库逻辑备份真正执行的地方)。凭据是挂载进来的 0600 ssh
|
|
// 配置文件与私钥;三项同时给出才接线,缺任一项即"未接线"(页面降级展示)。
|
|
BackupHostSSHConfig string
|
|
BackupHostTarget string
|
|
BackupHostScriptDir string
|
|
BackupHostRoot string
|
|
}
|
|
|
|
func Load(getenv func(string) string) (Config, error) {
|
|
config := Config{
|
|
DatabaseURL: getenv("DATABASE_URL"),
|
|
AppAddr: valueOrDefault(getenv("APP_ADDR"), ":8080"),
|
|
WorkerID: valueOrDefault(getenv("WORKER_ID"), "worker-1"),
|
|
PollInterval: time.Second,
|
|
LeaseDuration: 30 * time.Second,
|
|
RetryDelay: 5 * time.Second,
|
|
SourceRefreshSchedulerInterval: 30 * time.Second,
|
|
ReleaseConfigFile: strings.TrimSpace(getenv("RELEASE_CONFIG_FILE")),
|
|
ProductionReleaseAuthorizerRef: strings.TrimSpace(getenv("PRODUCTION_RELEASE_AUTHORIZER_REF")),
|
|
ProductionReleaseAuthTokenFile: strings.TrimSpace(getenv("PRODUCTION_RELEASE_AUTH_TOKEN_FILE")),
|
|
ProductionWeKnoraBaseURL: strings.TrimSpace(getenv("PRODUCTION_WEKNORA_BASE_URL")),
|
|
ProductionWeKnoraAPIKeyFile: strings.TrimSpace(getenv("PRODUCTION_WEKNORA_API_KEY_FILE")),
|
|
PPScanAllowedRoot: strings.TrimSpace(getenv("PP_SCAN_ALLOWED_ROOT")),
|
|
PPScanLedgerPath: strings.TrimSpace(getenv("PP_SCAN_LEDGER_PATH")),
|
|
PPScanManifestPath: strings.TrimSpace(getenv("PP_SCAN_MANIFEST_PATH")),
|
|
TestWeKnoraBaseURL: strings.TrimSpace(getenv("TEST_WEKNORA_BASE_URL")),
|
|
MinerUBaseURL: strings.TrimSpace(getenv("MINERU_BASE_URL")),
|
|
TestWeKnoraKBID: strings.TrimSpace(getenv("TEST_WEKNORA_KB_ID")),
|
|
InternalRoutingKnowledgeBaseID: strings.TrimSpace(getenv("TEST_INTERNAL_WEKNORA_KB_ID")),
|
|
ExternalRoutingKnowledgeBaseID: strings.TrimSpace(getenv("TEST_EXTERNAL_WEKNORA_KB_ID")),
|
|
TestWeKnoraAPIKeyFile: valueOrDefault(getenv("TEST_WEKNORA_API_KEY_FILE"), "/run/secrets/test_weknora_api_key"),
|
|
GovernanceSpoolRoot: valueOrDefault(getenv("GOVERNANCE_SPOOL_ROOT"), "/var/lib/governance/spool"),
|
|
IngestionSpoolRoot: strings.TrimSpace(getenv("INGESTION_SPOOL_ROOT")),
|
|
ArtifactFetchMaxBytes: 200 * 1024 * 1024,
|
|
WorkerLivenessStaleAfter: 2 * time.Minute,
|
|
TestWeKnoraHTTPTimeout: 60 * time.Second,
|
|
TestWeKnoraPollInitial: 10 * time.Second,
|
|
TestWeKnoraPollMax: time.Minute,
|
|
TestWeKnoraStalledAfter: 30 * time.Minute,
|
|
TestWeKnoraMaxUploadBytes: 50 * 1024 * 1024,
|
|
TestAcceptancePolicyVersion: valueOrDefault(getenv("TEST_ACCEPTANCE_POLICY_VERSION"), "structural-v1"),
|
|
TestAutoReviewPolicy: valueOrDefault(getenv("TEST_AUTO_REVIEW_POLICY"), string(governance.AutoReviewInternal)),
|
|
// 整库逻辑备份的目标主机:三项同时给出才接线(脚本目录/备份根有默认值)。
|
|
BackupHostSSHConfig: strings.TrimSpace(getenv("BACKUP_HOST_SSH_CONFIG")),
|
|
BackupHostTarget: strings.TrimSpace(getenv("BACKUP_HOST_TARGET")),
|
|
BackupHostScriptDir: valueOrDefault(strings.TrimSpace(getenv("BACKUP_HOST_SCRIPT_DIR")), "/opt/weknora-test"),
|
|
BackupHostRoot: valueOrDefault(strings.TrimSpace(getenv("BACKUP_HOST_ROOT")), "/srv/kb-backups"),
|
|
}
|
|
if config.DatabaseURL == "" {
|
|
return Config{}, errors.New("DATABASE_URL is required")
|
|
}
|
|
var err error
|
|
if config.PollInterval, err = durationOrDefault(getenv("WORKER_POLL_INTERVAL"), config.PollInterval); err != nil {
|
|
return Config{}, fmt.Errorf("WORKER_POLL_INTERVAL: %w", err)
|
|
}
|
|
if config.LeaseDuration, err = durationOrDefault(getenv("WORKER_LEASE_DURATION"), config.LeaseDuration); err != nil {
|
|
return Config{}, fmt.Errorf("WORKER_LEASE_DURATION: %w", err)
|
|
}
|
|
if config.RetryDelay, err = durationOrDefault(getenv("WORKER_RETRY_DELAY"), config.RetryDelay); err != nil {
|
|
return Config{}, fmt.Errorf("WORKER_RETRY_DELAY: %w", err)
|
|
}
|
|
if config.SourceRefreshSchedulerInterval, err = durationOrDefault(getenv("SOURCE_REFRESH_SCHEDULER_INTERVAL"), config.SourceRefreshSchedulerInterval); err != nil {
|
|
return Config{}, fmt.Errorf("SOURCE_REFRESH_SCHEDULER_INTERVAL: %w", err)
|
|
}
|
|
if config.SourceRefreshSchedulerInterval < 5*time.Second {
|
|
return Config{}, errors.New("SOURCE_REFRESH_SCHEDULER_INTERVAL must be at least 5s")
|
|
}
|
|
if raw := getenv("RELEASE_WORKER_ENABLED"); raw != "" {
|
|
config.ReleaseWorkerEnabled, err = strconv.ParseBool(raw)
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("RELEASE_WORKER_ENABLED: %w", err)
|
|
}
|
|
}
|
|
if config.ReleaseWorkerEnabled && (!filepath.IsAbs(config.ReleaseConfigFile) || filepath.Clean(config.ReleaseConfigFile) != config.ReleaseConfigFile) {
|
|
return Config{}, errors.New("RELEASE_CONFIG_FILE must be an absolute path when release worker is enabled")
|
|
}
|
|
if raw := getenv("PRODUCTION_RELEASE_REQUESTS_ENABLED"); raw != "" {
|
|
config.ProductionReleaseRequestsEnabled, err = strconv.ParseBool(raw)
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("PRODUCTION_RELEASE_REQUESTS_ENABLED: %w", err)
|
|
}
|
|
}
|
|
if raw := getenv("PRODUCTION_BACKUP_RESTORE_VERIFIED"); raw != "" {
|
|
config.ProductionBackupRestoreVerified, err = strconv.ParseBool(raw)
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("PRODUCTION_BACKUP_RESTORE_VERIFIED: %w", err)
|
|
}
|
|
}
|
|
if raw := getenv("PRODUCTION_WEKNORA_ALLOW_DOCKER_HTTP"); raw != "" {
|
|
config.ProductionWeKnoraAllowDockerHTTP, err = strconv.ParseBool(raw)
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("PRODUCTION_WEKNORA_ALLOW_DOCKER_HTTP: %w", err)
|
|
}
|
|
}
|
|
if config.ProductionWeKnoraBaseURL != "" {
|
|
if err := validateHTTPBaseURL(config.ProductionWeKnoraBaseURL); err != nil {
|
|
return Config{}, fmt.Errorf("PRODUCTION_WEKNORA_BASE_URL: %w", err)
|
|
}
|
|
}
|
|
if config.ProductionWeKnoraAPIKeyFile != "" {
|
|
if config.ProductionWeKnoraBaseURL == "" {
|
|
return Config{}, errors.New("PRODUCTION_WEKNORA_API_KEY_FILE requires PRODUCTION_WEKNORA_BASE_URL")
|
|
}
|
|
if !filepath.IsAbs(config.ProductionWeKnoraAPIKeyFile) {
|
|
return Config{}, errors.New("PRODUCTION_WEKNORA_API_KEY_FILE must be an absolute path")
|
|
}
|
|
}
|
|
if raw := getenv("PP_SCAN_ENABLED"); raw != "" {
|
|
config.PPScanEnabled, err = strconv.ParseBool(raw)
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("PP_SCAN_ENABLED: %w", err)
|
|
}
|
|
}
|
|
if raw := getenv("TEST_WEKNORA_ENABLED"); raw != "" {
|
|
config.TestWeKnoraEnabled, err = strconv.ParseBool(raw)
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("TEST_WEKNORA_ENABLED: %w", err)
|
|
}
|
|
}
|
|
if raw := getenv("TEST_WEKNORA_ALLOW_DOCKER_HTTP"); raw != "" {
|
|
config.TestWeKnoraAllowDockerHTTP, err = strconv.ParseBool(raw)
|
|
if err != nil {
|
|
return Config{}, fmt.Errorf("TEST_WEKNORA_ALLOW_DOCKER_HTTP: %w", err)
|
|
}
|
|
}
|
|
if config.WorkerLivenessStaleAfter, err = durationOrDefault(getenv("WORKER_LIVENESS_STALE_AFTER"), config.WorkerLivenessStaleAfter); err != nil {
|
|
return Config{}, fmt.Errorf("WORKER_LIVENESS_STALE_AFTER: %w", err)
|
|
}
|
|
if config.TestWeKnoraHTTPTimeout, err = durationOrDefault(getenv("TEST_WEKNORA_HTTP_TIMEOUT"), config.TestWeKnoraHTTPTimeout); err != nil {
|
|
return Config{}, fmt.Errorf("TEST_WEKNORA_HTTP_TIMEOUT: %w", err)
|
|
}
|
|
if config.TestWeKnoraPollInitial, err = durationOrDefault(getenv("TEST_WEKNORA_POLL_INITIAL"), config.TestWeKnoraPollInitial); err != nil {
|
|
return Config{}, fmt.Errorf("TEST_WEKNORA_POLL_INITIAL: %w", err)
|
|
}
|
|
if config.TestWeKnoraPollMax, err = durationOrDefault(getenv("TEST_WEKNORA_POLL_MAX"), config.TestWeKnoraPollMax); err != nil {
|
|
return Config{}, fmt.Errorf("TEST_WEKNORA_POLL_MAX: %w", err)
|
|
}
|
|
if config.TestWeKnoraStalledAfter, err = durationOrDefault(getenv("TEST_WEKNORA_STALLED_AFTER"), config.TestWeKnoraStalledAfter); err != nil {
|
|
return Config{}, fmt.Errorf("TEST_WEKNORA_STALLED_AFTER: %w", err)
|
|
}
|
|
if config.TestWeKnoraMaxUploadBytes, err = positiveInt64OrDefault(getenv("TEST_WEKNORA_MAX_UPLOAD_BYTES"), config.TestWeKnoraMaxUploadBytes); err != nil {
|
|
return Config{}, fmt.Errorf("TEST_WEKNORA_MAX_UPLOAD_BYTES: %w", err)
|
|
}
|
|
if config.TestWeKnoraMaxUploadBytes > 50*1024*1024 {
|
|
return Config{}, errors.New("TEST_WEKNORA_MAX_UPLOAD_BYTES must not exceed 50 MiB")
|
|
}
|
|
if config.TestWeKnoraPollMax < config.TestWeKnoraPollInitial {
|
|
return Config{}, errors.New("TEST_WEKNORA_POLL_MAX must not be less than TEST_WEKNORA_POLL_INITIAL")
|
|
}
|
|
if config.TestWeKnoraStalledAfter <= config.TestWeKnoraPollMax {
|
|
return Config{}, errors.New("TEST_WEKNORA_STALLED_AFTER must be greater than TEST_WEKNORA_POLL_MAX")
|
|
}
|
|
config.TestAcceptancePolicyVersion = strings.TrimSpace(config.TestAcceptancePolicyVersion)
|
|
if config.TestAcceptancePolicyVersion == "" {
|
|
return Config{}, errors.New("TEST_ACCEPTANCE_POLICY_VERSION must not be empty")
|
|
}
|
|
// 审核策略:值不认识就拒绝启动,避免"写错一个词 → 静默变成另一种审核强度"。
|
|
normalizedPolicy, err := governance.ParseAutoReviewPolicy(config.TestAutoReviewPolicy)
|
|
if err != nil {
|
|
return Config{}, errors.New("TEST_AUTO_REVIEW_POLICY must be one of off, internal, all")
|
|
}
|
|
config.TestAutoReviewPolicy = string(normalizedPolicy)
|
|
if config.ProductionReleaseRequestsEnabled {
|
|
// 目标主机凭据要么都给要么都不给:只给一半会把"未接线"伪装成"已接线"。
|
|
if (config.BackupHostSSHConfig == "") != (config.BackupHostTarget == "") {
|
|
return Config{}, errors.New("BACKUP_HOST_SSH_CONFIG and BACKUP_HOST_TARGET must be set together")
|
|
}
|
|
if config.ProductionReleaseAuthorizerRef == "" || len(config.ProductionReleaseAuthorizerRef) > 128 || strings.ContainsAny(config.ProductionReleaseAuthorizerRef, "\x00\r\n ") {
|
|
return Config{}, errors.New("PRODUCTION_RELEASE_AUTHORIZER_REF is required and must be safe when production release requests are enabled")
|
|
}
|
|
if config.ProductionReleaseAuthTokenFile == "" || !filepath.IsAbs(config.ProductionReleaseAuthTokenFile) {
|
|
return Config{}, errors.New("PRODUCTION_RELEASE_AUTH_TOKEN_FILE must be an absolute path when production release requests are enabled")
|
|
}
|
|
}
|
|
if config.PPScanEnabled {
|
|
for key, value := range map[string]string{
|
|
"PP_SCAN_ALLOWED_ROOT": config.PPScanAllowedRoot,
|
|
"PP_SCAN_LEDGER_PATH": config.PPScanLedgerPath,
|
|
"PP_SCAN_MANIFEST_PATH": config.PPScanManifestPath,
|
|
} {
|
|
if value == "" {
|
|
return Config{}, fmt.Errorf("%s is required when PP scan is enabled", key)
|
|
}
|
|
if !filepath.IsAbs(value) {
|
|
return Config{}, fmt.Errorf("%s must be absolute", key)
|
|
}
|
|
}
|
|
}
|
|
if config.TestWeKnoraEnabled {
|
|
if config.TestWeKnoraBaseURL == "" {
|
|
return Config{}, errors.New("TEST_WEKNORA_BASE_URL is required when test WeKnora is enabled")
|
|
}
|
|
if config.TestWeKnoraKBID == "" {
|
|
return Config{}, errors.New("TEST_WEKNORA_KB_ID is required when test WeKnora is enabled")
|
|
}
|
|
if strings.TrimSpace(config.TestWeKnoraAPIKeyFile) == "" {
|
|
return Config{}, errors.New("TEST_WEKNORA_API_KEY_FILE is required when test WeKnora is enabled")
|
|
}
|
|
if strings.TrimSpace(config.GovernanceSpoolRoot) == "" {
|
|
return Config{}, errors.New("GOVERNANCE_SPOOL_ROOT is required when test WeKnora is enabled")
|
|
}
|
|
}
|
|
if (config.InternalRoutingKnowledgeBaseID == "") != (config.ExternalRoutingKnowledgeBaseID == "") {
|
|
return Config{}, errors.New("both internal and external routing knowledge bases must be configured together")
|
|
}
|
|
for _, knowledgeBaseID := range []string{config.InternalRoutingKnowledgeBaseID, config.ExternalRoutingKnowledgeBaseID} {
|
|
if len(knowledgeBaseID) > 128 || strings.ContainsAny(knowledgeBaseID, "\x00\r\n /?#\\") {
|
|
return Config{}, errors.New("routing knowledge base ID is invalid")
|
|
}
|
|
}
|
|
return config, nil
|
|
}
|
|
|
|
// validateHTTPBaseURL accepts only a bare http(s) origin: no credentials, query
|
|
// or fragment, so a probe address can never smuggle in secrets. The rule itself
|
|
// lives with the domain type because the web-configured endpoint must obey the
|
|
// exact same shape.
|
|
func validateHTTPBaseURL(value string) error {
|
|
if !governance.ValidHTTPOrigin(value) {
|
|
return errors.New("must be an http(s) origin without credentials, query or fragment")
|
|
}
|
|
return nil
|
|
}
|
|
|
|
func valueOrDefault(value, fallback string) string {
|
|
if value == "" {
|
|
return fallback
|
|
}
|
|
return value
|
|
}
|
|
|
|
func durationOrDefault(value string, fallback time.Duration) (time.Duration, error) {
|
|
if value == "" {
|
|
return fallback, nil
|
|
}
|
|
parsed, err := time.ParseDuration(value)
|
|
if err != nil {
|
|
return 0, err
|
|
}
|
|
if parsed <= 0 {
|
|
return 0, errors.New("must be positive")
|
|
}
|
|
return parsed, nil
|
|
}
|
|
|
|
func positiveInt64OrDefault(value string, fallback int64) (int64, error) {
|
|
if value == "" {
|
|
return fallback, nil
|
|
}
|
|
parsed, err := strconv.ParseInt(value, 10, 64)
|
|
if err != nil {
|
|
return 0, err
|
|
}
|
|
if parsed <= 0 {
|
|
return 0, errors.New("must be positive")
|
|
}
|
|
return parsed, nil
|
|
}
|