Compare commits

..

6 Commits

Author SHA1 Message Date
狐友 GXXP & Miku ad02e8bb9c feat(settings): 页面录入测试/生产 WeKnora 连接与密钥并收敛可达性判定
build / build (push) Successful in 1m43s
连接(地址 + 写-only 密钥)从文件引用搬进治理库:AES-GCM 密文 + 独立 0600 主密钥,
页面与接口只回显尾 4 位;测试连接真实列库并区分 地址不可达/密钥无效/无权限/成功,
换地址或换密钥必须测通才能保存(迁移 000083)。

可达性判定收敛为 weknora.HTTPAllowed,客户端与监控探活共用;新增
WEKNORA_ALLOW_PRIVATE_HTTP 只放行 RFC1918/回环/链路本地。生产绑定只保留下拉,
保存后回报冻结/未冻结条数,已冻结目标不被改写。
2026-11-22 15:18:34 +08:00
zheromiku edd07a65d6 Merge pull request 'docs(ops): 更新架构、平台接入及反代运维手册' (#16) from docs/governance-ops-090-main into main
build / build (push) Successful in 1m28s
2026-11-22 13:17:09 +08:00
姚章浩 7c120b3adc test(qawemail): 用绝对域名断言不可解析主机
build / build (push) Successful in 1m57s
构建机 resolv.conf 带 search 域(Tailscale tailf49802.ts.net)时,
smtp.kg-test.invalid 会被拼成 ...tailf49802.ts.net 并由 Fake-IP DNS
解析成功,断言实际测的是构建机搜索域。改成带末尾点的绝对名后与
搜索域无关,仍保持“不可解析主机必须判非法”的原意。
2026-11-22 13:14:52 +08:00
姚章浩 ea159ca2fb docs(ops): 补充治理架构、平台接入与测试反代手册
build / build (push) Failing after 1m37s
2026-11-22 13:10:44 +08:00
zheromiku 034c0b3a1c Merge pull request 'fix(library): 跨源同步按目标范围改写并校验残留' (#13) from fix/library-cross-scope-sync-residue into main
build / build (push) Successful in 1m18s
2026-10-29 22:13:49 +08:00
姚章浩 5defe238e0 fix(library): 跨范围同步按目标范围改写并校验残留
build / build (push) Successful in 1m34s
真实事故:源 KB(10000/0c84a098) 同步到目标 KB(10001/e64e82b5) 时,归档行被原样
插入目标库,落在来源租户/知识库;资源阶段随即报"生产缺少资源 ... 的存储路径"
(资源查询按目标租户过滤,看不见刚写进去的行)。回滚只按目标范围回放,于是页面
报"已恢复到预备份",目标却多出 4 文件/38 分块/21 资源/4 绑定。

改:
- 写入与校验共用 retargetRow:tenant_id / knowledge_base_id / 资源定位符(后端、
  provider、桶、租户前缀) / storage_backend_id / provider / embedding_model_id
  统一映射到目标范围;行里的来源身份对不上就失败,宁可不写。
- Verify 增加"本次写集合仍以来源身份残留"判定;Restore 增加整库指纹比对(预备份
  时刻 vs 恢复后)与孤儿资源清理,回滚后用 Verify 复核,恢复工具退出 0 不再等于
  恢复完成。
- 新增 cmd/libraryresidue:按归档身份精确、可备份、可复核地清理已知的范围外残留。

验证:make verify 全绿;真实 PG+ES 用例复现"源目标 KB 不同 + 目标非空 + 资源阶段
失败",断言回滚后无来源范围残留、其它租户行未动、恢复后内容=预备份;正向用例断言
行落在目标范围且资源定位符指向目标存储。
2026-10-29 22:11:59 +08:00
56 changed files with 4490 additions and 285 deletions
+6
View File
@@ -43,6 +43,12 @@ PRODUCTION_RELEASE_REQUESTS_ENABLED=false
PRODUCTION_WEKNORA_BASE_URL=
PRODUCTION_WEKNORA_API_KEY_FILE=
PRODUCTION_WEKNORA_ALLOW_DOCKER_HTTP=false
# 连接(地址 + 写-only 密钥)密文存治理库;主密钥是独立的 0600 32 字节文件。
WEKNORA_BINDING_MASTER_KEY_FILE=
# 同网段 http(生产 18093 反代)放行:只接受 RFC1918/回环/链路本地,公网 http 仍拒绝。
WEKNORA_ALLOW_PRIVATE_HTTP=false
# 可信 TLS 反代之后录入密钥(X-Forwarded-Proto=https + 同源)。
WEKNORA_BINDING_TRUST_TLS_PROXY=false
PP_SCAN_ENABLED=false
PP_SCAN_HOST_DATA_ROOT=/home/zheromiku/dobot-portal-sync/data
PP_SCAN_CONTAINER_DATA_ROOT=/home/zheromiku/dobot-portal-sync/data
+11
View File
@@ -33,11 +33,22 @@
worker 完全相同的 `LIBRARY_*` 配置与维护协调器,绝不触碰封存源;任一步失败都
保持维护并写报告,不伪造通过。
- `internal/librarystack`:目标/预备份/维护栈的唯一装配点(同步、恢复、基线、演练共用)。
- **测试/生产 WeKnora 连接可在页面上配置**(`GET /api/v1/weknora-connections`、
`POST /api/v1/weknora-connections/{environment}/test`、
`PUT /api/v1/weknora-connections/{environment}`):API Key 以 AES-GCM 密文存治理库
(主密钥为独立 0600 文件),页面与接口只回显尾 4 位;「测试连接」真实请求该实例的
知识库列表并区分地址不可达 / 密钥无效(401) / 无权限(403) / 成功(库数量),换地址或
换密钥必须测通才能保存。密钥存储抽成 `internal/sealedsecret`,QA-AI 侧改为复用同一实现。
### Changed
- 基线作业的"已有生效版本"判定改为三种状态(own / foreign / none),并在
阶段里写明是否走"只收尾"续作。
- WeKnora 的可达性判定收敛为一份(`weknora.HTTPAllowed`):客户端与监控探活共用,
不再出现「卡片可达、列库失败」。新增 `WEKNORA_ALLOW_PRIVATE_HTTP` 只放行
RFC1918/回环/链路本地 http,公网 http 仍拒绝;生产 `18093` 同网段反代因此可用。
- 「生产知识库绑定」只保留下拉选择,删除手输 ID 的降级入口;保存后回报
「已冻结目标 / 仍可改指向」条数,已冻结的执行目标不被改写。
## 开发快照(首次发布入口:同步授权、目标权威基线与维护能力)
+21 -16
View File
@@ -71,15 +71,16 @@ IT、行政等是待扩展的业务资料范围,不是已经完成的新连接
## 访问与部署
### 已部署环境
### 运行架构与访问入口
Compose 中网页入口固定映射为 **宿主机 `8088` → Web 容器 `8080`**。网页和 `/api/` 使用同源入口。
| 组件 | 职责 | 接手时从这里看 |
|---|---|---|
| React/Vite + Web Nginx | 提供静态页面,同源转发 `/api/` 和 `/health` | [前端开发代理](web/vite.config.ts)、[Compose 的 Nginx 配置](deploy/nginx.conf) |
| Go API + 治理 PostgreSQL | 认证、角色、来源、绑定、审核、任务与发布记录。云测试使用独立逻辑库 `knowledge_governance_test` | [API](cmd/api)、[接口契约](api/openapi.yaml) |
| 普通 worker + 测试 WeKnora | worker 取件、上传、轮询并记录验收结果。WeKnora 负责解析、切片和索引 | [worker](cmd/worker)、[来源接入](docs/OPERATIONS_RUNBOOK.md#接入外部平台) |
| 发布 worker + 生产 WeKnora | 按已锁定的批次和目标执行受控发布与读回。问答端通过下游服务使用已发布知识 | [发布 worker](cmd/release-worker)、[操作手册](docs/OPERATIONS_RUNBOOK.md) |
- 本机访问:`http://127.0.0.1:8088/`
- 局域网访问:`http://<服务器内网IP>:8088/`
- 健康入口:`http://127.0.0.1:8088/health`
可以由 NAS 反向代理提供 HTTPS 域名入口,目标为平台服务器的 `8088`,保持路径原样转发。公网开放前必须验证鉴权、访问控制及转发头配置,不直接暴露数据库或内部管理接口。
从自己的电脑做系统设置验收,使用 SSH 转发后的 `http://127.0.0.1:28189/`。云测试主控制台使用受限内网入口 `http://100.85.111.121:8088/`。仓内 Compose 则将宿主 `8088` 映射到 Web 容器 `8080`。选择入口时先确认要看的部署。隧道命令、生产 `090` 服务地址和 HTTPS 反代示例见[运维手册](docs/OPERATIONS_RUNBOOK.md#入口与反向代理)。
**平台入口展示当前部署版本,不是每次 CI 的构建预览。** CI 静态预览由独立链路提供;CI 通过不会自动更新运行服务。
@@ -94,18 +95,12 @@ Compose 中网页入口固定映射为 **宿主机 `8088` → Web 容器 `8080`*
- 来源目录、spool 与运行状态挂载;按部署主机调整路径和 UID/GID。
- 测试 WeKnora 连接与对内/对外路由;生产发布按独立配置和授权接线。
准备完成后,由获授权的操作者执行:
```bash
docker compose up --build -d
```
主机需要提权时,由操作者按本机权限规则执行,不在自动化中默认使用 sudo。停止服务可用 `docker compose down`;**不要执行 `docker compose down -v`,除非明确批准删除数据库卷。**
仓内 Compose 含 `postgres` 演练服务。云测试和生产使用阿里云 RDS,部署前先核对目标逻辑库、当前迁移版本、待运行的迁移及所用的环境配置。API 和 worker 启动可执行迁移。已有环境先用 `docker compose config --services` 检查服务清单,按该环境的发布手册切换已验证的产物。前端开发可直接使用下面的 Vite 流程。
## 配置与安全边界
- 网页服务地址用于控制台探活与配置核验;真正的发布目标由受限发布运行时配置决定,两者不能混为一谈。
- 密钥通过文件或 secret 引用提供,不写入 Git、网页响应或日志。示例配置不是真实凭据。
- 测试/生产 WeKnora 的地址与 API Key 可在「系统设置 → 测试/生产知识库绑定」里录入:密钥以 AES-GCM 密文存治理库(主密钥是独立 0600 文件),页面与接口只回显尾 4 位,日志、审计与响应都不出现明文。部署期仍可用文件引用作为兜底;示例配置不是真实凭据。
- 生产请求与 Release Worker 默认关闭;启用状态必须现场确认。测试处理成功、自动审核或批次锁定都不能绕过生产门禁。
- 来源扫描与取件不应改写原始业务资料;目标知识库的发布、移除和回滚属于独立受控动作。
- 发布与恢复前必须按目标环境准备备份,并核验可恢复性。
@@ -114,7 +109,17 @@ docker compose up --build -d
## 开发与 CI
基础环境:Go 1.26、Node.js 22、npm,以及检查脚本所需的 Python 3 和工具链。依赖以仓库锁文件和实际 CI 配置为准。
开发环境使用 Go 1.26、Node.js 22、npm 和 Python 3。前端可独立开发。先准备一个获准用于开发的测试治理 API,在开发机的 `localhost:8088` 上确认 `/health` 可达。Vite 按仓内配置将 `/api` 和 `/health` 转给该地址,浏览器中的操作会作用于所连的环境。
```bash
# 仓库根目录,检查测试后端及前端依赖,再启动开发页面
curl -fsS --max-time 5 http://localhost:8088/health
cd web
npm ci
npm run dev -- --host 127.0.0.1
```
打开 Vite 打印的地址。修改组件后运行 `npm test` 和 `npm run build`。后端改动在仓库根目录用 `go test ./internal/httpapi` 这类按包测试缩小范围,再运行下方完整门禁。涉及持久化的联调使用独立测试 RDS 逻辑库,启动 API 或 worker 前核实库名和迁移版本。真实页面的通过条件见[运维手册](docs/OPERATIONS_RUNBOOK.md#开发测试和现场验收)。
```bash
make verify # 完整检查入口:本地与 CI 使用同一套规则
+162
View File
@@ -3387,6 +3387,117 @@
}
}
},
"/api/v1/weknora-connections": {
"get": {
"tags": [
"Settings"
],
"operationId": "listWeKnoraConnections",
"summary": "读取测试/生产 WeKnora 连接状态(只回显密钥尾4位)",
"responses": {
"200": {
"$ref": "#/components/responses/Envelope"
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"500": {
"$ref": "#/components/responses/InternalError"
}
}
}
},
"/api/v1/weknora-connections/{environment}/test": {
"post": {
"tags": [
"Settings"
],
"operationId": "testWeKnoraConnection",
"summary": "用真实列库区分地址不可达/密钥无效/无权限/成功(管理员)",
"parameters": [
{
"name": "environment",
"in": "path",
"required": true,
"schema": {
"type": "string",
"enum": [
"test",
"production"
]
}
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/WeKnoraConnectionTest"
}
}
}
},
"responses": {
"200": {
"$ref": "#/components/responses/Envelope"
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"500": {
"$ref": "#/components/responses/InternalError"
}
}
}
},
"/api/v1/weknora-connections/{environment}": {
"put": {
"tags": [
"Settings"
],
"operationId": "saveWeKnoraConnection",
"summary": "保存测通的连接;密钥密文入库,响应只回尾4位(管理员)",
"parameters": [
{
"$ref": "#/components/parameters/IdempotencyKey"
},
{
"name": "environment",
"in": "path",
"required": true,
"schema": {
"type": "string",
"enum": [
"test",
"production"
]
}
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/WeKnoraConnectionSave"
}
}
}
},
"responses": {
"200": {
"$ref": "#/components/responses/Envelope"
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"500": {
"$ref": "#/components/responses/InternalError"
}
}
}
},
"/api/v1/ui-settings": {
"get": {
"tags": [
@@ -5676,6 +5787,57 @@
}
},
"schemas": {
"WeKnoraConnectionTest": {
"type": "object",
"additionalProperties": false,
"properties": {
"endpoint": {
"type": "string"
},
"api_key": {
"type": "string",
"writeOnly": true
},
"key_action": {
"type": "string",
"enum": [
"keep",
"replace",
"clear"
]
}
}
},
"WeKnoraConnectionSave": {
"type": "object",
"additionalProperties": false,
"required": [
"endpoint",
"key_action",
"base_revision"
],
"properties": {
"endpoint": {
"type": "string"
},
"api_key": {
"type": "string",
"writeOnly": true
},
"key_action": {
"type": "string",
"enum": [
"keep",
"replace",
"clear"
]
},
"base_revision": {
"type": "integer",
"minimum": 0
}
}
},
"QAReportedHistoricalRate": {
"type": "object",
"additionalProperties": false,
+50 -8
View File
@@ -26,6 +26,7 @@ import (
"knowledge-governance/internal/qawemail"
"knowledge-governance/internal/serviceverify"
"knowledge-governance/internal/version"
"knowledge-governance/internal/weknoraconn"
)
// dingtalkSourceLister 是「系统设置 → 数据源配置」的数据源发现实现:只读列举钉钉知识库与
@@ -129,17 +130,26 @@ func run(ctx context.Context, getenv func(string) string) error {
}
return settings.TestWeKnoraBaseURL
}
monitorProvider, err := monitoring.NewRealProvider(monitoring.RealProviderConfig{DB: pool, WeKnoraBaseURL: config.TestWeKnoraBaseURL, TestWeKnoraResolver: testEndpointResolver, ProductionWeKnoraBaseURL: config.ProductionWeKnoraBaseURL, ProductionWeKnoraResolver: productionEndpointResolver, MinerUBaseURL: config.MinerUBaseURL, WeKnoraReadiness: weknoraReadiness, WorkerStaleAfter: config.WorkerLivenessStaleAfter})
// 「连接」是页面可配的一等状态:地址 + 写-only 密钥密文入库,其他功能只从
// 这里解析,不再各自读文件凭据,避免出现两个来源。
connectionService := weknoraconn.New(postgresstore.NewStore(pool), weknoraconn.Config{
MasterKeyFile: config.WeKnoraBindingMasterKeyFile,
AllowPrivateHTTP: config.WeKnoraAllowPrivateHTTP,
})
monitorProvider, err := monitoring.NewRealProvider(monitoring.RealProviderConfig{DB: pool, WeKnoraBaseURL: config.TestWeKnoraBaseURL, TestWeKnoraResolver: testEndpointResolver, ProductionWeKnoraBaseURL: config.ProductionWeKnoraBaseURL, ProductionWeKnoraResolver: productionEndpointResolver, MinerUBaseURL: config.MinerUBaseURL, WeKnoraReadiness: weknoraReadiness, WorkerStaleAfter: config.WorkerLivenessStaleAfter, AllowPrivateHTTP: config.WeKnoraAllowPrivateHTTP})
if err != nil {
return err
}
serviceTargetVerifier := serviceverify.New(serviceverify.Config{
TestAPIKeyFile: config.TestWeKnoraAPIKeyFile,
ProductionAPIKeyFile: config.ProductionWeKnoraAPIKeyFile,
TestBaseURLFallback: config.TestWeKnoraBaseURL,
ProductionBaseURLFallback: config.ProductionWeKnoraBaseURL,
TestAllowDockerHTTP: config.TestWeKnoraAllowDockerHTTP,
ProductionAllowDockerHTTP: config.ProductionWeKnoraAllowDockerHTTP,
TestAPIKeyFile: config.TestWeKnoraAPIKeyFile,
ProductionAPIKeyFile: config.ProductionWeKnoraAPIKeyFile,
TestBaseURLFallback: config.TestWeKnoraBaseURL,
ProductionBaseURLFallback: config.ProductionWeKnoraBaseURL,
TestAllowDockerHTTP: config.TestWeKnoraAllowDockerHTTP,
ProductionAllowDockerHTTP: config.ProductionWeKnoraAllowDockerHTTP,
ProductionAllowPrivateHTTP: config.WeKnoraAllowPrivateHTTP,
TestAllowPrivateHTTP: config.WeKnoraAllowPrivateHTTP,
ConnectionResolver: connectionService,
})
// 「版本记录」:只读查询面在 API 侧接线;真正的抓取在 worker 侧(未配置来源即未接线)。
versionHistoryStore, versionHistoryRepo := httpapi.VersionHistoryStore(nil), ""
@@ -185,10 +195,14 @@ func run(ctx context.Context, getenv func(string) string) error {
InternalRoutingKnowledgeBaseID: config.InternalRoutingKnowledgeBaseID,
ExternalRoutingKnowledgeBaseID: config.ExternalRoutingKnowledgeBaseID,
RoutingBindings: postgresstore.NewStore(pool),
KnowledgeBases: knowledgeBaseLister,
KnowledgeBases: connectionAwareBases{service: connectionService, fallback: knowledgeBaseLister, allowPrivate: config.WeKnoraAllowPrivateHTTP},
ProductionBindings: postgresstore.NewStore(pool),
ProductionKnowledgeBases: serviceTargetVerifier,
ProductionWeKnoraBaseURL: config.ProductionWeKnoraBaseURL,
WeKnoraConnections: connectionService,
WeKnoraAllowPrivateHTTP: config.WeKnoraAllowPrivateHTTP,
WeKnoraTrustTLSProxy: config.WeKnoraBindingTrustTLSProxy,
WeKnoraConnectionResolver: connectionService,
ServiceTargetVerifier: serviceTargetVerifier,
ProductionReleaseRequestsEnabled: config.ProductionReleaseRequestsEnabled,
TestAutoReviewPolicy: config.TestAutoReviewPolicy,
@@ -255,3 +269,31 @@ func run(ctx context.Context, getenv func(string) string) error {
return server.Shutdown(shutdownCtx)
}
}
// connectionAwareBases prefers the admin-managed connection over the bootstrap
// env credential, so 「测试知识库绑定」 lists the very instance the connection card
// probes — one source, not two.
type connectionAwareBases struct {
service *weknoraconn.Service
fallback httpapi.KnowledgeBaseLister
allowPrivate bool
}
func (lister connectionAwareBases) ListKnowledgeBases(ctx context.Context) ([]weknora.KnowledgeBase, error) {
if endpoint, apiKey, ok := lister.service.Resolve(ctx, weknoraconn.EnvironmentTest); ok {
client, err := weknora.NewClient(weknora.ClientConfig{
BaseURL: endpoint,
APIKey: apiKey,
Timeout: 10 * time.Second,
AllowPrivateHTTP: lister.allowPrivate,
})
if err != nil {
return nil, err
}
return client.ListKnowledgeBases(ctx)
}
if lister.fallback == nil {
return nil, errors.New("测试 WeKnora 未配置:请先在系统设置里录入连接")
}
return lister.fallback.ListKnowledgeBases(ctx)
}
+115
View File
@@ -0,0 +1,115 @@
// command libraryresidue 处理"整库同步写到目标范围外"留下的残留行:
//
// 读指定快照(来源范围)→ 在目标里找同内容、仍带来源身份的行
// → 先备份成 0600 文件并记录摘要 → 精确按归档身份删除 → 复核来源身份已空、
// 目标内容与预备份一致、整库指纹里没有别的租户
//
// 它不做范围广删,也不改任何别的东西:判据是"归档里确实有这些行、内容逐字段
// 一致",任何一行对不上就整体拒绝。默认是干跑(只报告、不删)。
//
// 真实事故:跨范围同步把 4 文件/38 分块/21 资源/4 绑定写成来源租户的行,
// 回滚只按目标范围回放,于是"已恢复"却留下这些行。这个工具的用途就是在
// 修复后的代码上线前,用可审计的方式清掉它们。
package main
import (
"context"
"encoding/json"
"flag"
"fmt"
"log"
"os"
"strconv"
"strings"
"knowledge-governance/internal/appconfig"
"knowledge-governance/internal/libraryrelease"
"knowledge-governance/internal/librarysnapshot"
"knowledge-governance/internal/librarystack"
v071 "knowledge-governance/internal/releaseadapter/v071"
)
func main() {
scope := flag.String("scope", "internal", "internal|external")
snapshot := flag.String("snapshot", "", "sealed snapshot whose source scope is the residue (required)")
backupSnapshot := flag.String("backup-snapshot", "", "prebackup snapshot the target must be restored to (required)")
expectSource := flag.String("expect-source", "", "expected source scope as <tenant>/<knowledge-base> (required guard against typos)")
out := flag.String("out", "", "directory for the residue backup (required)")
apply := flag.Bool("apply", false, "actually delete the residue rows (default: dry run)")
reportPath := flag.String("report", "", "write the JSON report here")
flag.Parse()
if *snapshot == "" || *backupSnapshot == "" || *out == "" || *expectSource == "" {
log.Fatal("snapshot, backup-snapshot, out and expect-source are required")
}
config, err := appconfig.Load(os.Getenv)
if err != nil {
log.Fatalf("load config: %v", err)
}
ctx := context.Background()
report, err := run(ctx, config, *scope, *snapshot, *backupSnapshot, *expectSource, *out, *apply)
if *reportPath != "" && report != nil {
raw, marshalErr := json.MarshalIndent(report, "", " ")
if marshalErr == nil {
raw = append(raw, '\n')
if writeErr := os.WriteFile(*reportPath, raw, 0o600); writeErr != nil {
log.Printf("report write failed: %v", writeErr)
}
}
}
if err != nil {
log.Fatalf("residue cleanup failed: %v", err)
}
log.Printf("residue cleanup ok: source=%s/%s tables=%v cleared=%d applied=%v verified=%v",
reportSource(report), report.SourceKB, report.Tables, report.Cleared, report.Applied, report.Verified)
}
func reportSource(report *v071.ResidueReport) string {
return strconv.FormatUint(report.SourceTenant, 10)
}
func run(ctx context.Context, config appconfig.Config, scope, snapshotID, backupSnapshotID, expectSource, out string,
apply bool) (*v071.ResidueReport, error) {
if scope != "internal" && scope != "external" {
return nil, fmt.Errorf("scope must be internal or external")
}
archive, err := librarysnapshot.Open(config.LibraryArchiveRoot, 256*1024*1024)
if err != nil {
return nil, fmt.Errorf("归档不可用: %w", err)
}
manifest, err := archive.Load(snapshotID)
archive.Close()
if err != nil {
return nil, fmt.Errorf("指定快照不可读: %w", err)
}
source := v071.ProductionScope{TenantID: manifest.TenantID, KnowledgeBaseID: manifest.KnowledgeBaseID}
want := strconv.FormatUint(source.TenantID, 10) + "/" + source.KnowledgeBaseID
// 范围写错会把"别人的行"当成残留:必须由调用方显式确认。
if strings.TrimSpace(expectSource) != want {
return nil, fmt.Errorf("来源范围与 -expect-source 不一致:归档为 %s,参数为 %s", want, expectSource)
}
target, _, err := librarystack.OpenProductionStack(ctx, config, scope, backupSnapshotID, "")
if err != nil {
return nil, fmt.Errorf("装配目标失败: %w", err)
}
defer target.Close()
report, err := target.ClearForeignScope(ctx, source, snapshotID, out, apply)
if err != nil {
return report, err
}
report.Foreign, _ = target.ForeignScopes(ctx)
if !apply {
return report, nil
}
if err := target.VerifyResidueCleared(ctx, source); err != nil {
return report, err
}
if err := target.Verify(ctx, libraryrelease.Expectation{SnapshotID: backupSnapshotID}); err != nil {
return report, fmt.Errorf("清理后目标内容仍与预备份不一致(保持停写): %w", err)
}
report.Verified = true
if len(report.Foreign) > 0 {
raw, _ := json.Marshal(report.Foreign)
return report, fmt.Errorf("清理后目标仍有其它租户的数据,需人工确认才能开放:%s", raw)
}
return report, nil
}
+6
View File
@@ -70,6 +70,12 @@ services:
PRODUCTION_WEKNORA_BASE_URL: "${PRODUCTION_WEKNORA_BASE_URL:-}"
PRODUCTION_WEKNORA_API_KEY_FILE: "${PRODUCTION_WEKNORA_API_KEY_FILE:-}"
PRODUCTION_WEKNORA_ALLOW_DOCKER_HTTP: "${PRODUCTION_WEKNORA_ALLOW_DOCKER_HTTP:-false}"
# 连接(地址 + 写-only 密钥)密文存治理库:主密钥是独立的 0600 文件。
# 生产 18093 是同网段 http 反代,因此需要显式放行私网 http(只放行
# RFC1918/回环/链路本地,公网 http 仍被拒绝)。
WEKNORA_BINDING_MASTER_KEY_FILE: "${WEKNORA_BINDING_MASTER_KEY_FILE:-}"
WEKNORA_ALLOW_PRIVATE_HTTP: "${WEKNORA_ALLOW_PRIVATE_HTTP:-false}"
WEKNORA_BINDING_TRUST_TLS_PROXY: "${WEKNORA_BINDING_TRUST_TLS_PROXY:-false}"
# 数据源发现(系统设置 → 数据源配置):api 用 dws 只读列举钉钉知识库与钉盘空间。
DWS_BIN: /usr/local/bin/dws
DWS_CONFIG_DIR: /data/dws
+83 -3
View File
@@ -3,7 +3,86 @@
按"实际干活顺序"排列的**现在的**操作流程。标注:**【新】**本期新增(含本次会话与 `f4e1ecc/6496ee3/9155bd3` 的 dws 取件)|**【变】**既有流程本期有改动|**【既有】**未变,仍照旧走。
逐条验收证据请看 `docs/USER_FLOWS_ACCEPTANCE.md`;细节规则看各自专题文档。
统一入口 **http://127.0.0.1:8088/**(测试栈,右侧面板可直接操作)。角色:管理员 / 文档工程师 / 只读。
管理员配置连接和发布授权,文档工程师维护资料,只读成员查看进度。先确定要操作的环境,再按下面的接线与验收步骤进入业务流程。
## 入口与反向代理
| 请求从哪里发出 | 地址或路径 | 实际目的地 |
|---|---|---|
| 已接入内网的运维电脑 | `http://100.85.111.121:8088/` | 云测试治理主站 |
| 开发电脑的浏览器 | `http://127.0.0.1:28189/` | SSH 转发到测试机回环 `28189`,由候选 Web 提供静态资源 |
| 测试候选 Nginx(host 网络) | `/api/`、`/health` | 测试机 `127.0.0.1:18099` 候选 API |
| 同一候选 Nginx | `/api/v1/qa-ops/ai/` | 测试机 `127.0.0.1:18098` 现用治理 API,负责模型配置和连接验证 |
| 标准 Compose Web(容器网络) | `/api/`、`/health` | Docker DNS 解析 `api:8080`。静态资源和 SPA 回退由同一 Web 提供 |
**在开发电脑恢复验收入口。** 前提是 SSH 别名 `weknora-test` 能登录测试机。先查远端服务,再查本机监听。本机端口空闲且远端正常时,在独立终端运行第三条命令,保留该终端。在另一终端执行最后一条并用浏览器打开同一地址。
```bash
ssh weknora-test 'curl -fsS --max-time 5 http://127.0.0.1:28189/health'
ss -ltn 'sport = :28189'
ssh -NT -o ExitOnForwardFailure=yes -o ServerAliveInterval=30 -o ServerAliveCountMax=3 -L 127.0.0.1:28189:127.0.0.1:28189 weknora-test
curl -fsS --max-time 5 http://127.0.0.1:28189/health
```
**为内网域名配置 HTTPS。** 在能访问测试主站的受信任反代机上配置证书、内部 DNS 和访问范围。以下片段放入已有 HTTPS `server`,完整保留 URI、Host 和会话 Cookie。上游应选治理 Web 入口,使 `/assets/`、`/api/`、`/health` 走同一域名。若该入口的内层代理也设置 `X-Forwarded-Proto`,联调时检查其保留受信任 TLS 终止层的协议值以及登录回调的实际 URL。
```nginx
location / {
proxy_pass http://100.85.111.121:8088;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
```
在反代机执行 `nginx -t` 校验待应用配置。入口启用后,通过该域名验证登录、一次 `/api/` 请求和首页实际引用的 JS/CSS。`index.html` 应重新校验,带哈希的资源可长期缓存,缺失的 JS 返回 404。候选切换前后分别记录通用上游和 AI 专用上游。现场可在测试机用以下只读命令查看,容器名以当时部署为准。仓内同源配置见 [deploy/nginx.conf](../deploy/nginx.conf)。
```bash
docker exec kg-test-storage-ui-6444814 nginx -T 2>&1 | grep -E 'listen .*28189|location /api/v1/qa-ops/ai/|set \$(api|health)_upstream|proxy_pass http://127[.]0[.]0[.]1:18098'
docker inspect -f '{{.Name}} {{index .Config.Labels "release"}}' kg-test-storage-ui-6444814 kg-accept-090-api
```
**现场版本与回退。** 本次在测试机读到 Web 标签 `4b615abe534b`、API 标签 `341c3154ac91`。完整候选包的 `current` 链接位于 `/opt/knowledge-governance/acceptance/090/current`,单独切换的 Web 位于同级 `web-only/4b615abe534b/dist`,实际反代配置位于 `integration/28189.conf`。切换前记录这三处、容器标签与旧 Web 名称。静态页回退时恢复上次的 Web 资源和对应代理配置。完整候选回退按同批装配工具的 `deploy/accept-090/README.md` 执行。回退后用浏览器重新验证首页资源、模型连接和列库。数据库迁移另按备份与恢复方案处理。
本次已核实共享测试治理库到 `000082`。对内、对外独立启用的实现依赖 `000083`,待独立库联调或共享库迁移获准后,再验证单侧暂停、保存和刷新。生产 `070` 不在该套验收范围。
下文系统设置中的绑定、盘点和模型连接步骤适用于上表的测试候选版本。根据现场标签选择对应版本的代码和交付物。
## 接入外部平台
| 平台与用途 | 管理员或运维怎样接 | 接通的证据 |
|---|---|---|
| 钉钉知识库,取目录和文件 | 系统设置 → 数据源配置 → 钉钉授权。用 DWS 设备授权,以来源账号选择组织、空间和范围。API 与 worker 使用部署约定的 DWS 身份目录 | 登录后读回预期空间,选一个有权限的目录校验。登记来源、扫描完成和文件取件分别核验。 |
| AList,取工业或教育资料 | 按来源登记服务根地址和允许的目录。普通 worker 读取专用令牌文件。目录接口 `POST /api/fs/list` 使用 `X-Gateway-Token` | 在来源页扫描已知根目录,对照一个子目录和一份文件。下载遵循来源签名链接,凭据和签名 URL 仅留在受控运行环境。[目录与取件](SOURCE_CATALOG_LIVE_ROUTING_ACCEPTANCE.md) |
| PP 资料门户 | 在数据源配置填写 HTTPS 根地址和来源账号,点「测试连接」后「保存并校验」。API 和 worker 共享 `PP_AUTH_STATE_FILE` 所在目录 | 真实登录并读取一条目录记录,刷新后核对生效配置。超时或限流时查看待验证状态。 |
| WeKnora 测试 090,处理与验收 | 系统设置 → 知识库绑定 → 测试 090。当前测试 API 地址 `http://127.0.0.1:8092`,由治理服务器访问。输入该实例的专用 Key 或使用已配置凭据 | 「验证并读取知识库」能列出预期库。核对名称和 ID,为对内、对外选择不同的目标,保存后刷新读回 |
| WeKnora 生产 090,列库与映射 | 在同页切到生产 090,使用下节的转发入口与生产专用 Key。新连接的安全存储由 `WEKNORA_BINDING_MASTER_KEY_FILE` 提供 | 读取生产目录并核对两个目标 ID,保存后重新进入生产页核实。执行发布另按 §5–6 授权 |
| 问答 AI 模型 | 系统设置 → AI 模型连接,填写 OpenAI 兼容基础地址(含服务要求的 `/v1`)、模型名及专用 Key。API 与普通 worker 使用同一 `QA_AI_MASTER_KEY_FILE` | 点「测试连接」等待成功,需生效时再保存并刷新检查模型名。存储密钥、权限与 DNS 配置见 [README](../README.md#问答运营模型配置) |
**生产 090 的两段地址。** 生产机私网 IP 为 `172.29.151.11`,其 WeKnora API 对宿主回环发布 `127.0.0.1:18092`。测试治理机上已有 SSH 本地转发,治理 API 使用 host 网络,故页面的生产服务地址填写 **`http://127.0.0.1:18092`**。请求从治理 API 发出,经测试机的回环监听到生产机的回环 API。浏览器所在电脑的 `127.0.0.1:28189` 则用于打开治理页面。迁到 bridge 网络时要从新 API 容器重新验证上游可达性。
在**测试治理机**上检验已有通道。`/health` 返回 200 证明转发可用,无凭据列库返回 401 表明请求已到认证层。接着在治理页面安全输入该实例的 Key 并验证真实库列表。本次核验时,生产通道探活成功,匿名列库返回 401。带生产凭据列库和保存映射仍待验收。
```bash
ss -ltnp 'sport = :18092'
curl --noproxy '*' --max-time 5 -sS -o /dev/null -w 'health %{http_code}\n' http://127.0.0.1:18092/health
curl --noproxy '*' --max-time 5 -sS -o /dev/null -w 'catalog %{http_code}\n' http://127.0.0.1:18092/api/v1/knowledge-bases
```
## 开发测试和现场验收
1. 在开发机运行 `make list` 查看门禁,`make verify` 做整套检查。以 `ci-report.json` 的 `commit_sha` 对照 `git rev-parse HEAD`,记录失败或跳过项。前端启动和组件测试见 [开发与 CI](../README.md#开发与-ci)。
2. 针对后端改动,在独立测试逻辑库核对 `SELECT current_database(), max(version) FROM schema_migrations;` 与候选版本的迁移目录。云端多个候选可能共用治理 RDS。只改静态页面时单独切换 Web,保留原资源和代理配置供回滚。
3. 在指定入口以真实账号分别验证。来源页读取一条已知记录,模型页测试到成功终态,知识库绑定在正确环境列库,保存后刷新核对。存储页按最近批次和子库目录分别与 `du -sB1 -- <目录>` 对账。记录入口、Web/API 提交、时间和结果。
4. 候选备份页读取宿主采集的 `metadata.json`,API 只读挂载输出目录。「重新盘点」只重读文件。超过 `BACKUP_HOST_METADATA_MAX_AGE` 时,目录、占盘和预测转为未知。宿主重新采集后核对采集时间是否前进、批次是否恢复。测试现场使用按次采集、15 分钟有效期。采集脚本和挂载参数以同批测试候选交付物为准。
| 症状 | 第一步怎么查 | 如何判定 |
|---|---|---|
| 浏览器连接被拒 | 对照本机 `28189` 监听和测试机 `/health` | 远端正常、本机无监听时恢复本机 SSH 转发 |
| 页面打开,某个 API 404 或 502 | 查浏览器请求路径、Nginx 生效上游及对应 API 版本 | 404 先查路由是否存在,502 先查上游是否可达。AI 页单独核对 `18098` |
| 能探活,列库仍灰或返回 401 | 核对选中环境、凭据配置状态、容器运行 UID 与凭据文件可读性 | 用该环境的专用身份实际列库。检查过程只记录状态码和可读性 |
| 历史可读,计划或配额显示未知 | 核对快照年龄、执行主机连接与配额字段 | 盘点、SSH 计划、配额分别验证。只有主机实际读回 cron 才能报告下一次执行时间 |
---
@@ -81,8 +160,9 @@ bash scripts/target_kb_snapshot.sh # 默认云内执行;PG 8 表 +
| 分区 | 谁 | 现在要管什么 |
|---|---|---|
| 服务配置 | 管理员 | 测试地址 / 生产地址 / 生产 ID / 知识库 ID(保存时真探活,无效才拒) |
| 测试知识库绑定 | 管理员 | 标注往哪个测试库路由(对内 ≠ 对外) |
| 生产知识库绑定 | 管理员 | 源库 → 目标库映射(发布写哪个生产库) |
| 测试知识库绑定 | 管理员 | 顶部连接卡:测试地址 + API Key(测试连接 → 保存;只回显尾4位);下方标注往哪个测试库路由(对内 ≠ 对外) |
| 生产连接 | 管理员 | 生产地址填同网段反代 `http://172.29.151.11:18093`(需要 `WEKNORA_ALLOW_PRIVATE_HTTP=true`),密钥录入后只回显尾4位;停止 `ssh -L 18092` 隧道后仍可用 |
| 生产知识库绑定 | 管理员 | 源库 → 目标库映射(发布写哪个生产库);选项来自生产实例真实库,禁止手填 ID;保存后回报「已冻结 / 仍可改指向」条数 |
| **存储与备份** **【新】** | 管理员 | **定时时间 + 储存天数**(+每月归档、启用开关)→ `保存并下发到主机`;另有 `重新下发当前设置`、`立即备份一次`;页面显示主机上**实际生效**的计划、cron 条目、最近一次结果与"已一致/待下发" |
| 刷新策略 | 文档工程师 | 文档/资源刷新间隔、停滞判定、finalizing 时长(版本化 + 原子切换) |
| 成员与角色 | 管理员 | 成员与 viewer/engineer/admin |
+6
View File
@@ -260,6 +260,12 @@
- 规则:只配四项(测试地址、生产地址、生产 ID、知识库 ID);**保存时真探测**,无效才拒;凭据只写不读
- 状态:✅(见 `P41/P42` 与 `ui_settings_*` 测试)
### D1b 连接(地址 + API Key,测试 / 生产各一张)
- 入口:**系统设置 → 测试知识库绑定 / 生产知识库绑定** 分区顶部的连接卡(管理员)
- 步骤:填地址 → 粘贴 API Key(密码框)→ **测试连接** → 成功后才可 **保存连接**;换地址或换密钥必须重新测通
- 期望:密钥密文入库、页面只显示「已配置 · 尾4位」;四态文案分别是 地址不可达 / 密钥无效 / 密钥无权限读取知识库列表 / 可达(读到 N 个知识库);保存不触发任何同步或发布
- 状态:✅(`internal/weknoraconn` 单测 + `internal/httpapi/weknora_connections_test.go`;迁移 000083)
### D2 测试知识库绑定(对内 / 对外)
- 入口:**系统设置 → 测试知识库绑定** → 选库 → 保存绑定
- 期望:保存立即生效并重排"还没上传"的待入库项;对内 ≠ 对外
+43 -22
View File
@@ -31,28 +31,33 @@ type Config struct {
ProductionWeKnoraBaseURL string
ProductionWeKnoraAPIKeyFile string
ProductionWeKnoraAllowDockerHTTP bool
PPScanEnabled bool
PPScanAllowedRoot string
PPScanLedgerPath string
PPScanManifestPath string
TestWeKnoraEnabled bool
TestWeKnoraAllowDockerHTTP bool
TestWeKnoraBaseURL string
MinerUBaseURL string
TestWeKnoraKBID string
InternalRoutingKnowledgeBaseID string
ExternalRoutingKnowledgeBaseID string
TestWeKnoraAPIKeyFile string
GovernanceSpoolRoot string
IngestionSpoolRoot string
ArtifactFetchMaxBytes int64
WorkerLivenessStaleAfter time.Duration
TestWeKnoraHTTPTimeout time.Duration
TestWeKnoraPollInitial time.Duration
TestWeKnoraPollMax time.Duration
TestWeKnoraStalledAfter time.Duration
TestWeKnoraMaxUploadBytes int64
TestAcceptancePolicyVersion string
// 连接(地址 + 写-only 密钥)密文存治理库;主密钥是独立的 0600 文件。
WeKnoraBindingMasterKeyFile string
// 允许同网段 http(生产 18093 反代)与可信 TLS 反代后录入密钥。
WeKnoraAllowPrivateHTTP bool
WeKnoraBindingTrustTLSProxy bool
PPScanEnabled bool
PPScanAllowedRoot string
PPScanLedgerPath string
PPScanManifestPath string
TestWeKnoraEnabled bool
TestWeKnoraAllowDockerHTTP bool
TestWeKnoraBaseURL string
MinerUBaseURL string
TestWeKnoraKBID string
InternalRoutingKnowledgeBaseID string
ExternalRoutingKnowledgeBaseID string
TestWeKnoraAPIKeyFile string
GovernanceSpoolRoot string
IngestionSpoolRoot string
ArtifactFetchMaxBytes int64
WorkerLivenessStaleAfter time.Duration
TestWeKnoraHTTPTimeout time.Duration
TestWeKnoraPollInitial time.Duration
TestWeKnoraPollMax time.Duration
TestWeKnoraStalledAfter time.Duration
TestWeKnoraMaxUploadBytes int64
TestAcceptancePolicyVersion string
// TestAutoReviewPolicy:验收通过后是否自动批准(off|internal|all;默认 internal)。
TestAutoReviewPolicy string
// 目标库所在主机(整库逻辑备份真正执行的地方)。凭据是挂载进来的 0600 ssh
@@ -168,6 +173,7 @@ func Load(getenv func(string) string) (Config, error) {
ProductionReleaseAuthTokenFile: strings.TrimSpace(getenv("PRODUCTION_RELEASE_AUTH_TOKEN_FILE")),
ProductionWeKnoraBaseURL: strings.TrimSpace(getenv("PRODUCTION_WEKNORA_BASE_URL")),
ProductionWeKnoraAPIKeyFile: strings.TrimSpace(getenv("PRODUCTION_WEKNORA_API_KEY_FILE")),
WeKnoraBindingMasterKeyFile: strings.TrimSpace(getenv("WEKNORA_BINDING_MASTER_KEY_FILE")),
PPScanAllowedRoot: strings.TrimSpace(getenv("PP_SCAN_ALLOWED_ROOT")),
PPScanLedgerPath: strings.TrimSpace(getenv("PP_SCAN_LEDGER_PATH")),
PPScanManifestPath: strings.TrimSpace(getenv("PP_SCAN_MANIFEST_PATH")),
@@ -288,6 +294,21 @@ func Load(getenv func(string) string) (Config, error) {
return Config{}, fmt.Errorf("PRODUCTION_WEKNORA_BASE_URL: %w", err)
}
}
if raw := getenv("WEKNORA_ALLOW_PRIVATE_HTTP"); raw != "" {
config.WeKnoraAllowPrivateHTTP, err = strconv.ParseBool(raw)
if err != nil {
return Config{}, fmt.Errorf("WEKNORA_ALLOW_PRIVATE_HTTP: %w", err)
}
}
if raw := getenv("WEKNORA_BINDING_TRUST_TLS_PROXY"); raw != "" {
config.WeKnoraBindingTrustTLSProxy, err = strconv.ParseBool(raw)
if err != nil {
return Config{}, fmt.Errorf("WEKNORA_BINDING_TRUST_TLS_PROXY: %w", err)
}
}
if config.WeKnoraBindingMasterKeyFile != "" && !filepath.IsAbs(config.WeKnoraBindingMasterKeyFile) {
return Config{}, errors.New("WEKNORA_BINDING_MASTER_KEY_FILE must be an absolute path")
}
if config.ProductionWeKnoraAPIKeyFile != "" {
if config.ProductionWeKnoraBaseURL == "" {
return Config{}, errors.New("PRODUCTION_WEKNORA_API_KEY_FILE requires PRODUCTION_WEKNORA_BASE_URL")
+6
View File
@@ -138,6 +138,12 @@ func classifyRole(method, path string) string {
if path == "/api/v1/routing-bindings" && method != http.MethodGet {
return roleAdmin
}
// Recording an endpoint or an API key changes which instance the whole
// platform talks to, and the key is a credential: read is viewer, every
// write (including 测试连接, which spends the key) is admin.
if strings.HasPrefix(path, "/api/v1/weknora-connections") && method != http.MethodGet {
return roleAdmin
}
// Choosing the production base a release writes into is a production
// authority change, exactly like the annotation binding above.
if path == "/api/v1/production-kb-bindings" && method != http.MethodGet {
+10 -2
View File
@@ -411,8 +411,16 @@ func TestClassifyRoleMatrix(t *testing.T) {
http.MethodPost + " /api/v1/users": roleAdmin,
http.MethodPost + " /api/v1/source-connections/c-1/settings/revisions": roleAdmin,
// Choosing a release's production base is a production authority change.
http.MethodPut + " /api/v1/production-kb-bindings": roleAdmin,
http.MethodGet + " /api/v1/production-kb-bindings": roleViewer,
http.MethodPut + " /api/v1/production-kb-bindings": roleAdmin,
http.MethodGet + " /api/v1/production-kb-bindings": roleViewer,
// An endpoint or an API key decides which instance the platform talks
// to; 测试连接 also spends the credential, so every write is admin.
http.MethodGet + " /api/v1/weknora-connections": roleViewer,
http.MethodPost + " /api/v1/weknora-connections/test/test": roleAdmin,
http.MethodPut + " /api/v1/weknora-connections/production": roleAdmin,
// The test-side annotation binding is the same authority as production.
http.MethodGet + " /api/v1/routing-bindings": roleViewer,
http.MethodPut + " /api/v1/routing-bindings": roleAdmin,
http.MethodPost + " /api/v1/review-candidates/tp-1/decision": roleEngineer,
http.MethodPost + " /api/v1/release-batches/rb-1/seal": roleEngineer,
http.MethodPost + " /api/v1/test-publications/start": roleEngineer,
+26 -2
View File
@@ -16,6 +16,10 @@ import (
type ProductionBindingStore interface {
LoadProductionKBBindings(context.Context) (string, string, string, error)
SaveProductionKBBindings(ctx context.Context, internalID, externalID, updatedBy string, now time.Time) error
// ProductionExecutionCounts reports frozen vs still re-targetable release
// items. A binding change never rewrites a target that was already frozen
// when an execution started; the count is how the page says so.
ProductionExecutionCounts(ctx context.Context) (frozen int, retargetable int, err error)
}
// ProductionKnowledgeBaseLister enumerates the knowledge bases of the production
@@ -33,11 +37,26 @@ type productionBindingView struct {
ProductionBaseURL string `json:"production_base_url"`
ProductionConnected bool `json:"production_connected"`
ProductionMessage string `json:"production_message"`
// Frozen / Retargetable describe what a binding change can still affect.
Frozen int `json:"frozen"`
Retargetable int `json:"retargetable"`
}
// WeKnoraConnectionResolver reads the admin-managed connection for one
// environment. When it answers, that endpoint is the only truth.
type WeKnoraConnectionResolver interface {
Resolve(ctx context.Context, environment string) (endpoint string, apiKey string, ok bool)
}
// productionTargetBaseURL is the production endpoint the picker reads from: the
// operator-managed 「服务配置 · 生产地址」 first, the deployment default second.
// connection card first, the operator-managed 「服务配置 · 生产地址」 second, the
// deployment default last.
func (config Config) productionTargetBaseURL(ctx context.Context) string {
if config.WeKnoraConnectionResolver != nil {
if endpoint, _, ok := config.WeKnoraConnectionResolver.Resolve(ctx, "production"); ok {
return endpoint
}
}
if settings, err := config.Store.GetActiveUISettings(ctx); err == nil {
if resolved := strings.TrimSpace(settings.ProductionWeKnoraBaseURL); resolved != "" {
return resolved
@@ -55,7 +74,7 @@ func (config Config) productionKnowledgeBases(ctx context.Context) ([]weknora.Kn
}
baseURL := config.productionTargetBaseURL(ctx)
if baseURL == "" {
return nil, "未配置生产地址(系统设置 → 服务配置)"
return nil, "未配置生产地址(系统设置 → 生产知识库绑定 → 连接卡)"
}
bases, err := config.ProductionKnowledgeBases.ListProductionKnowledgeBases(ctx, baseURL)
if err != nil {
@@ -74,11 +93,16 @@ func (config Config) productionBindingPayload(ctx context.Context) (productionBi
for _, base := range bases {
names[base.ID] = base.Name
}
frozen, retargetable := 0, 0
if frozenItems, retargetableItems, countsErr := config.ProductionBindings.ProductionExecutionCounts(ctx); countsErr == nil {
frozen, retargetable = frozenItems, retargetableItems
}
return productionBindingView{
InternalKBID: storedInternal, ExternalKBID: storedExternal,
InternalKBName: names[storedInternal], ExternalKBName: names[storedExternal],
UpdatedBy: updatedBy, ProductionBaseURL: config.productionTargetBaseURL(ctx),
ProductionConnected: message == "", ProductionMessage: message,
Frozen: frozen, Retargetable: retargetable,
}, nil
}
@@ -18,6 +18,7 @@ type productionBindingStoreFake struct {
internal, external, saver string
savedInternal, savedExternal string
saveCallCount int
frozen, retargetable int
}
func (fake *productionBindingStoreFake) LoadProductionKBBindings(context.Context) (string, string, string, error) {
@@ -32,6 +33,10 @@ func (fake *productionBindingStoreFake) SaveProductionKBBindings(_ context.Conte
return nil
}
func (fake *productionBindingStoreFake) ProductionExecutionCounts(context.Context) (int, int, error) {
return fake.frozen, fake.retargetable, nil
}
type productionListerFake struct {
bases []weknora.KnowledgeBase
err error
+13 -9
View File
@@ -31,12 +31,15 @@ type routingBindingView struct {
ExternalKBID string `json:"external_kb_id"`
InternalEnvFallback bool `json:"internal_env_fallback"`
ExternalEnvFallback bool `json:"external_env_fallback"`
InternalBoundByUI bool `json:"internal_bound_by_ui"`
ExternalBoundByUI bool `json:"external_bound_by_ui"`
InternalKBName string `json:"internal_kb_name"`
ExternalKBName string `json:"external_kb_name"`
UpdatedBy string `json:"updated_by"`
WeKnoraConnected bool `json:"weknora_connected"`
// 取消绑定时必须说清「回退到了哪个库」,否则「留空」等于没说。
InternalFallbackKBID string `json:"internal_fallback_kb_id"`
ExternalFallbackKBID string `json:"external_fallback_kb_id"`
InternalBoundByUI bool `json:"internal_bound_by_ui"`
ExternalBoundByUI bool `json:"external_bound_by_ui"`
InternalKBName string `json:"internal_kb_name"`
ExternalKBName string `json:"external_kb_name"`
UpdatedBy string `json:"updated_by"`
WeKnoraConnected bool `json:"weknora_connected"`
// Replanned / RetiredDuplicates describe what THIS save did to 待入库 items
// that had not been uploaded yet. A plain read reports zeros.
Replanned int `json:"replanned"`
@@ -98,9 +101,10 @@ func (config Config) routingBindingPayload(ctx context.Context, replan postgress
return routingBindingView{
InternalKBID: internal, ExternalKBID: external,
InternalBoundByUI: storedInternal != "", ExternalBoundByUI: storedExternal != "",
InternalEnvFallback: storedInternal == "" && config.InternalRoutingKnowledgeBaseID != "",
ExternalEnvFallback: storedExternal == "" && config.ExternalRoutingKnowledgeBaseID != "",
InternalKBName: names[storedInternal], ExternalKBName: names[storedExternal],
InternalEnvFallback: storedInternal == "" && config.InternalRoutingKnowledgeBaseID != "",
ExternalEnvFallback: storedExternal == "" && config.ExternalRoutingKnowledgeBaseID != "",
InternalFallbackKBID: config.InternalRoutingKnowledgeBaseID, ExternalFallbackKBID: config.ExternalRoutingKnowledgeBaseID,
InternalKBName: names[storedInternal], ExternalKBName: names[storedExternal],
UpdatedBy: updatedBy, WeKnoraConnected: connected,
Replanned: replan.Replanned, RetiredDuplicates: replan.RetiredDuplicates,
}, nil
+27 -8
View File
@@ -177,14 +177,24 @@ type Config struct {
// 它不是运行时开关:生产部署不设置它,那次探测仍然直连目标库逐表计数。
LibraryScopeProbe func(ctx context.Context, scope string) (bool, map[string]int64, error)
// LibraryRetentionVersions/Days 是保留策略;都为 0 = 未设置(不生成清理候选)。
LibraryRetentionVersions int
LibraryRetentionDays int
BackupSchedule BackupScheduleStore
BackupHost BackupHost
BackupRuns *BackupRunTracker
ProductionBindings ProductionBindingStore
ProductionKnowledgeBases ProductionKnowledgeBaseLister
ProductionWeKnoraBaseURL string
LibraryRetentionVersions int
LibraryRetentionDays int
BackupSchedule BackupScheduleStore
BackupHost BackupHost
BackupRuns *BackupRunTracker
ProductionBindings ProductionBindingStore
ProductionKnowledgeBases ProductionKnowledgeBaseLister
ProductionWeKnoraBaseURL string
// WeKnoraConnections 是「连接(地址 + 写-only 密钥)」服务;未接线时三条路由
// 返回 503,页面显示「未接线」而不是假装已配置。
WeKnoraConnections WeKnoraConnections
// WeKnoraAllowPrivateHTTP 允许同网段 http(生产 18093 反代);与客户端、监控
// 探活共用同一个判定。WeKnoraTrustTLSProxy 允许在可信 TLS 反代后录入密钥。
WeKnoraAllowPrivateHTTP bool
WeKnoraTrustTLSProxy bool
// WeKnoraConnectionResolver 读页面保存的连接(地址 + 密钥);页面上显示的
// 生产地址必须来自这里,否则会出现「连接卡写 18093、绑定页还在用旧地址」。
WeKnoraConnectionResolver WeKnoraConnectionResolver
ServiceTargetVerifier ServiceTargetVerifier
Now func() time.Time
RequireWeKnoraReady bool
@@ -421,6 +431,15 @@ func NewHandler(config Config) (http.Handler, error) {
}
config.handleSaveProductionBindings(w, r)
})
mux.HandleFunc("GET /api/v1/weknora-connections", func(w http.ResponseWriter, r *http.Request) {
config.handleWeKnoraConnections(w, r)
})
mux.HandleFunc("POST /api/v1/weknora-connections/{environment}/test", func(w http.ResponseWriter, r *http.Request) {
config.handleWeKnoraConnectionTest(w, r)
})
mux.HandleFunc("PUT /api/v1/weknora-connections/{environment}", func(w http.ResponseWriter, r *http.Request) {
config.handleSaveWeKnoraConnection(w, r)
})
mux.HandleFunc("POST /api/v1/source-catalog/route", func(w http.ResponseWriter, r *http.Request) {
routeCatalogFiles(w, r, config)
})
+153
View File
@@ -0,0 +1,153 @@
package httpapi
import (
"context"
"errors"
"net"
"net/http"
"net/url"
"strings"
"knowledge-governance/internal/weknoraconn"
)
// WeKnoraConnections is the connection service the routes below call. It is
// deliberately narrow: the page may read state, run one real test, and save a
// tested configuration — nothing here triggers a sync, a release or a reparse.
type WeKnoraConnections interface {
View(ctx context.Context, environment string) ([]weknoraconn.View, error)
Test(ctx context.Context, environment string, input weknoraconn.TestInput) (weknoraconn.TestResult, error)
Save(ctx context.Context, environment string, input weknoraconn.SaveInput, actor string) (weknoraconn.View, error)
}
func weKnoraConnectionError(w http.ResponseWriter, err error) {
switch {
case errors.Is(err, weknoraconn.ErrNotFound):
writeJSON(w, http.StatusNotFound, map[string]string{"error": "未知的 WeKnora 环境(只支持 test / production)"})
case errors.Is(err, weknoraconn.ErrStaleRevision):
writeJSON(w, http.StatusConflict, map[string]string{"error": "连接配置已被他人修改,请重新读取后再保存"})
case errors.Is(err, weknoraconn.ErrNotTested):
writeJSON(w, http.StatusBadRequest, map[string]string{
"code": "weknora_connection_not_tested",
"error": "请先点「测试连接」并成功,才能保存这一侧的配置",
})
case errors.Is(err, weknoraconn.ErrStorageUnavailable):
writeJSON(w, http.StatusServiceUnavailable, map[string]string{
"error": "密钥存储不可用:主密钥文件缺失或权限不是 0600,本次未保存任何密钥",
})
case errors.Is(err, weknoraconn.ErrInvalid):
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "连接配置不合法:地址必填,且更换地址必须重新录入密钥"})
default:
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "WeKnora 连接配置处理失败"})
}
}
// weKnoraSecretTransportWithPrivateHTTP decides whether a plaintext key may be
// accepted on this request. It reuses the QA-AI rule (TLS, loopback, trusted
// TLS proxy + same origin) and adds the one case this deployment needs: the
// operator console itself is reached over plain http on a private address, so a
// same-origin fetch from that console is the only accepted non-TLS shape.
func weKnoraSecretTransportWithPrivateHTTP(config Config, r *http.Request) bool {
if qaAISecretTransport(r, config) {
return true
}
if config.WeKnoraTrustTLSProxy && r.Header.Get("X-Forwarded-Proto") == "https" &&
r.Header.Get("Sec-Fetch-Site") == "same-origin" {
origin, err := url.Parse(r.Header.Get("Origin"))
if err == nil && origin.Scheme == "https" {
return true
}
}
if !config.WeKnoraAllowPrivateHTTP {
return false
}
if r.Header.Get("Sec-Fetch-Site") != "same-origin" {
return false
}
origin := strings.TrimSpace(r.Header.Get("Origin"))
if origin == "" {
return false
}
parsed, err := url.Parse(origin)
if err != nil || parsed.Host == "" {
return false
}
if !strings.EqualFold(parsed.Host, r.Host) {
return false
}
host := parsed.Hostname()
if ip := net.ParseIP(host); ip != nil {
return ip.IsPrivate() || ip.IsLoopback() || ip.IsLinkLocalUnicast()
}
return strings.EqualFold(host, "localhost")
}
func (config Config) handleWeKnoraConnections(w http.ResponseWriter, r *http.Request) {
if config.WeKnoraConnections == nil {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "WeKnora 连接配置未接线"})
return
}
w.Header().Set("Cache-Control", "no-store")
views, err := config.WeKnoraConnections.View(r.Context(), r.PathValue("environment"))
if err != nil {
weKnoraConnectionError(w, err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"data": views})
}
func (config Config) handleWeKnoraConnectionTest(w http.ResponseWriter, r *http.Request) {
if config.WeKnoraConnections == nil {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "WeKnora 连接配置未接线"})
return
}
w.Header().Set("Cache-Control", "no-store")
var input weknoraconn.TestInput
if err := decodeStrictJSON(r, &input); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "请求体不合法"})
return
}
if strings.TrimSpace(input.APIKey) != "" && !weKnoraSecretTransportWithPrivateHTTP(config, r) {
writeJSON(w, http.StatusBadRequest, map[string]string{
"code": "weknora_secret_secure_transport_required",
"error": "密钥仅可在有效 HTTPS、本机回环地址,或已放行的私网治理台同源页面录入;本次未保存、未发送密钥",
})
return
}
result, err := config.WeKnoraConnections.Test(r.Context(), r.PathValue("environment"), input)
if err != nil {
weKnoraConnectionError(w, err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"data": result})
}
func (config Config) handleSaveWeKnoraConnection(w http.ResponseWriter, r *http.Request) {
if config.WeKnoraConnections == nil {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "WeKnora 连接配置未接线"})
return
}
w.Header().Set("Cache-Control", "no-store")
if _, ok := requireOperationID(w, r); !ok {
return
}
var input weknoraconn.SaveInput
if err := decodeStrictJSON(r, &input); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "请求体不合法"})
return
}
if input.KeyAction != weknoraconn.KeyActionClear && strings.TrimSpace(input.APIKey) != "" &&
!weKnoraSecretTransportWithPrivateHTTP(config, r) {
writeJSON(w, http.StatusBadRequest, map[string]string{
"code": "weknora_secret_secure_transport_required",
"error": "密钥仅可在有效 HTTPS、本机回环地址,或已放行的私网治理台同源页面录入;本次未保存、未发送密钥",
})
return
}
view, err := config.WeKnoraConnections.Save(r.Context(), r.PathValue("environment"), input, actorFor(r))
if err != nil {
weKnoraConnectionError(w, err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"data": view})
}
@@ -0,0 +1,174 @@
package httpapi
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"knowledge-governance/internal/weknoraconn"
)
type weknoraConnectionsFake struct {
views []weknoraconn.View
testResult weknoraconn.TestResult
testErr error
savedResult weknoraconn.View
saveErr error
lastTest weknoraconn.TestInput
lastSave weknoraconn.SaveInput
lastActor string
}
func (fake *weknoraConnectionsFake) View(_ context.Context, environment string) ([]weknoraconn.View, error) {
if environment == "" {
return fake.views, nil
}
if !weknoraconn.ValidEnvironment(environment) {
return nil, weknoraconn.ErrNotFound
}
for _, view := range fake.views {
if view.Environment == environment {
return []weknoraconn.View{view}, nil
}
}
return []weknoraconn.View{{Environment: environment, Verification: weknoraconn.VerificationNotConfigured}}, nil
}
func (fake *weknoraConnectionsFake) Test(_ context.Context, environment string, input weknoraconn.TestInput) (weknoraconn.TestResult, error) {
if !weknoraconn.ValidEnvironment(environment) {
return weknoraconn.TestResult{}, weknoraconn.ErrNotFound
}
fake.lastTest = input
return fake.testResult, fake.testErr
}
func (fake *weknoraConnectionsFake) Save(_ context.Context, environment string, input weknoraconn.SaveInput, actor string) (weknoraconn.View, error) {
if !weknoraconn.ValidEnvironment(environment) {
return weknoraconn.View{}, weknoraconn.ErrNotFound
}
fake.lastSave, fake.lastActor = input, actor
return fake.savedResult, fake.saveErr
}
func weknoraConnectionsHandler(t *testing.T, fake *weknoraConnectionsFake, allowPrivate bool) http.Handler {
t.Helper()
handler, err := NewHandler(Config{
Store: &fakeJobStore{},
WeKnoraConnections: fake,
WeKnoraAllowPrivateHTTP: allowPrivate,
Now: func() time.Time { return time.Date(2026, time.November, 22, 7, 0, 0, 0, time.UTC) },
})
if err != nil {
t.Fatalf("NewHandler: %v", err)
}
return handler
}
func TestWeKnoraConnectionRoutesProjectStateWithoutSecrets(t *testing.T) {
fake := &weknoraConnectionsFake{views: []weknoraconn.View{
{Environment: "test", Endpoint: "http://127.0.0.1:8092", KeyConfigured: true, KeyTail: "9f2c", Verification: weknoraconn.OutcomeOK, LibraryCount: 3, Revision: 2, StorageReady: true},
{Environment: "production", Verification: weknoraconn.VerificationNotConfigured, StorageReady: true},
}}
handler := weknoraConnectionsHandler(t, fake, true)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/v1/weknora-connections", nil))
if rec.Code != http.StatusOK {
t.Fatalf("status = %d %s", rec.Code, rec.Body.String())
}
if rec.Header().Get("Cache-Control") != "no-store" {
t.Fatalf("cache-control = %q", rec.Header().Get("Cache-Control"))
}
body := rec.Body.String()
if !strings.Contains(body, `"key_tail":"9f2c"`) || strings.Contains(body, "key_ciphertext") || strings.Contains(body, "api_key") {
t.Fatalf("body leaks or loses key projection: %s", body)
}
}
func TestWeKnoraConnectionTestKeepsFourAnswers(t *testing.T) {
for _, outcome := range []string{weknoraconn.OutcomeOK, weknoraconn.OutcomeUnreachable, weknoraconn.OutcomeKeyInvalid, weknoraconn.OutcomeKeyForbidden} {
fake := &weknoraConnectionsFake{testResult: weknoraconn.TestResult{
Environment: "production", Endpoint: "http://172.29.151.11:18093", Outcome: outcome, Message: outcome,
}}
handler := weknoraConnectionsHandler(t, fake, true)
rec := httptest.NewRecorder()
request := httptest.NewRequest(http.MethodPost, "/api/v1/weknora-connections/production/test", strings.NewReader(`{"endpoint":"http://172.29.151.11:18093","key_action":"keep"}`))
request.Header.Set("Content-Type", "application/json")
handler.ServeHTTP(rec, request)
if rec.Code != http.StatusOK || !strings.Contains(rec.Body.String(), `"outcome":"`+outcome+`"`) {
t.Fatalf("%s: status = %d body = %s", outcome, rec.Code, rec.Body.String())
}
}
}
func TestWeKnoraConnectionTestRejectsPlainKeyFromAPublicOrigin(t *testing.T) {
fake := &weknoraConnectionsFake{}
handler := weknoraConnectionsHandler(t, fake, true)
rec := httptest.NewRecorder()
request := httptest.NewRequest(http.MethodPost, "/api/v1/weknora-connections/test/test", strings.NewReader(`{"endpoint":"http://127.0.0.1:8092","api_key":"sk-live","key_action":"replace"}`))
request.Header.Set("Content-Type", "application/json")
handler.ServeHTTP(rec, request)
if rec.Code != http.StatusBadRequest || !strings.Contains(rec.Body.String(), "weknora_secret_secure_transport_required") {
t.Fatalf("status = %d body = %s", rec.Code, rec.Body.String())
}
if fake.lastTest.APIKey != "" {
t.Fatal("a rejected key must never reach the service")
}
}
func TestWeKnoraConnectionTestAcceptsSameOriginPrivateConsole(t *testing.T) {
fake := &weknoraConnectionsFake{testResult: weknoraconn.TestResult{Environment: "test", Outcome: weknoraconn.OutcomeOK}}
handler := weknoraConnectionsHandler(t, fake, true)
rec := httptest.NewRecorder()
request := httptest.NewRequest(http.MethodPost, "/api/v1/weknora-connections/test/test", strings.NewReader(`{"endpoint":"http://127.0.0.1:8092","api_key":"sk-live","key_action":"replace"}`))
request.Host = "172.29.151.10:8088"
request.Header.Set("Content-Type", "application/json")
request.Header.Set("Origin", "http://172.29.151.10:8088")
request.Header.Set("Sec-Fetch-Site", "same-origin")
handler.ServeHTTP(rec, request)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d body = %s", rec.Code, rec.Body.String())
}
if fake.lastTest.APIKey != "sk-live" {
t.Fatalf("service saw key %q", fake.lastTest.APIKey)
}
}
func TestSaveWeKnoraConnectionRequiresIdempotencyKeyAndMapsNotTested(t *testing.T) {
fake := &weknoraConnectionsFake{saveErr: weknoraconn.ErrNotTested}
handler := weknoraConnectionsHandler(t, fake, false)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, httptest.NewRequest(http.MethodPut, "/api/v1/weknora-connections/production", strings.NewReader(`{"endpoint":"https://prod.example","key_action":"keep","base_revision":1}`)))
if rec.Code != http.StatusBadRequest || !strings.Contains(rec.Body.String(), "Idempotency-Key") {
t.Fatalf("missing idempotency key: %d %s", rec.Code, rec.Body.String())
}
rec = httptest.NewRecorder()
request := httptest.NewRequest(http.MethodPut, "/api/v1/weknora-connections/production", strings.NewReader(`{"endpoint":"https://prod.example","key_action":"keep","base_revision":1}`))
request.Header.Set("Content-Type", "application/json")
request.Header.Set("Idempotency-Key", "op-1")
handler.ServeHTTP(rec, request)
if rec.Code != http.StatusBadRequest || !strings.Contains(rec.Body.String(), "weknora_connection_not_tested") {
t.Fatalf("not tested mapping: %d %s", rec.Code, rec.Body.String())
}
}
func TestWeKnoraConnectionUnknownEnvironmentIs404(t *testing.T) {
fake := &weknoraConnectionsFake{}
handler := weknoraConnectionsHandler(t, fake, false)
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, httptest.NewRequest(http.MethodGet, "/api/v1/weknora-connections?environment=staging", nil))
// The list route reads the path value, not the query: unknown environments
// only matter on the per-environment routes.
rec = httptest.NewRecorder()
request := httptest.NewRequest(http.MethodPost, "/api/v1/weknora-connections/staging/test", strings.NewReader(`{"endpoint":"https://x.example","key_action":"keep"}`))
request.Header.Set("Content-Type", "application/json")
handler.ServeHTTP(rec, request)
if rec.Code != http.StatusNotFound {
t.Fatalf("status = %d body = %s", rec.Code, rec.Body.String())
}
}
var _ = json.Marshal
+20 -1
View File
@@ -59,6 +59,14 @@ type Expectation struct {
Digests map[string]string
}
// TableFingerprint 是一张知识域表在**整实例**上的行数与身份集合摘要(不限租户/
// 知识库)。恢复后必须与预备份时刻逐表一致:只比目标范围会让"写到范围外"的
// 残留被当成已恢复(真实事故:恢复报成功,目标仍多 67 行)。
type TableFingerprint struct {
Count int64 `json:"count"`
Digest string `json:"digest"`
}
// BackupRef 指向一份可独立恢复的完整预备份。
type BackupRef struct {
ID string `json:"id"`
@@ -67,6 +75,9 @@ type BackupRef struct {
SnapshotID string `json:"snapshot_id"`
Bytes int64 `json:"bytes"`
CreatedAt time.Time `json:"created_at"`
// Universe 是预备份时刻的整实例指纹,恢复后逐表比对。空表示这次没有采集
// (例如 worker 被杀后用已持久化的快照接管),此时恢复只能靠内容校验兜底。
Universe map[string]TableFingerprint `json:"universe,omitempty"`
}
type Target interface {
@@ -224,8 +235,16 @@ func failAndRestore(ctx context.Context, outcome Outcome, target Target, recorde
recorder.Stage(ctx, "rollback", "failed", outcome.Rollback)
return outcome
}
// "恢复工具跑完"不是"恢复完成":必须再校验一次目标内容与预备份一致,
// 否则页面会把半恢复状态报成已恢复(真实事故就是这样放开了维护)。
if verifyErr := target.Verify(ctx, Expectation{SnapshotID: outcome.Backup.SnapshotID}); verifyErr != nil {
outcome.State = "rollback_failed"
outcome.Rollback = "恢复后校验失败(目标可能仍是半写状态):" + verifyErr.Error()
recorder.Stage(ctx, "rollback", "failed", outcome.Rollback)
return outcome
}
outcome.State = "rolled_back"
outcome.Rollback = "已恢复到预备份 " + outcome.Backup.ID
outcome.Rollback = "已恢复到预备份 " + outcome.Backup.ID + ",并逐表/索引/资源复核一致"
recorder.Stage(ctx, "rollback", "succeeded", outcome.Rollback)
return outcome
}
+50 -8
View File
@@ -13,9 +13,14 @@ type fakeTarget struct {
applyErr map[Stage]error
verifyErr error
restoreErr error
applied []Stage
restored int
currentN int
// 恢复之后校验是否通过:用来区分"恢复跑完"和"恢复已验证"。
verifyPassesAfterRestore bool
applied []Stage
restored int
currentN int
verifyN int
// 记录最后一次 Verify 是否发生在 Restore 之后(恢复必须被校验兜底)。
verifiedAfterRestore bool
}
func (f *fakeTarget) Current(context.Context) (CurrentState, error) {
@@ -35,7 +40,17 @@ func (f *fakeTarget) Apply(_ context.Context, stage Stage, _ string) error {
return nil
}
func (f *fakeTarget) Verify(context.Context, Expectation) error { return f.verifyErr }
func (f *fakeTarget) Verify(context.Context, Expectation) error {
f.verifyN++
f.verifiedAfterRestore = f.restored > 0
if f.verifyErr == nil {
return nil
}
if f.verifyPassesAfterRestore && f.restored > 0 {
return nil
}
return f.verifyErr
}
func (f *fakeTarget) Restore(context.Context, BackupRef) error {
f.restored++
@@ -185,12 +200,39 @@ func TestRunRollsBackWhenStageFails(t *testing.T) {
}
}
func TestRunRollsBackWhenVerificationFails(t *testing.T) {
target := &fakeTarget{verifyErr: errors.New("生产分块数不一致")}
outcome := Run(context.Background(), request(time.Now()), target, &fakeBackups{}, nil)
if outcome.State != "rolled_back" || target.restored != 1 {
// 恢复跑完但恢复后校验仍失败:不能报 rolled_back。真实事故里页面就是在这种
// 状态下显示"已恢复到预备份并验证",而目标其实还留着 67 行残留。
func TestRunDoesNotClaimRollbackWhenRecoveryVerificationFails(t *testing.T) {
target := &fakeTarget{applyErr: map[Stage]error{StageResources: errors.New("资源写失败")},
verifyErr: errors.New("生产分块数不一致")}
recorder := &stageLog{}
outcome := Run(context.Background(), request(time.Now()), target, &fakeBackups{}, recorder)
if outcome.State != "rollback_failed" || target.restored != 1 {
t.Fatalf("outcome = %+v restored=%d", outcome, target.restored)
}
if !strings.Contains(outcome.Rollback, "恢复后校验失败") {
t.Fatalf("rollback message = %q", outcome.Rollback)
}
if !recorder.has("rollback:failed") {
t.Fatalf("stage log = %v", recorder.entries)
}
}
// 只有"恢复到预备份且恢复后校验通过"才允许报 rolled_back。
func TestRunReportsRolledBackOnlyAfterRecoveryVerifies(t *testing.T) {
target := &fakeTarget{applyErr: map[Stage]error{StageResources: errors.New("资源写失败")},
verifyErr: errors.New("半写状态"), verifyPassesAfterRestore: true}
recorder := &stageLog{}
outcome := Run(context.Background(), request(time.Now()), target, &fakeBackups{}, recorder)
if outcome.State != "rolled_back" {
t.Fatalf("outcome = %+v", outcome)
}
if !target.verifiedAfterRestore {
t.Fatal("restore must be followed by a content verification")
}
if !recorder.has("rollback:succeeded") {
t.Fatalf("stage log = %v", recorder.entries)
}
}
func TestRunReportsRollbackFailureDistinctly(t *testing.T) {
+6 -1
View File
@@ -13,6 +13,7 @@ import (
"time"
"knowledge-governance/internal/governance"
"knowledge-governance/internal/platform/weknora"
)
// P10: every configured dependency reports Liveness ("process reachable") and
@@ -197,10 +198,14 @@ func (provider *RealProvider) productionWeKnoraProbe(ctx context.Context, now ti
func (provider *RealProvider) httpProbeOutcome(ctx context.Context, baseURL string, now time.Time) ProbeOutcome {
parsed, err := url.Parse(baseURL)
outcome := ProbeOutcome{ObservedAt: now}
if err != nil || parsed.Scheme != "http" && parsed.Scheme != "https" || parsed.Host == "" {
if err != nil || (parsed.Scheme != "http" && parsed.Scheme != "https") || parsed.Host == "" {
outcome.Reachability = errors.New("探测配置无效")
return outcome
}
if !weknora.HTTPAllowed(baseURL, weknora.HTTPPolicy{AllowPrivateHTTP: provider.allowPrivateHTTP}) {
outcome.Reachability = errors.New("探测配置无效:该地址不被允许(https 或已放行的私网/回环 http)")
return outcome
}
parsed.Path = strings.TrimSuffix(parsed.Path, "/") + "/health"
parsed.RawQuery, parsed.Fragment = "", ""
request, err := http.NewRequestWithContext(ctx, http.MethodGet, parsed.String(), nil)
+6 -1
View File
@@ -50,6 +50,10 @@ type RealProviderConfig struct {
// WorkerStaleAfter bounds how long a missing worker heartbeat still counts
// as alive (P10).
WorkerStaleAfter time.Duration
// AllowPrivateHTTP relaxes the plain-http rule for same-LAN addresses. The
// console probes use the very same judgement as the WeKnora client, so a
// card can never say 「可达」 while the client refuses the same URL.
AllowPrivateHTTP bool
}
type RealProvider struct {
@@ -63,6 +67,7 @@ type RealProvider struct {
productionWeKnoraResolver func(ctx context.Context) string
weknoraReadiness func(ctx context.Context) error
workerStaleAfter time.Duration
allowPrivateHTTP bool
health *healthTracker
}
@@ -103,7 +108,7 @@ func NewRealProvider(config RealProviderConfig) (*RealProvider, error) {
if stale <= 0 {
stale = time.Minute
}
return &RealProvider{db: config.DB, now: config.Now, httpClient: config.HTTPClient, weknoraBaseURL: strings.TrimSpace(config.WeKnoraBaseURL), minerUBaseURL: strings.TrimSpace(config.MinerUBaseURL), testWeKnoraResolver: config.TestWeKnoraResolver, productionWeKnoraBaseURL: strings.TrimSpace(config.ProductionWeKnoraBaseURL), productionWeKnoraResolver: config.ProductionWeKnoraResolver, weknoraReadiness: config.WeKnoraReadiness, workerStaleAfter: stale, health: newHealthTracker()}, nil
return &RealProvider{db: config.DB, now: config.Now, httpClient: config.HTTPClient, weknoraBaseURL: strings.TrimSpace(config.WeKnoraBaseURL), minerUBaseURL: strings.TrimSpace(config.MinerUBaseURL), testWeKnoraResolver: config.TestWeKnoraResolver, productionWeKnoraBaseURL: strings.TrimSpace(config.ProductionWeKnoraBaseURL), productionWeKnoraResolver: config.ProductionWeKnoraResolver, weknoraReadiness: config.WeKnoraReadiness, workerStaleAfter: stale, allowPrivateHTTP: config.AllowPrivateHTTP, health: newHealthTracker()}, nil
}
func (provider *RealProvider) Overview(ctx context.Context) (governance.MonitorOverview, error) {
@@ -0,0 +1,3 @@
DROP TABLE IF EXISTS weknora_environment_connection_tests;
DROP TABLE IF EXISTS weknora_environment_connection_history;
DROP TABLE IF EXISTS weknora_environment_connections;
@@ -0,0 +1,45 @@
-- 连接是「地址 + 写-only 密钥」的一份状态,绑定仍然是「对内/对外目标库」的另一份
-- 状态(routing_kb_bindings / production_kb_bindings)。两者刻意不合并:现场那张
-- 把两者塞进一张表的孤表正是重复列的来源。
--
-- 编号从 000083 起:现场 0.9.0 的 schema_migrations 已登记到 82,而迁移器按号跳过、
-- 没有顺序守卫 —— 76..82 号会被静默丢弃。仓内最大号到 75 不影响 83 的排序与执行。
CREATE TABLE weknora_environment_connections (
environment VARCHAR(16) PRIMARY KEY CHECK (environment IN ('test','production')),
revision BIGINT NOT NULL DEFAULT 1 CHECK (revision > 0),
endpoint TEXT NOT NULL DEFAULT '',
key_ciphertext BYTEA NOT NULL DEFAULT '\x',
key_tail VARCHAR(8) NOT NULL DEFAULT '',
verification TEXT NOT NULL DEFAULT 'not_configured'
CHECK (verification IN ('not_configured','ok','unreachable','key_invalid','key_forbidden')),
library_count INTEGER NOT NULL DEFAULT 0 CHECK (library_count >= 0),
checked_at TIMESTAMPTZ,
updated_by TEXT NOT NULL DEFAULT '',
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- 历史只记「有没有配过密钥」,不记尾号也不记密文:一个可审计的时间线不需要能
-- 反推出任何一位密钥。
CREATE TABLE weknora_environment_connection_history (
environment VARCHAR(16) NOT NULL CHECK (environment IN ('test','production')),
revision BIGINT NOT NULL,
endpoint TEXT NOT NULL,
key_configured BOOLEAN NOT NULL,
verification TEXT NOT NULL,
actor TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (environment, revision)
);
-- 「测通才可保存」需要一条可核对的证据:这次测试用的是哪个地址、哪把密钥(只存
-- 指纹)、结论如何。保存时按 (environment, endpoint, key_fingerprint, ok) 匹配。
CREATE TABLE weknora_environment_connection_tests (
environment VARCHAR(16) PRIMARY KEY CHECK (environment IN ('test','production')),
endpoint TEXT NOT NULL,
key_fingerprint CHAR(64) NOT NULL CHECK (key_fingerprint ~ '^[0-9a-f]{64}$'),
outcome TEXT NOT NULL
CHECK (outcome IN ('ok','unreachable','key_invalid','key_forbidden')),
library_count INTEGER NOT NULL DEFAULT 0 CHECK (library_count >= 0),
message TEXT NOT NULL DEFAULT '',
checked_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
+20 -12
View File
@@ -11,51 +11,59 @@ func TestEmbeddedMigrationsAreVersionSorted(t *testing.T) {
if err != nil {
t.Fatalf("loadMigrations() error = %v", err)
}
if len(migrations) != 75 {
t.Fatalf("migration count = %d, want 75", len(migrations))
if len(migrations) != 76 {
t.Fatalf("migration count = %d, want 76", len(migrations))
}
// 83 把「连接(地址 + 写-only 密钥)」从文件引用搬进治理库;编号必须 ≥83,
// 因为现场 0.9.0 的 schema_migrations 已到 82,迁移器按号跳过且无顺序守卫。
if connections := migrations[len(migrations)-1]; connections.Version != 83 ||
!strings.Contains(connections.UpSQL, "weknora_environment_connections") ||
!strings.Contains(connections.UpSQL, "key_ciphertext") ||
!strings.Contains(connections.DownSQL, "DROP TABLE IF EXISTS weknora_environment_connections") {
t.Fatalf("migration 83 = %#v", connections)
}
// 75 允许"建立目标权威基线"作为独立业务作业,并与 sync/restore 共用写互斥。
if baseline := migrations[len(migrations)-1]; baseline.Version != 75 ||
if baseline := migrations[len(migrations)-2]; baseline.Version != 75 ||
!strings.Contains(baseline.UpSQL, "library_jobs_write_target") ||
!strings.Contains(baseline.UpSQL, "'baseline'") {
t.Fatalf("migration 75 = %#v", baseline)
}
// 74 记录封存执行器的运行时状态:页面可用性来自真实观测,不来自配置字符串。
if executor := migrations[len(migrations)-2]; executor.Version != 74 ||
if executor := migrations[len(migrations)-3]; executor.Version != 74 ||
!strings.Contains(executor.UpSQL, "library_executor_status") {
t.Fatalf("migration 74 = %#v", executor)
}
// 73 记录死信精确解除:保留上游行,只绑定指纹、重放引用与观测摘要。
if release := migrations[len(migrations)-3]; release.Version != 73 ||
if release := migrations[len(migrations)-4]; release.Version != 73 ||
!strings.Contains(release.UpSQL, "library_dead_letter_releases") {
t.Fatalf("migration 73 = %#v", release)
}
// 72 让维护占用按实例互斥(internal/external 不再各有各的锁)。
if instance := migrations[len(migrations)-4]; instance.Version != 72 ||
if instance := migrations[len(migrations)-5]; instance.Version != 72 ||
!strings.Contains(instance.UpSQL, "library_instance_maintenance") {
t.Fatalf("migration 72 = %#v", instance)
}
// 71 让维护记录能区分实例级维护与仅治理库互斥。
if level := migrations[len(migrations)-5]; level.Version != 71 ||
if level := migrations[len(migrations)-6]; level.Version != 71 ||
!strings.Contains(level.UpSQL, "level") || !strings.Contains(level.UpSQL, "ingress_state") {
t.Fatalf("migration 71 = %#v", level)
}
if maint := migrations[len(migrations)-6]; maint.Version != 70 || !strings.Contains(maint.UpSQL, "library_maintenance") {
if maint := migrations[len(migrations)-7]; maint.Version != 70 || !strings.Contains(maint.UpSQL, "library_maintenance") {
t.Fatalf("migration 70 = %#v", maint)
}
if tail := migrations[len(migrations)-7]; tail.Version != 69 ||
if tail := migrations[len(migrations)-8]; tail.Version != 69 ||
!strings.Contains(tail.UpSQL, "library_jobs_write_target") {
t.Fatalf("migration 69 = %#v", tail)
}
if restore := migrations[len(migrations)-8]; restore.Version != 68 || !strings.Contains(restore.UpSQL, "restore") {
if restore := migrations[len(migrations)-9]; restore.Version != 68 || !strings.Contains(restore.UpSQL, "restore") {
t.Fatalf("migration 68 = %#v", restore)
}
if last := migrations[len(migrations)-9]; last.Version != 67 ||
if last := migrations[len(migrations)-10]; last.Version != 67 ||
!strings.Contains(last.UpSQL, "library_release_authorizations") ||
!strings.Contains(last.UpSQL, "params_digest") {
t.Fatalf("migration 67 = %#v", last)
}
if flow := migrations[len(migrations)-10]; flow.Version != 66 ||
if flow := migrations[len(migrations)-11]; flow.Version != 66 ||
!strings.Contains(flow.UpSQL, "CREATE TABLE library_versions") ||
!strings.Contains(flow.UpSQL, "library_jobs_active") {
t.Fatalf("migration 66 = %#v", flow)
@@ -0,0 +1,203 @@
package postgres
import (
"context"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
// WeKnoraConnection is one environment's 地址 + 写-only 密钥. The ciphertext is
// opened only in the service layer; everything below this line treats it as an
// opaque blob so no query result can accidentally be rendered as a secret.
type WeKnoraConnection struct {
Environment string
Revision int64
Endpoint string
KeyCiphertext []byte
KeyTail string
Verification string
LibraryCount int
CheckedAt *time.Time
UpdatedBy string
UpdatedAt time.Time
}
// WeKnoraConnectionTest is the evidence a save needs: which endpoint, which key
// (fingerprint only) and what the last real listing said.
type WeKnoraConnectionTest struct {
Environment string
Endpoint string
KeyFingerprint string
Outcome string
LibraryCount int
Message string
CheckedAt time.Time
}
// LoadWeKnoraConnections reads both environments. A missing row is simply absent
// from the result: 「未配置」 is a state, not an error.
func (s *Store) LoadWeKnoraConnections(ctx context.Context) ([]WeKnoraConnection, error) {
rows, err := s.db.Query(ctx, `
SELECT environment,revision,endpoint,key_ciphertext,key_tail,verification,library_count,checked_at,updated_by,updated_at
FROM weknora_environment_connections
ORDER BY environment`)
if err != nil {
return nil, errors.New("load weknora connections")
}
defer rows.Close()
connections := make([]WeKnoraConnection, 0, 2)
for rows.Next() {
var connection WeKnoraConnection
if err := rows.Scan(&connection.Environment, &connection.Revision, &connection.Endpoint,
&connection.KeyCiphertext, &connection.KeyTail, &connection.Verification,
&connection.LibraryCount, &connection.CheckedAt, &connection.UpdatedBy, &connection.UpdatedAt); err != nil {
return nil, errors.New("load weknora connections")
}
connections = append(connections, connection)
}
if rows.Err() != nil {
return nil, errors.New("load weknora connections")
}
return connections, nil
}
// LoadWeKnoraConnection reads one environment. The boolean is false when the
// environment was never configured.
func (s *Store) LoadWeKnoraConnection(ctx context.Context, environment string) (WeKnoraConnection, bool, error) {
var connection WeKnoraConnection
err := s.db.QueryRow(ctx, `
SELECT environment,revision,endpoint,key_ciphertext,key_tail,verification,library_count,checked_at,updated_by,updated_at
FROM weknora_environment_connections WHERE environment=$1`, environment).
Scan(&connection.Environment, &connection.Revision, &connection.Endpoint,
&connection.KeyCiphertext, &connection.KeyTail, &connection.Verification,
&connection.LibraryCount, &connection.CheckedAt, &connection.UpdatedBy, &connection.UpdatedAt)
if errors.Is(err, pgx.ErrNoRows) {
return WeKnoraConnection{}, false, nil
}
if err != nil {
return WeKnoraConnection{}, false, errors.New("load weknora connection")
}
return connection, true, nil
}
// SaveWeKnoraConnection writes one revision and its history row in a single
// transaction. The advisory lock is per environment so two concurrent saves
// cannot both pick the same revision.
func (s *Store) SaveWeKnoraConnection(ctx context.Context, connection WeKnoraConnection, now time.Time) error {
if now.IsZero() {
return errors.New("weknora connection save requires a timestamp")
}
tx, err := s.db.Begin(ctx)
if err != nil {
return fmt.Errorf("begin weknora connection save: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock(hashtext('weknora_connection:'||$1))`, connection.Environment); err != nil {
return fmt.Errorf("lock weknora connection: %w", err)
}
if _, err := tx.Exec(ctx, `
INSERT INTO weknora_environment_connections(environment,revision,endpoint,key_ciphertext,key_tail,verification,library_count,checked_at,updated_by,updated_at)
VALUES($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)
ON CONFLICT(environment) DO UPDATE SET
revision=EXCLUDED.revision,
endpoint=EXCLUDED.endpoint,
key_ciphertext=EXCLUDED.key_ciphertext,
key_tail=EXCLUDED.key_tail,
verification=EXCLUDED.verification,
library_count=EXCLUDED.library_count,
checked_at=EXCLUDED.checked_at,
updated_by=EXCLUDED.updated_by,
updated_at=EXCLUDED.updated_at`,
connection.Environment, connection.Revision, connection.Endpoint, connection.KeyCiphertext,
connection.KeyTail, connection.Verification, connection.LibraryCount, connection.CheckedAt,
connection.UpdatedBy, now); err != nil {
return errors.New("save weknora connection")
}
if _, err := tx.Exec(ctx, `
INSERT INTO weknora_environment_connection_history(environment,revision,endpoint,key_configured,verification,actor,created_at)
VALUES($1,$2,$3,$4,$5,$6,$7)
ON CONFLICT(environment,revision) DO NOTHING`,
connection.Environment, connection.Revision, connection.Endpoint, len(connection.KeyCiphertext) > 0,
connection.Verification, connection.UpdatedBy, now); err != nil {
return errors.New("record weknora connection history")
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("commit weknora connection save: %w", err)
}
return nil
}
// SaveWeKnoraConnectionTest stores the outcome of the real listing a save will
// later point at. Only the key fingerprint is kept — never the key.
func (s *Store) SaveWeKnoraConnectionTest(ctx context.Context, result WeKnoraConnectionTest, now time.Time) error {
if now.IsZero() {
return errors.New("weknora connection test requires a timestamp")
}
if _, err := s.db.Exec(ctx, `
INSERT INTO weknora_environment_connection_tests(environment,endpoint,key_fingerprint,outcome,library_count,message,checked_at)
VALUES($1,$2,$3,$4,$5,$6,$7)
ON CONFLICT(environment) DO UPDATE SET
endpoint=EXCLUDED.endpoint,
key_fingerprint=EXCLUDED.key_fingerprint,
outcome=EXCLUDED.outcome,
library_count=EXCLUDED.library_count,
message=EXCLUDED.message,
checked_at=EXCLUDED.checked_at`,
result.Environment, result.Endpoint, result.KeyFingerprint, result.Outcome,
result.LibraryCount, result.Message, now); err != nil {
return errors.New("save weknora connection test")
}
return nil
}
// LoadWeKnoraConnectionTest reads the last test for one environment.
func (s *Store) LoadWeKnoraConnectionTest(ctx context.Context, environment string) (WeKnoraConnectionTest, bool, error) {
var result WeKnoraConnectionTest
err := s.db.QueryRow(ctx, `
SELECT environment,endpoint,key_fingerprint,outcome,library_count,message,checked_at
FROM weknora_environment_connection_tests WHERE environment=$1`, environment).
Scan(&result.Environment, &result.Endpoint, &result.KeyFingerprint, &result.Outcome,
&result.LibraryCount, &result.Message, &result.CheckedAt)
if errors.Is(err, pgx.ErrNoRows) {
return WeKnoraConnectionTest{}, false, nil
}
if err != nil {
return WeKnoraConnectionTest{}, false, errors.New("load weknora connection test")
}
return result, true, nil
}
// UpdateWeKnoraConnectionVerification refreshes the card state without touching
// the saved endpoint or ciphertext.
func (s *Store) UpdateWeKnoraConnectionVerification(ctx context.Context, environment, verification string, libraryCount int, checkedAt time.Time, now time.Time) error {
if _, err := s.db.Exec(ctx, `
UPDATE weknora_environment_connections
SET verification=$2, library_count=$3, checked_at=$4, updated_at=$5
WHERE environment=$1`, environment, verification, libraryCount, checkedAt, now); err != nil {
return errors.New("update weknora connection verification")
}
return nil
}
// ProductionExecutionCounts reports, for executions that are still in flight,
// how many items already have a frozen production target and how many do not yet.
// A production binding change never rewrites the frozen ones: an execution that
// already started keeps the knowledge base it was authorized against. Lifetime
// counts are deliberately excluded — completed executions are history, not
// something a binding change could still affect.
func (s *Store) ProductionExecutionCounts(ctx context.Context) (int, int, error) {
var frozen, retargetable int
if err := s.db.QueryRow(ctx, `
SELECT
COUNT(*) FILTER (WHERE item.target_knowledge_base_id <> ''),
COUNT(*) FILTER (WHERE item.target_knowledge_base_id = '')
FROM release_item_executions item
JOIN release_executions execution ON execution.id = item.release_execution_id
WHERE execution.state IN ('pending','running')`).Scan(&frozen, &retargetable); err != nil {
return 0, 0, errors.New("load production execution counts")
}
return frozen, retargetable, nil
}
@@ -0,0 +1,118 @@
package postgres
import (
"context"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// 连接(地址 + 写-only 密钥)是新的持久状态:一环境一行、修订递增、历史只记
// 「有没有配过密钥」。测试必须证明 000083 真能 up/down,而不是只证明 SQL 解析通过。
func TestWeKnoraConnectionsMigrationUpDownAndStore(t *testing.T) {
databaseURL := isolatedIntegrationDatabaseURL(t, os.Getenv("TEST_DATABASE_URL"))
if databaseURL == "" {
t.Skip("TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, databaseURL)
if err != nil {
t.Fatal(err)
}
defer pool.Close()
if err = ApplyMigrations(ctx, pool); err != nil {
t.Fatalf("apply migrations: %v", err)
}
var applied bool
if err = pool.QueryRow(ctx, "SELECT EXISTS (SELECT 1 FROM schema_migrations WHERE version = 83)").Scan(&applied); err != nil || !applied {
t.Fatalf("migration 83 not recorded: %v %v", applied, err)
}
store := NewStore(pool)
now := time.Date(2026, time.November, 22, 7, 0, 0, 0, time.UTC)
checked := now.Add(-time.Minute)
connection := WeKnoraConnection{
Environment: "production",
Revision: 2,
Endpoint: "http://172.29.151.11:18093",
KeyCiphertext: []byte{1, 2, 3, 4},
KeyTail: "9f2c",
Verification: "ok",
LibraryCount: 4,
CheckedAt: &checked,
UpdatedBy: "admin",
}
if err := store.SaveWeKnoraConnection(ctx, connection, now); err != nil {
t.Fatalf("save connection: %v", err)
}
loaded, exists, err := store.LoadWeKnoraConnection(ctx, "production")
if err != nil || !exists {
t.Fatalf("load connection: %v %v", exists, err)
}
if loaded.Endpoint != connection.Endpoint || loaded.Revision != 2 || loaded.KeyTail != "9f2c" || string(loaded.KeyCiphertext) != string(connection.KeyCiphertext) {
t.Fatalf("loaded = %+v", loaded)
}
var configured bool
var actor string
if err := pool.QueryRow(ctx, `SELECT key_configured,actor FROM weknora_environment_connection_history WHERE environment='production' AND revision=2`).Scan(&configured, &actor); err != nil {
t.Fatalf("history row: %v", err)
}
if !configured || actor != "admin" {
t.Fatalf("history = %v %q", configured, actor)
}
// 历史里不能出现尾号或密文:只有布尔。
var historyColumns int
if err := pool.QueryRow(ctx, `SELECT COUNT(*) FROM information_schema.columns WHERE table_schema=current_schema() AND table_name='weknora_environment_connection_history' AND column_name IN ('key_tail','key_ciphertext')`).Scan(&historyColumns); err != nil {
t.Fatal(err)
}
if historyColumns != 0 {
t.Fatalf("history table exposes %d secret columns", historyColumns)
}
// 测试证据:同一环境只保留最后一次,且只存密钥指纹。
fingerprint := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
if err := store.SaveWeKnoraConnectionTest(ctx, WeKnoraConnectionTest{
Environment: "production", Endpoint: connection.Endpoint, KeyFingerprint: fingerprint,
Outcome: "ok", LibraryCount: 4, Message: "接口可达",
}, now); err != nil {
t.Fatalf("save test evidence: %v", err)
}
proof, found, err := store.LoadWeKnoraConnectionTest(ctx, "production")
if err != nil || !found || proof.KeyFingerprint != fingerprint || proof.Outcome != "ok" {
t.Fatalf("load test evidence = %+v %v %v", proof, found, err)
}
// 冻结/未冻结计数按 target_knowledge_base_id 判定,不能用 IS NULL。
frozen, retargetable, err := store.ProductionExecutionCounts(ctx)
if err != nil {
t.Fatalf("production execution counts: %v", err)
}
if frozen != 0 || retargetable != 0 {
t.Fatalf("empty execution counts = %d/%d", frozen, retargetable)
}
// down 必须只删这三张新表,不碰既有绑定表。
if err := RollbackLatestMigration(ctx, pool); err != nil {
t.Fatalf("rollback 83: %v", err)
}
for _, table := range []string{"weknora_environment_connections", "weknora_environment_connection_history", "weknora_environment_connection_tests"} {
var present bool
if err := pool.QueryRow(ctx, `SELECT EXISTS (SELECT 1 FROM information_schema.tables WHERE table_schema=current_schema() AND table_name=$1)`, table).Scan(&present); err != nil {
t.Fatal(err)
}
if present {
t.Fatalf("%s survived the down migration", table)
}
}
for _, table := range []string{"routing_kb_bindings", "production_kb_bindings"} {
var present bool
if err := pool.QueryRow(ctx, `SELECT EXISTS (SELECT 1 FROM information_schema.tables WHERE table_schema=current_schema() AND table_name=$1)`, table).Scan(&present); err != nil {
t.Fatal(err)
}
if !present {
t.Fatalf("%s must not be dropped by migration 83 down", table)
}
}
}
+49 -1
View File
@@ -48,6 +48,43 @@ type ClientConfig struct {
Timeout time.Duration
MaxResponseBytes int64
AllowDockerHTTP bool
// AllowPrivateHTTP relaxes the http-only rule for RFC1918, link-local and
// ULA literals. It exists because the production instance sits behind a
// same-LAN nginx that speaks plain http; https stays the default and any
// public http host is still refused.
AllowPrivateHTTP bool
}
// HTTPPolicy is the single reachability judgement shared by the WeKnora client
// and the console probes: two callers must never disagree about whether a
// configured endpoint is acceptable.
type HTTPPolicy struct {
AllowPrivateHTTP bool
AllowDockerHTTP bool
}
// HTTPAllowed reports whether baseURL may be called at all. https is always
// allowed; http only for loopback, an opted-in private address, or (legacy)
// a single-label docker host.
func HTTPAllowed(rawURL string, policy HTTPPolicy) bool {
parsed, err := url.Parse(strings.TrimSpace(rawURL))
if err != nil || parsed.Host == "" || parsed.User != nil {
return false
}
switch parsed.Scheme {
case "https":
return true
case "http":
host := parsed.Hostname()
if isLoopbackHost(host) {
return true
}
if policy.AllowPrivateHTTP && isPrivateHost(host) {
return true
}
return policy.AllowDockerHTTP && isSingleLabelDockerHost(host)
}
return false
}
type Client struct {
@@ -65,7 +102,7 @@ func NewClient(config ClientConfig) (*Client, error) {
if baseURL.Scheme == "" || baseURL.Host == "" || baseURL.User != nil || baseURL.RawQuery != "" || baseURL.Fragment != "" {
return nil, errors.New("invalid WeKnora base URL")
}
if baseURL.Scheme != "https" && !(baseURL.Scheme == "http" && (isLoopbackHost(baseURL.Hostname()) || (config.AllowDockerHTTP && isSingleLabelDockerHost(baseURL.Hostname())))) {
if !HTTPAllowed(config.BaseURL, HTTPPolicy{AllowPrivateHTTP: config.AllowPrivateHTTP, AllowDockerHTTP: config.AllowDockerHTTP}) {
return nil, errors.New("WeKnora base URL must use HTTPS")
}
baseURL.Path = strings.TrimRight(baseURL.Path, "/")
@@ -99,6 +136,17 @@ func isLoopbackHost(host string) bool {
return ip != nil && ip.IsLoopback()
}
// isPrivateHost reports the addresses a same-LAN deployment legitimately uses:
// RFC1918, link-local and IPv6 ULA. A public name is never private, so opting
// in cannot widen the client to the internet.
func isPrivateHost(host string) bool {
ip := net.ParseIP(host)
if ip == nil {
return false
}
return ip.IsPrivate() || ip.IsLinkLocalUnicast() || ip.IsLinkLocalMulticast()
}
func isSingleLabelDockerHost(host string) bool {
if host == "" || strings.Contains(host, ".") || net.ParseIP(host) != nil || len(host) > 63 {
return false
@@ -0,0 +1,44 @@
package weknora
import "testing"
func TestHTTPAllowedSharesOneJudgement(t *testing.T) {
cases := []struct {
name string
url string
policy HTTPPolicy
want bool
}{
{"https always", "https://weknora.example", HTTPPolicy{}, true},
{"loopback http", "http://127.0.0.1:8092", HTTPPolicy{}, true},
{"localhost http", "http://localhost:8092", HTTPPolicy{}, true},
{"private http refused by default", "http://172.29.151.11:18093", HTTPPolicy{}, false},
{"private http allowed when opted in", "http://172.29.151.11:18093", HTTPPolicy{AllowPrivateHTTP: true}, true},
{"rfc1918 /8", "http://10.1.2.3", HTTPPolicy{AllowPrivateHTTP: true}, true},
{"rfc1918 /16", "http://192.168.1.10", HTTPPolicy{AllowPrivateHTTP: true}, true},
{"link-local", "http://169.254.10.10", HTTPPolicy{AllowPrivateHTTP: true}, true},
{"public http still refused", "http://203.0.113.9", HTTPPolicy{AllowPrivateHTTP: true}, false},
{"public name still refused", "http://weknora.example", HTTPPolicy{AllowPrivateHTTP: true}, false},
{"docker single label opt-in", "http://weknora", HTTPPolicy{AllowDockerHTTP: true}, true},
{"docker single label off", "http://weknora", HTTPPolicy{}, false},
{"credentials refused", "http://user:pass@127.0.0.1:8092", HTTPPolicy{}, false},
{"empty host", "http://", HTTPPolicy{}, false},
{"other scheme", "ftp://127.0.0.1", HTTPPolicy{}, false},
}
for _, testCase := range cases {
if got := HTTPAllowed(testCase.url, testCase.policy); got != testCase.want {
t.Fatalf("%s: HTTPAllowed(%q, %+v) = %v, want %v", testCase.name, testCase.url, testCase.policy, got, testCase.want)
}
}
}
func TestNewClientAcceptsPrivateHTTPOnlyWhenOptedIn(t *testing.T) {
base := ClientConfig{BaseURL: "http://172.29.151.11:18093", APIKey: "k", Timeout: 1}
if _, err := NewClient(base); err == nil {
t.Fatal("private http must be refused without the opt-in")
}
base.AllowPrivateHTTP = true
if _, err := NewClient(base); err != nil {
t.Fatalf("private http must be accepted with the opt-in: %v", err)
}
}
+21 -41
View File
@@ -1,55 +1,35 @@
package qaopsai
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"fmt"
"os"
"knowledge-governance/internal/sealedsecret"
)
// The independently provisioned master key never lives in source, settings JSON or database backups.
func (s *Store) masterKey() ([]byte, error) {
info, err := os.Lstat(s.options.KeyFile)
if err != nil || !info.Mode().IsRegular() || info.Mode().Perm() != 0600 {
return nil, ErrUnavailable
}
raw, err := os.ReadFile(s.options.KeyFile)
if err != nil || len(raw) != 32 {
return nil, ErrUnavailable
}
return raw, nil
}
func (s *Store) aead() (cipher.AEAD, error) {
key, err := s.masterKey()
if err != nil {
return nil, err
}
block, err := aes.NewCipher(key)
if err != nil {
return nil, ErrUnavailable
}
return cipher.NewGCM(block)
}
// QA-AI keeps one write-only credential per revision. The AES-GCM convention
// itself lives in internal/sealedsecret so a second feature cannot grow a
// second, slightly different crypto path; only the AAD domain and the error
// vocabulary stay here.
const qaAISecretScope = "qa-ai"
func (s *Store) seal(secret string, revision int64) ([]byte, error) {
aead, err := s.aead()
key, err := sealedsecret.LoadKeyFile(s.options.KeyFile)
if err != nil {
return nil, err
}
nonce := make([]byte, aead.NonceSize())
if _, err = rand.Read(nonce); err != nil {
return nil, ErrUnavailable
}
return aead.Seal(nonce, nonce, []byte(secret), []byte(fmt.Sprintf("qa-ai:%s:%d", s.tenant, revision))), nil
}
func (s *Store) unseal(sealed []byte, revision int64) (string, error) {
aead, err := s.aead()
if err != nil || len(sealed) < aead.NonceSize() {
return "", ErrUnavailable
sealed, err := sealedsecret.Seal(key, secret, sealedsecret.AAD(qaAISecretScope, s.tenant, revision))
if err != nil {
return nil, ErrUnavailable
}
raw, err := aead.Open(nil, sealed[:aead.NonceSize()], sealed[aead.NonceSize():], []byte(fmt.Sprintf("qa-ai:%s:%d", s.tenant, revision)))
return sealed, nil
}
func (s *Store) unseal(sealed []byte, revision int64) (string, error) {
key, err := sealedsecret.LoadKeyFile(s.options.KeyFile)
if err != nil {
return "", ErrUnavailable
}
return string(raw), nil
plain, err := sealedsecret.Open(key, sealed, sealedsecret.AAD(qaAISecretScope, s.tenant, revision))
if err != nil {
return "", ErrUnavailable
}
return plain, nil
}
+3 -1
View File
@@ -9,6 +9,8 @@ import (
"net/url"
"strings"
"time"
"knowledge-governance/internal/sealedsecret"
)
type Options struct {
@@ -67,7 +69,7 @@ func (s *Store) readSettings(ctx context.Context, q settingsQuery) (privateSetti
return out, ErrUnavailable
}
out.KeyConfigured = len(out.sealed) > 0
_, keyErr := s.masterKey()
_, keyErr := sealedsecret.LoadKeyFile(s.options.KeyFile)
out.StorageReady = keyErr == nil
return out, nil
}
+4 -1
View File
@@ -48,7 +48,10 @@ func TestNormalizeRefusesEnabledSettingsWithoutTransportAndBadRecipients(t *test
t.Fatalf("defaults not applied: %+v %v", good, err)
}
// 解析不了的主机名必须仍被判为非法:.invalid 是 RFC 6761 保留域,永远不解析。
if _, err := normalize(SettingsInput{Host: "smtp.kg-test.invalid", From: "kb@example.com",
// 末尾的点是必须的:构建机的 resolv.conf 可能带 search 域(本环境 Tailscale
// 的 tailf49802.ts.net),不带点就会被拼成 smtp.kg-test.invalid.<search> 并
// 由 Fake-IP DNS 解析出来,于是这条断言测的是构建机的搜索域而不是被测逻辑。
if _, err := normalize(SettingsInput{Host: "smtp.kg-test.invalid.", From: "kb@example.com",
Recipients: []string{"a@example.com"}}); err != ErrInvalid {
t.Fatalf("unresolvable host must stay invalid, got %v", err)
}
@@ -113,6 +113,12 @@ func (s *ProductionBackupStore) Prepare(ctx context.Context, _ string, current l
if err != nil {
return libraryrelease.BackupRef{}, err
}
// 整实例指纹:恢复之后要拿它证明"多写的、少写的都没有"。只按目标范围比,
// 写到范围外的行会被漏掉(真实事故:恢复报成功但目标多 67 行)。
universeBefore, err := universeFingerprint(ctx, s.pool)
if err != nil {
return libraryrelease.BackupRef{}, err
}
pg, err := ExportLibrary(ctx, s.pool, s.identity)
if err != nil {
return libraryrelease.BackupRef{}, fmt.Errorf("读取生产整库失败: %w", err)
@@ -154,6 +160,13 @@ func (s *ProductionBackupStore) Prepare(ctx context.Context, _ string, current l
return libraryrelease.BackupRef{}, fmt.Errorf("预备份期间生产 %s 内容发生变化,切点不一致", table)
}
}
universeAfter, err := universeFingerprint(ctx, s.pool)
if err != nil {
return libraryrelease.BackupRef{}, err
}
if diff := universeDiff(universeBefore, universeAfter); diff != "" {
return libraryrelease.BackupRef{}, fmt.Errorf("预备份期间整实例知识域发生变化(%s),切点不一致", diff)
}
for table, want := range current.Counts {
if table == "index" {
continue
@@ -171,7 +184,7 @@ func (s *ProductionBackupStore) Prepare(ctx context.Context, _ string, current l
total += part.Bytes
}
return libraryrelease.BackupRef{ID: "backup-" + snapshotID, SnapshotID: snapshotID, Bytes: total,
CreatedAt: createdAt}, nil
CreatedAt: createdAt, Universe: universeBefore}, nil
}
func (s *ProductionBackupStore) rowCounts(ctx context.Context) (map[string]int64, error) {
@@ -84,8 +84,16 @@ type ProductionLibraryTarget struct {
scope ProductionScope
search *SearchExporter
now func() time.Time
// storageBackend 是本次写入使用的目标存储后端 id(写入与校验共用)。
storageBackend string
// storageBackend / storageProvider 是本次写入使用的目标存储后端与提供方
// (写入与校验共用,缺一不可:只换桶不换提供方,目标取文件仍会走来源实例的路)。
storageBackend string
storageProvider string
// snapshotTenant / snapshotKB 是这次要写入的快照来源身份。写入时按它做
// 来源→目标的范围映射;校验时按它反查"目标里还有没有来源范围的行"。
snapshotTenant uint64
snapshotKB string
// targetEmbeddingModel 是目标知识库绑定的 embedding 模型 id(文档行要换成它)。
targetEmbeddingModel string
}
// resolveStorageBackend 在目标库里找本次写入的存储后端。找不到就报错:宁可不写,
@@ -98,24 +106,39 @@ func (t *ProductionLibraryTarget) resolveStorageBackend(ctx context.Context) err
if !validSearchIdentifier(t.config.ObjectsBackendID) {
return errors.New("目标存储后端 id 形状不合法")
}
t.storageBackend = t.config.ObjectsBackendID
provider, err := t.storageProviderOf(ctx, t.config.ObjectsBackendID)
if err != nil {
return err
}
t.storageBackend, t.storageProvider = t.config.ObjectsBackendID, provider
return nil
}
var id string
var id, provider string
err := t.pool.QueryRow(ctx, `
SELECT id FROM storage_backends
SELECT id, coalesce(provider,'') FROM storage_backends
WHERE tenant_id=$1 AND coalesce(config->>'bucket_name','')=$2
ORDER BY created_at LIMIT 1`, t.scope.TenantID, t.config.ObjectsBucket).Scan(&id)
ORDER BY created_at LIMIT 1`, t.scope.TenantID, t.config.ObjectsBucket).Scan(&id, &provider)
if err != nil {
return errors.New("目标实例没有与目标桶匹配的存储后端,拒绝写入不可解析的资源定位符")
}
if !validSearchIdentifier(id) {
return errors.New("目标存储后端 id 形状不合法")
}
t.storageBackend = id
t.storageBackend, t.storageProvider = id, provider
return nil
}
// storageProviderOf 读出目标存储后端的提供方(minio / oss / s3…)。资源行里的
// 提供方必须与它一致,否则目标应用会按来源提供方去找一个不存在的存储服务。
func (t *ProductionLibraryTarget) storageProviderOf(ctx context.Context, id string) (string, error) {
var provider string
if err := t.pool.QueryRow(ctx, `SELECT coalesce(provider,'') FROM storage_backends WHERE id=$1 AND tenant_id=$2`,
id, t.scope.TenantID).Scan(&provider); err != nil {
return "", errors.New("目标实例没有这个存储后端,拒绝写入不可解析的资源定位符")
}
return provider, nil
}
func NewProductionLibraryTarget(ctx context.Context, config ProductionLibraryConfig, scope ProductionScope) (*ProductionLibraryTarget, error) {
if err := config.validate(scope); err != nil {
return nil, err
@@ -259,6 +282,7 @@ func (t *ProductionLibraryTarget) Apply(ctx context.Context, stage libraryreleas
if err := t.resolveStorageBackend(ctx); err != nil {
return err
}
t.resolveEmbeddingModel(ctx)
archive, err := librarysnapshot.Open(t.config.ArchiveRoot, 256*1024*1024)
if err != nil {
return errors.New("归档不可用")
@@ -268,6 +292,9 @@ func (t *ProductionLibraryTarget) Apply(ctx context.Context, stage libraryreleas
if err != nil {
return errors.New("预发布快照不可读")
}
if err := t.bindSnapshotScope(manifestScope{tenantID: manifest.TenantID, knowledgeBaseID: manifest.KnowledgeBaseID}); err != nil {
return err
}
switch stage {
case libraryrelease.StageDatabase, libraryrelease.StageWiki:
kind := "database"
@@ -356,10 +383,10 @@ func (t *ProductionLibraryTarget) insertRows(ctx context.Context, tx pgx.Tx, arc
if len(row) == 0 {
continue
}
if table == "resources" {
if err := t.applyResourceBucket(row); err != nil {
return err
}
// 行必须先映射到目标范围再落库:原样插入会把行写进来源租户/知识库,
// 目标范围的删除、校验与恢复都看不见它们。
if err := t.retargetRow(table, row); err != nil {
return err
}
if columns == nil {
columns = make([]string, 0, len(row))
@@ -408,53 +435,9 @@ type resourceLocation struct {
}
// rowTransform 是"写入时对行做过的变换"的统一入口:校验要用同一份,否则摘要比的是
// 变换前的行、目标里是变换后的行,永远不一致。
// 变换前的行、目标里是变换后的行,永远不一致。范围映射同样必须两边一致。
func (t *ProductionLibraryTarget) rowTransform(table string) func(map[string]json.RawMessage) error {
if table == "resources" {
return t.applyResourceBucket
}
return nil
}
// applyResourceBucket 把资源行定位符里的桶换成本次写入的目标桶。
// 写入与校验必须用同一个变换:只改写入侧会让"行内容摘要"对不上(真跑踩过)。
func (t *ProductionLibraryTarget) applyResourceBucket(row map[string]json.RawMessage) error {
if t.config.ObjectsBucket == "" {
return nil
}
raw, ok := row["physical_path"]
if !ok {
return errors.New("资源行缺少定位符")
}
var locator string
if err := json.Unmarshal(raw, &locator); err != nil {
return errors.New("资源定位符不可解析")
}
rewritten, ok := rewriteLocatorTarget(locator, t.storageBackend, t.config.ObjectsBucket)
if !ok {
return fmt.Errorf("资源定位符无法改写到目标桶:%s", locator)
}
encoded, err := json.Marshal(rewritten)
if err != nil {
return err
}
row["physical_path"] = encoded
// 原版应用先按 resources.storage_backend_id 找存储后端,再回落到 KB,再落默认服务。
// 这一列留着来源 id,目标仍然 500(真跑:storage backend not found → invalid path)。
if t.storageBackend == "" {
return nil
}
if raw, ok := row["storage_backend_id"]; ok {
var current string
if json.Unmarshal(raw, &current) == nil && current != "" && current != t.storageBackend {
encodedBackend, err := json.Marshal(t.storageBackend)
if err != nil {
return err
}
row["storage_backend_id"] = encodedBackend
}
}
return nil
return func(row map[string]json.RawMessage) error { return t.retargetRow(table, row) }
}
func (t *ProductionLibraryTarget) importResources(ctx context.Context, archive *librarysnapshot.Archive, manifest librarysnapshot.Manifest) error {
@@ -591,6 +574,7 @@ func (t *ProductionLibraryTarget) Verify(ctx context.Context, expect libraryrele
if err := t.resolveStorageBackend(ctx); err != nil {
return err
}
t.resolveEmbeddingModel(ctx)
archive, err := librarysnapshot.Open(t.config.ArchiveRoot, 256*1024*1024)
if err != nil {
return errors.New("归档不可用")
@@ -600,6 +584,9 @@ func (t *ProductionLibraryTarget) Verify(ctx context.Context, expect libraryrele
if err != nil {
return errors.New("预发布快照不可读")
}
if err := t.bindSnapshotScope(manifestScope{tenantID: manifest.TenantID, knowledgeBaseID: manifest.KnowledgeBaseID}); err != nil {
return err
}
for _, part := range manifest.Parts {
if part.Kind != "database" && part.Kind != "wiki" {
continue
@@ -668,6 +655,72 @@ func (t *ProductionLibraryTarget) Verify(ctx context.Context, expect libraryrele
if err := t.verifyObjects(ctx, archive, manifest); err != nil {
return err
}
// 目标里不允许留下"快照来源范围"的行:那说明有写入逃出了目标范围,
// 而恢复只按目标范围回放,永远删不到它们(真实事故就是这样被漏掉的)。
if err := t.assertNoSourceScopeRows(ctx, archive, manifest); err != nil {
return err
}
return nil
}
// dropOrphanResources 清掉"本次恢复后已无任何归属"的目标租户资源行。
//
// 判据必须同时满足:属于目标租户、不在预备份快照里、没有任何 resource_binding、
// 也没有任何分块正文以 resource://<handle> 引用它。这样的行对目标应用完全不可见
// (没有知识库能按它取到文件),只可能是失败写入留下的残渣。
func (t *ProductionLibraryTarget) dropOrphanResources(ctx context.Context, backupSnapshotID string) error {
archive, err := librarysnapshot.Open(t.config.ArchiveRoot, 256*1024*1024)
if err != nil {
return errors.New("归档不可用")
}
defer archive.Close()
manifest, err := archive.Load(backupSnapshotID)
if err != nil {
return errors.New("预备份快照不可读")
}
kept := []string{}
for _, part := range manifest.Parts {
if part.Kind != "database" || part.Name != "resources" {
continue
}
ids, err := archivedResourceIDs(archive, part)
if err != nil {
return err
}
kept = append(kept, ids...)
}
rows, err := t.pool.Query(ctx, `
SELECT r.id FROM resources AS r
WHERE r.tenant_id=$1
AND NOT (r.id = ANY($2::text[]))
AND NOT EXISTS (SELECT 1 FROM resource_bindings b WHERE b.resource_id=r.id)
AND NOT EXISTS (SELECT 1 FROM chunks c WHERE c.tenant_id=r.tenant_id AND c.deleted_at IS NULL
AND coalesce(r.handle,'') <> ''
AND position('resource://'||r.handle IN coalesce(c.content,'')) > 0)`,
t.scope.TenantID, kept)
if err != nil {
return fmt.Errorf("查找目标残留资源失败: %w", err)
}
orphans := make([]string, 0, 8)
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
rows.Close()
return err
}
orphans = append(orphans, id)
}
rows.Close()
if err := rows.Err(); err != nil {
return err
}
if len(orphans) == 0 {
return nil
}
if _, err := t.pool.Exec(ctx, `DELETE FROM resources WHERE tenant_id=$1 AND id = ANY($2::text[])`,
t.scope.TenantID, orphans); err != nil {
return fmt.Errorf("清理目标残留资源失败: %w", err)
}
return nil
}
@@ -772,5 +825,21 @@ func (t *ProductionLibraryTarget) Restore(ctx context.Context, backup libraryrel
return fmt.Errorf("恢复阶段 %s 失败: %w", stage, err)
}
}
// resources 是多知识库共享表,回放阶段不删它:本次写入新增、恢复后已经
// 没有任何归属的资源行会残留。它们对应用不可见,但属于"多写的"。
if err := t.dropOrphanResources(ctx, backup.SnapshotID); err != nil {
return err
}
// 恢复工具把四个阶段都跑完不算恢复完成:整实例必须回到预备份时刻的样子。
// 只比目标范围会把"写到范围外"的残留当成已恢复(真实事故:恢复后仍多 67 行)。
if len(backup.Universe) > 0 {
after, err := universeFingerprint(ctx, t.pool)
if err != nil {
return fmt.Errorf("恢复后读取整库指纹失败: %w", err)
}
if diff := universeDiff(backup.Universe, after); diff != "" {
return fmt.Errorf("恢复后目标与预备份时刻不一致:%s(很可能有写入逃出了预备份范围,不能当作已恢复)", diff)
}
}
return nil
}
@@ -4,12 +4,15 @@ import (
"bytes"
"context"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"testing"
"time"
@@ -390,3 +393,339 @@ func sealTargetFixture(t *testing.T, archive *librarysnapshot.Archive, scope str
}
return id
}
// memoryObjectStore 是"真实字节"的资源存储替身:Put/Open/Stat/Copy 都按摘要核对,
// 可以强制写入失败,用来复现"资源阶段失败"这一半写场景。
type memoryObjectStore struct {
objects map[string][]byte
failPut bool
puts []string
}
func newMemoryObjectStore(objects map[string][]byte) *memoryObjectStore {
copied := map[string][]byte{}
for key, value := range objects {
copied[key] = append([]byte(nil), value...)
}
return &memoryObjectStore{objects: copied}
}
func objectKey(provider, path string) string { return provider + "|" + path }
func (s *memoryObjectStore) Put(_ context.Context, provider, path string, reader io.Reader, size int64) error {
if s.failPut {
return errors.New("对象存储不可写")
}
body, err := io.ReadAll(reader)
if err != nil || int64(len(body)) != size {
return errors.New("对象字节数与声明不符")
}
s.objects[objectKey(provider, path)] = body
s.puts = append(s.puts, path)
return nil
}
func (s *memoryObjectStore) Open(_ context.Context, provider, path string) (io.ReadCloser, error) {
body, ok := s.objects[objectKey(provider, path)]
if !ok {
return nil, errors.New("对象不存在")
}
return io.NopCloser(bytes.NewReader(body)), nil
}
func (s *memoryObjectStore) Stat(_ context.Context, provider, path string) (int64, string, error) {
body, ok := s.objects[objectKey(provider, path)]
if !ok {
return 0, "", errors.New("对象不存在")
}
digest := sha256.Sum256(body)
return int64(len(body)), hex.EncodeToString(digest[:]), nil
}
func (s *memoryObjectStore) Copy(_ context.Context, provider, source, target string) error {
body, ok := s.objects[objectKey(provider, source)]
if !ok {
return errors.New("对象不存在")
}
s.objects[objectKey(provider, target)] = append([]byte(nil), body...)
return nil
}
// 真实事故的形状:来源 KB 与目标 KB 不同(租户也不同)、目标非空、资源阶段写失败。
// 旧代码把归档行原样写进目标(落在来源租户/知识库),随后的回滚只清目标范围,
// 于是"已恢复到预备份"却留下 4 文件 / 38 分块 / 21 资源 / 4 绑定。
func TestLibraryCrossScopeSyncRollbackLeavesNoResidueIsolated(t *testing.T) {
dsn, base := os.Getenv("KG_LIBRARY_TEST_DATABASE_URL"), os.Getenv("KG_LIBRARY_TEST_ELASTIC_URL")
if dsn == "" || base == "" || os.Getenv("KG_LIBRARY_TEST_ISOLATED") != "1" {
t.Skip("requires isolated PG and ES opt-in")
}
cfg, err := pgxpool.ParseConfig(dsn)
if err != nil || !strings.HasPrefix(cfg.ConnConfig.Database, "kg_library_test_") {
t.Fatal("isolated database required")
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
pool, err := pgxpool.NewWithConfig(ctx, cfg)
if err != nil {
t.Fatalf("open isolated PG: %v", err)
}
defer pool.Close()
for _, statement := range []string{
`DROP SCHEMA public CASCADE; CREATE SCHEMA public`,
libraryTestSchema,
`ALTER TABLE wiki_pages ADD COLUMN status text DEFAULT 'published'`,
`ALTER TABLE knowledge_tag_relations ADD COLUMN tenant_id bigint`,
`ALTER TABLE resources ADD COLUMN provider text, ADD COLUMN size bigint, ADD COLUMN storage_backend_id text`,
`ALTER TABLE resources ADD PRIMARY KEY (id)`,
`ALTER TABLE resource_bindings ADD COLUMN relation text`,
`INSERT INTO knowledge_bases VALUES('kb-target',10001,'{}',NULL),('kb-source',10000,'{}',NULL)`,
`CREATE TABLE storage_backends(id text PRIMARY KEY, tenant_id bigint, provider text, config jsonb, created_at timestamptz)`,
`INSERT INTO storage_backends VALUES('backend-target',10001,'minio','{"bucket_name":"kg-test-bucket"}',now())`,
// 目标非空:目标自己有文档/分块,另一个租户也有数据(回滚不得碰它)。
`INSERT INTO knowledges VALUES('doc-target',10001,'kb-target','completed',0,'',NULL)`,
`INSERT INTO knowledges VALUES('doc-foreign',20000,'kb-other','completed',0,'',NULL)`,
`INSERT INTO chunks VALUES('chunk-target',10001,'kb-target','doc-target','target content',NULL)`,
`INSERT INTO chunks VALUES('chunk-foreign',20000,'kb-other','doc-foreign','foreign content',NULL)`,
`UPDATE resource_bindings SET tenant_id=10000 WHERE tenant_id IS NULL`,
} {
if _, err := pool.Exec(ctx, statement); err != nil {
t.Fatalf("fixture schema: %v", err)
}
}
token := make([]byte, 8)
if _, err := rand.Read(token); err != nil {
t.Fatal(err)
}
index := "kg_library_test_" + hex.EncodeToString(token)
client := &http.Client{Timeout: 20 * time.Second, CheckRedirect: func(*http.Request, []*http.Request) error {
return http.ErrUseLastResponse
}}
call := func(method, path, body string, want int) {
t.Helper()
request, err := http.NewRequestWithContext(ctx, method, base+path, strings.NewReader(body))
if err != nil {
t.Fatal(err)
}
request.Header.Set("Content-Type", "application/json")
response, err := client.Do(request)
if err != nil {
t.Fatalf("fixture HTTP: %v", err)
}
defer response.Body.Close()
if response.StatusCode != want {
t.Fatalf("fixture %s %s => %d, want %d", method, path, response.StatusCode, want)
}
}
call("PUT", "/"+index, `{"mappings":{"properties":{"tenant_id":{"type":"long"},"knowledge_base_id":{"type":"keyword"},"knowledge_id":{"type":"keyword"},"chunk_id":{"type":"keyword"},"content":{"type":"text"},"embedding":{"type":"dense_vector","dims":2}}}}`, 200)
defer func() {
call("DELETE", "/"+index, "", 200)
call("HEAD", "/"+index, "", 404)
}()
// 目标当前索引自洽:目标分块必须有对应文档,否则预备份会(正确地)拒绝。
call("PUT", "/"+index+"/_doc/chunk-target?refresh=true",
`{"tenant_id":10001,"knowledge_base_id":"kb-target","knowledge_id":"doc-target","chunk_id":"chunk-target","source_id":"chunk-target","content":"target content","embedding":[0.1,0.2]}`, 201)
archiveRoot := t.TempDir()
if err := os.Chmod(archiveRoot, 0700); err != nil {
t.Fatal(err)
}
archive, err := librarysnapshot.Open(archiveRoot, 64*1024*1024)
if err != nil {
t.Fatal(err)
}
defer archive.Close()
snapshotID := sealCrossScopeFixture(t, archive, 10000, "kb-source")
store := newMemoryObjectStore(nil)
target, err := NewProductionLibraryTarget(ctx, ProductionLibraryConfig{
Postgres: dsn, SearchURL: base, SearchIndex: index, Objects: store, ObjectsReader: store,
ArchiveRoot: archiveRoot, ObjectsBucket: "kg-test-bucket", MaxDocuments: 100, HTTPClient: client,
}, ProductionScope{TenantID: 10001, KnowledgeBaseID: "kb-target"})
if err != nil {
t.Fatalf("target: %v", err)
}
defer target.Close()
mapping, err := target.Search().MappingSHA256(ctx)
if err != nil {
t.Fatal(err)
}
backups, err := NewProductionBackupStore(ProductionBackupConfig{
Pool: target.Pool(), Search: target.Search(), Objects: store, ArchiveRoot: archiveRoot,
Scope: "internal", Identity: target.Identity(), MaxDocuments: 100,
Compatibility: librarysnapshot.Compatibility{WeKnoraVersion: "v0.7.1", SchemaVersion: 74,
RetrieverEngine: "elasticsearch_v8", EmbeddingModel: "text-embedding-v4",
EmbeddingDimension: 1024, IndexMappingSHA256: mapping},
})
if err != nil {
t.Fatalf("backup store: %v", err)
}
now := time.Now().UTC()
request := libraryrelease.Request{Target: "internal", SnapshotID: snapshotID,
ParamsDigest: "digest-cross-scope", Now: now,
Grant: libraryrelease.Grant{Target: "internal", SnapshotID: snapshotID, ParamsDigest: "digest-cross-scope",
ExpiresAt: now.Add(10 * time.Minute), AuthorizedBy: "acceptance"}}
// 1) 资源阶段失败 → 必须回滚且不留下来源范围的行。
store.failPut = true
outcome := libraryrelease.Run(ctx, request, target, backups, nil)
if outcome.State != "rolled_back" {
t.Fatalf("outcome = %+v", outcome)
}
if outcome.FailedAt != libraryrelease.StageResources {
t.Fatalf("failed_at = %s, want resources", outcome.FailedAt)
}
if outcome.Backup.SnapshotID == "" || len(outcome.Backup.Universe) == 0 {
t.Fatalf("backup must carry the whole-instance fingerprint: %+v", outcome.Backup)
}
for _, table := range []string{"knowledges", "chunks", "wiki_pages"} {
var residue int64
if err := pool.QueryRow(ctx, fmt.Sprintf(`SELECT count(*) FROM %s WHERE tenant_id=$1 AND knowledge_base_id=$2`, table),
10000, "kb-source").Scan(&residue); err != nil {
t.Fatal(err)
}
if residue != 0 {
t.Fatalf("%s still carries %d source-scope rows after rollback", table, residue)
}
}
// 夹具在租户 10000 里本来就有自己的行(res-safe / binding-safe):它们必须原样
// 留下,只有这次同步写进去的行(res-source / 绑到 doc-source)才算残留。
for query, want := range map[string]int64{
`SELECT count(*) FROM resources WHERE id='res-source'`: 0,
`SELECT count(*) FROM resource_bindings WHERE tenant_id=10000 AND owner_id='doc-source'`: 0,
`SELECT count(*) FROM resources WHERE tenant_id=10000`: 1,
} {
var got int64
if err := pool.QueryRow(ctx, query).Scan(&got); err != nil {
t.Fatal(err)
}
if got != want {
t.Fatalf("%s = %d, want %d", query, got, want)
}
}
var leftovers int64
if err := pool.QueryRow(ctx, `SELECT count(*) FROM resources WHERE tenant_id=10001`).Scan(&leftovers); err != nil {
t.Fatal(err)
}
if leftovers != 0 {
t.Fatalf("aborted write left %d resource rows in the target tenant", leftovers)
}
var foreign int64
if err := pool.QueryRow(ctx, `SELECT count(*) FROM chunks WHERE tenant_id=20000`).Scan(&foreign); err != nil {
t.Fatal(err)
}
// 夹具本来就有 1 行 other-scope(20000),加上本用例的 chunk-foreign,共 2 行。
if foreign != 2 {
t.Fatalf("cross-tenant rows must survive rollback, got %d", foreign)
}
if err := target.Verify(ctx, libraryrelease.Expectation{SnapshotID: outcome.Backup.SnapshotID}); err != nil {
t.Fatalf("rollback must verify against the prebackup: %v", err)
}
// 2) 同样的跨范围同步,资源能写 → 行必须落在目标范围,资源定位符指向目标存储。
store.failPut = false
succeed := libraryrelease.Run(ctx, request, target, backups, nil)
if succeed.State != "succeeded" {
t.Fatalf("second run outcome = %+v", succeed)
}
if err := target.Verify(ctx, libraryrelease.Expectation{SnapshotID: snapshotID}); err != nil {
t.Fatalf("verify after cross-scope sync: %v", err)
}
var knowledgeKB, chunkKB, provider, locator string
var tenant int64
if err := pool.QueryRow(ctx, `SELECT tenant_id, knowledge_base_id FROM knowledges WHERE id='doc-source'`).
Scan(&tenant, &knowledgeKB); err != nil {
t.Fatalf("source document must exist in the target scope: %v", err)
}
if tenant != 10001 || knowledgeKB != "kb-target" {
t.Fatalf("cross-scope row landed at %d/%s", tenant, knowledgeKB)
}
if err := pool.QueryRow(ctx, `SELECT knowledge_base_id FROM chunks WHERE id='chunk-source'`).Scan(&chunkKB); err != nil {
t.Fatal(err)
}
if chunkKB != "kb-target" {
t.Fatalf("chunk landed in %s", chunkKB)
}
if err := pool.QueryRow(ctx, `SELECT provider, physical_path FROM resources WHERE id='res-source'`).
Scan(&provider, &locator); err != nil {
t.Fatal(err)
}
if provider != "minio" {
t.Fatalf("resource provider = %s, want minio", provider)
}
if !strings.Contains(locator, "storage://backend-target/minio://kg-test-bucket/10001/") {
t.Fatalf("resource locator was not retargeted: %s", locator)
}
if strings.Contains(locator, "weknora/10000") {
t.Fatalf("resource locator still points at the source tenant: %s", locator)
}
t.Logf("cross_scope_sync verified: snapshot=%s backup=%s locator=%s", snapshotID, succeed.Backup.SnapshotID, locator)
}
// sealCrossScopeFixture 造一份"来源范围与目标不同"的整库快照:文档/分块/资源/绑定/
// Wiki/索引都带来源身份,资源部件是可核对的真实字节。
func sealCrossScopeFixture(t *testing.T, archive *librarysnapshot.Archive, sourceTenant uint64, sourceKB string) string {
t.Helper()
encode := func(value any) []byte {
encoded, err := json.Marshal(value)
if err != nil {
t.Fatal(err)
}
return encoded
}
const handle = "abcdefghijklmnopqrstuv"
objectBytes := []byte("12345678")
parts := make([]librarysnapshot.Part, 0, 12)
put := func(kind, name string, count int64, body []byte) {
part, err := archive.Put(bytes.NewReader(body))
if err != nil {
t.Fatal(err)
}
part.Kind, part.Name, part.Count = kind, name, count
parts = append(parts, part)
}
put("database", "knowledges", 1, encode([]map[string]any{{
"id": "doc-source", "tenant_id": sourceTenant, "knowledge_base_id": sourceKB,
"parse_status": "completed", "pending_subtasks_count": 0, "file_path": ""}}))
put("database", "chunks", 1, encode([]map[string]any{{
"id": "chunk-source", "tenant_id": sourceTenant, "knowledge_base_id": sourceKB,
"knowledge_id": "doc-source", "content": "见 resource://" + handle}}))
put("database", "knowledge_tags", 0, encode([]map[string]any{}))
put("database", "knowledge_tag_relations", 0, encode([]map[string]any{}))
put("database", "resources", 1, encode([]map[string]any{{
"id": "res-source", "handle": handle, "tenant_id": sourceTenant,
"storage_backend_id": "backend-source", "provider": "oss",
"physical_path": "storage://backend-source/oss://source-bucket/weknora/" +
formatTenant(sourceTenant) + "/res-source/a.png",
"size": int64(len(objectBytes))}}))
put("database", "resource_bindings", 1, encode([]map[string]any{{
"id": "bind-source", "resource_id": "res-source", "tenant_id": sourceTenant,
"owner_type": "knowledge", "owner_id": "doc-source", "relation": "attachment"}}))
put("resource", "res-source", 1, objectBytes)
put("wiki", "wiki_folders", 0, encode([]map[string]any{}))
put("wiki", "wiki_pages", 1, encode([]map[string]any{{
"id": "wiki-source", "tenant_id": sourceTenant, "knowledge_base_id": sourceKB,
"slug": "index", "content": "wiki"}}))
for _, name := range []string{"wiki_page_issues", "wiki_log_entries"} {
put("wiki", name, 0, encode([]map[string]any{}))
}
put("index", "chunks", 1, encode([]map[string]any{{"id": "chunk-source", "source": map[string]any{
"tenant_id": sourceTenant, "knowledge_base_id": sourceKB, "knowledge_id": "doc-source",
"chunk_id": "chunk-source", "source_id": "chunk-source", "content": "见 resource://" + handle,
"embedding": []float64{0.1, 0.2}}}}))
manifest := librarysnapshot.Manifest{ValidationPolicy: librarysnapshot.ArtifactAcceptancePolicy,
Format: librarysnapshot.Format, Scope: "internal", TenantID: sourceTenant, KnowledgeBaseID: sourceKB,
CreatedAt: time.Now().UTC(), Parts: parts,
Compatibility: librarysnapshot.Compatibility{
WeKnoraVersion: "v0.7.1", SchemaVersion: 74, RetrieverEngine: "elasticsearch_v8",
EmbeddingModel: "text-embedding-v4", EmbeddingDimension: 1024,
IndexMappingSHA256: strings.Repeat("a", 64),
}}
id, err := archive.Seal(manifest)
if err != nil {
t.Fatalf("seal cross-scope fixture: %v", err)
}
return id
}
func formatTenant(tenant uint64) string { return strconv.FormatUint(tenant, 10) }
@@ -0,0 +1,334 @@
package v071
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"knowledge-governance/internal/librarysnapshot"
)
// ResidueReport 是"清理范围外残留"的审计记录:清掉了什么、依据是哪份归档、
// 备份写到哪儿、清理后目标处于什么状态。没有这份记录就不许动目标。
type ResidueReport struct {
SnapshotID string `json:"snapshot_id"`
SourceTenant uint64 `json:"source_tenant"`
SourceKB string `json:"source_kb"`
Tables map[string]int `json:"tables"`
BackupPath string `json:"backup_path"`
BackupSHA256 string `json:"backup_sha256"`
Applied bool `json:"applied"`
Cleared int `json:"cleared"`
Verified bool `json:"verified"`
Foreign map[string]string `json:"foreign_scopes,omitempty"`
}
// ClearForeignScope 清掉"归档里确实有、却仍以来源身份躺在目标里"的行。
//
// 它只做这一件事,而且必须全部条件成立才允许删:
// 1. 目标这些行的内容与指定归档行逐字段一致(先证明它们是这次写入的副本,
// 不是目标自己的数据);
// 2. 先把它们完整导出成 0600 的备份文件并记下摘要;
// 3. 删除只按"归档身份集合 + 来源租户"精确命中,不做范围广删。
//
// 清完还要复核:来源身份不再存在、目标内容与预备份一致、整库指纹里没有别的租户。
func (t *ProductionLibraryTarget) ClearForeignScope(ctx context.Context, source ProductionScope, snapshotID,
backupDir string, apply bool) (*ResidueReport, error) {
if source.TenantID == 0 || !validSearchIdentifier(source.KnowledgeBaseID) {
return nil, errors.New("来源范围不完整,拒绝清理")
}
if source.TenantID == t.scope.TenantID && source.KnowledgeBaseID == t.scope.KnowledgeBaseID {
return nil, errors.New("来源与目标范围相同,不需要清理")
}
if backupDir == "" {
return nil, errors.New("清理前必须指定备份目录")
}
if err := t.resolveStorageBackend(ctx); err != nil {
return nil, err
}
archive, err := librarysnapshot.Open(t.config.ArchiveRoot, 256*1024*1024)
if err != nil {
return nil, errors.New("归档不可用")
}
defer archive.Close()
manifest, err := archive.Load(snapshotID)
if err != nil {
return nil, errors.New("指定快照不可读")
}
report := &ResidueReport{SnapshotID: snapshotID, SourceTenant: source.TenantID, SourceKB: source.KnowledgeBaseID,
Tables: map[string]int{}, Applied: false}
backup := map[string][]map[string]json.RawMessage{}
digests := map[string]string{}
identities := map[string][]string{}
for _, part := range manifest.Parts {
if part.Kind != "database" && part.Kind != "wiki" {
continue
}
archived, archivedDigests, err := archivedRowsWithDigests(archive, part)
if err != nil {
return nil, err
}
// 事故里的残留行是"归档行 + 只把桶/后端换成目标"的旧变换结果:判等时必须
// 同时接受原样与这一种变换,否则会把真残留当成"不是这次写的"而拒绝清理。
if part.Name == "resources" {
for _, row := range archived {
legacy, legacyErr := t.legacyResourceRow(row)
if legacyErr != nil {
return nil, legacyErr
}
digest, digestErr := canonicalRowDigest(legacy)
if digestErr != nil {
return nil, digestErr
}
archivedDigests[digest] = true
}
}
if len(archived) == 0 {
continue
}
live, err := t.foreignScopeRows(ctx, part.Name, source)
if err != nil {
return nil, err
}
liveDigests := make(map[string]string, len(live))
for _, row := range live {
digest, err := canonicalRowDigest(row)
if err != nil {
return nil, err
}
liveDigests[digest] = archiveRowIdentity(part.Name, row)
}
// 目标里这一范围的行必须都能在归档里找到同样内容:否则它们不是这次写入的
// 副本,删了就是删别人的数据。
rows := make([]map[string]json.RawMessage, 0, len(live))
ids := make([]string, 0, len(live))
for digest, identity := range liveDigests {
if _, ok := archivedDigests[digest]; !ok {
return nil, fmt.Errorf("目标 %s 的行 %s 与指定归档不一致,拒绝按本次写入清理", part.Name, identity)
}
ids = append(ids, identity)
}
for _, row := range live {
rows = append(rows, row)
}
sort.Strings(ids)
report.Tables[part.Name] = len(rows)
backup[part.Name] = rows
identities[part.Name] = ids
digests[part.Name] = combineRowDigests(sortedKeys(liveDigests))
}
total := 0
for _, count := range report.Tables {
total += count
}
if total == 0 {
report.Verified = true
return report, nil
}
if err := os.MkdirAll(backupDir, 0o700); err != nil {
return nil, fmt.Errorf("创建备份目录失败: %w", err)
}
payload, err := json.MarshalIndent(map[string]any{
"snapshot_id": snapshotID, "source_tenant": source.TenantID, "source_kb": source.KnowledgeBaseID,
"target_tenant": t.scope.TenantID, "target_kb": t.scope.KnowledgeBaseID,
"tables": report.Tables, "rows": backup, "digests": digests,
}, "", " ")
if err != nil {
return nil, err
}
report.BackupPath = filepath.Join(backupDir, "foreign-scope-residue-"+snapshotID[:12]+".json")
if err := os.WriteFile(report.BackupPath, payload, 0o600); err != nil {
return nil, fmt.Errorf("写残留备份失败: %w", err)
}
sum := sha256.Sum256(payload)
report.BackupSHA256 = hex.EncodeToString(sum[:])
if !apply {
return report, nil
}
// 删除顺序:子表先、父表后(resource_bindings → resources,chunks → knowledges)。
for index := len(productionLibraryTables) - 1; index >= 0; index-- {
table := productionLibraryTables[index]
ids, ok := identities[table]
if !ok || len(ids) == 0 {
continue
}
statement := fmt.Sprintf(`DELETE FROM %s WHERE id = ANY($1::text[])`, table)
if !tableHasTenant(table) {
// knowledge_tag_relations 没有 id:身份就是父文档 + 标签。
statement = `DELETE FROM knowledge_tag_relations WHERE (knowledge_id||':'||tag_id) = ANY($1::text[])`
}
if _, err := t.pool.Exec(ctx, statement, ids); err != nil {
return nil, fmt.Errorf("清理 %s 失败: %w", table, err)
}
}
report.Applied = true
report.Cleared = total
return report, nil
}
// legacyResourceRow 复现旧实现写资源行时的变换:只换定位符里的存储后端与桶,
// 租户/知识库/provider 原样保留。判等要认这一种形状,事故残留才是"可证明"的。
func (t *ProductionLibraryTarget) legacyResourceRow(row map[string]json.RawMessage) (map[string]json.RawMessage, error) {
copied := make(map[string]json.RawMessage, len(row))
for key, value := range row {
copied[key] = value
}
if t.config.ObjectsBucket == "" {
return copied, nil
}
raw, ok := copied["physical_path"]
if !ok {
return nil, errors.New("资源行缺少定位符")
}
var locator string
if err := json.Unmarshal(raw, &locator); err != nil {
return nil, errors.New("资源定位符不可解析")
}
if rewritten, ok := rewriteLocatorTarget(locator, t.storageBackend, t.config.ObjectsBucket); ok {
encoded, err := json.Marshal(rewritten)
if err != nil {
return nil, err
}
copied["physical_path"] = encoded
}
if t.storageBackend == "" {
return copied, nil
}
if raw, ok := copied["storage_backend_id"]; ok {
var current string
if json.Unmarshal(raw, &current) == nil && current != "" && current != t.storageBackend {
encoded, err := json.Marshal(t.storageBackend)
if err != nil {
return nil, err
}
copied["storage_backend_id"] = encoded
}
}
return copied, nil
}
// VerifyResidueCleared 复核清理结果:来源身份不再存在、其它租户的解释得清、
// 且(可选)目标内容与预备份完全一致。
func (t *ProductionLibraryTarget) VerifyResidueCleared(ctx context.Context, source ProductionScope) error {
for _, table := range productionLibraryTables {
if !tableHasTenant(table) {
continue
}
statement := fmt.Sprintf(`SELECT count(*) FROM %s WHERE tenant_id=$1`, table)
arguments := []any{source.TenantID}
if tablesWithKB[table] {
statement += " AND knowledge_base_id=$2"
arguments = append(arguments, source.KnowledgeBaseID)
}
var count int64
if err := t.pool.QueryRow(ctx, statement, arguments...).Scan(&count); err != nil {
return fmt.Errorf("复核 %s 失败: %w", table, err)
}
if count > 0 {
return fmt.Errorf("目标 %s 仍有 %d 行属于来源范围 %d/%s", table, count, source.TenantID, source.KnowledgeBaseID)
}
}
return nil
}
// ForeignScopes 列出目标里所有"不是目标租户"的租户,用于人工确认没有别的泄漏。
func (t *ProductionLibraryTarget) ForeignScopes(ctx context.Context) (map[string]string, error) {
foreign := map[string]string{}
for _, table := range productionLibraryTables {
if !tableHasTenant(table) {
continue
}
rows, err := t.pool.Query(ctx, fmt.Sprintf(
`SELECT tenant_id, count(*) FROM %s WHERE tenant_id <> $1 GROUP BY 1`, table), t.scope.TenantID)
if err != nil {
return nil, fmt.Errorf("读取 %s 租户分布失败: %w", table, err)
}
for rows.Next() {
var tenant, count int64
if err := rows.Scan(&tenant, &count); err != nil {
rows.Close()
return nil, err
}
foreign[fmt.Sprintf("%s:%d", table, tenant)] = fmt.Sprintf("%d 行", count)
}
rows.Close()
if err := rows.Err(); err != nil {
return nil, err
}
}
return foreign, nil
}
// foreignScopeRows 读出目标里属于来源范围的行(resources / resource_bindings
// 只有租户列,按租户取;其余按租户+知识库取)。
func (t *ProductionLibraryTarget) foreignScopeRows(ctx context.Context, table string, source ProductionScope) (
[]map[string]json.RawMessage, error) {
if !tableHasTenant(table) {
return nil, nil
}
statement := fmt.Sprintf(`SELECT to_jsonb(%s) FROM %s WHERE %s`, table, table, scopeWhere(table))
arguments := []any{source.TenantID, source.KnowledgeBaseID}
if table == "resources" || table == "resource_bindings" {
statement = fmt.Sprintf(`SELECT to_jsonb(%s) FROM %s WHERE tenant_id=$1`, table, table)
arguments = arguments[:1]
}
rows, err := t.pool.Query(ctx, statement, arguments...)
if err != nil {
return nil, fmt.Errorf("读取目标 %s 来源范围行失败: %w", table, err)
}
defer rows.Close()
result := make([]map[string]json.RawMessage, 0, 8)
for rows.Next() {
var row map[string]json.RawMessage
if err := rows.Scan(&row); err != nil {
return nil, err
}
result = append(result, row)
}
return result, rows.Err()
}
// archivedRowsWithDigests 读出一份归档部件的行与规范化内容摘要。
func archivedRowsWithDigests(archive *librarysnapshot.Archive, part librarysnapshot.Part) (
[]map[string]json.RawMessage, map[string]bool, error) {
reader, err := archive.OpenPart(part)
if err != nil {
return nil, nil, err
}
defer reader.Close()
decoder := json.NewDecoder(reader)
token, err := decoder.Token()
if err != nil || token != json.Delim('[') {
return nil, nil, errors.New("归档部件不是行数组")
}
rows := make([]map[string]json.RawMessage, 0, 64)
digests := map[string]bool{}
for decoder.More() {
var row map[string]json.RawMessage
if err := decoder.Decode(&row); err != nil {
return nil, nil, err
}
digest, err := canonicalRowDigest(row)
if err != nil {
return nil, nil, err
}
digests[digest] = true
rows = append(rows, row)
}
return rows, digests, nil
}
func sortedKeys(values map[string]string) []string {
keys := make([]string, 0, len(values))
for key := range values {
keys = append(keys, key)
}
sort.Strings(keys)
return keys
}
@@ -0,0 +1,262 @@
package v071
import (
"context"
"encoding/json"
"errors"
"fmt"
"strconv"
"strings"
)
// 整库同步是"把来源知识库的内容搬到目标的租户/知识库",但归档行带着来源身份
// (tenant_id / knowledge_base_id)。原样插入会让行落在来源范围里:目标范围的
// 删除、校验、预备份与恢复全都看不见它们(真实事故:同步失败后"已恢复到预备份",
// 目标却多出 4 文件 / 38 分块 / 21 资源 / 4 绑定,行所属租户还是来源租户)。
//
// 因此写入与校验必须共用这一个改写入口,并且对不上就失败——宁可不写,也不写
// 一行目标范围外的数据。
// tablesWithKB 是真实 WeKnora v0.7.1 里带 knowledge_base_id 的知识域表。
// resources / resource_bindings 是多知识库共享表,只有 tenant_id;
// knowledge_tag_relations 两列都没有,归属完全由父表 knowledges 决定。
var tablesWithKB = map[string]bool{
"knowledges": true, "chunks": true, "knowledge_tags": true,
"wiki_folders": true, "wiki_pages": true, "wiki_page_issues": true, "wiki_log_entries": true,
}
// tableHasTenant 报告这张表是否自带租户列。
func tableHasTenant(table string) bool { return table != "knowledge_tag_relations" }
// bindSnapshotScope 记下快照的来源身份。没有来源身份就无法判断"这行要搬到哪个
// 范围",宁可失败也不要按行内容猜。
func (t *ProductionLibraryTarget) bindSnapshotScope(manifest manifestScope) error {
if manifest.tenantID == 0 || !validSearchIdentifier(manifest.knowledgeBaseID) {
return errors.New("快照缺少来源范围身份(tenant_id / knowledge_base_id),无法确定目标范围映射")
}
t.snapshotTenant = manifest.tenantID
t.snapshotKB = manifest.knowledgeBaseID
return nil
}
// manifestScope 只是把快照的两个范围字段抽出来,避免调用点重复解构。
type manifestScope struct {
tenantID uint64
knowledgeBaseID string
}
// retargetRow 把一行归档数据改写成目标范围的数据:tenant_id / knowledge_base_id
// 换成目标身份,资源行还要把定位符、存储后端、提供方换成目标实例自己的值。
func (t *ProductionLibraryTarget) retargetRow(table string, row map[string]json.RawMessage) error {
if !tableHasTenant(table) {
return nil
}
if err := t.retargetTenantField(row); err != nil {
return fmt.Errorf("%s:%w", table, err)
}
if tablesWithKB[table] {
if err := t.retargetKnowledgeBaseField(row); err != nil {
return fmt.Errorf("%s:%w", table, err)
}
}
if table == "resources" {
return t.rewriteResourceIdentity(row)
}
if table == "knowledges" {
return t.retargetEmbeddingModelField(row)
}
return nil
}
// retargetEmbeddingModelField 把文档的 embedding 模型换成目标知识库绑定的那一个。
// 来源模型 id 在目标实例里通常不存在(目标 models 表可能是空的),留着它等于让
// 应用拿一个解析不到的模型去查——库里的向量与查询向量就不是同一套了。
func (t *ProductionLibraryTarget) retargetEmbeddingModelField(row map[string]json.RawMessage) error {
if t.targetEmbeddingModel == "" {
return nil
}
if _, ok := row["embedding_model_id"]; !ok {
return nil
}
encoded, err := json.Marshal(t.targetEmbeddingModel)
if err != nil {
return errors.New("编码目标 embedding 模型失败")
}
row["embedding_model_id"] = encoded
return nil
}
// resolveEmbeddingModel 读目标知识库自己的 embedding 模型 id。读不到不算错误
// (目标可能确实没绑模型),此时保持来源值不动,由契约校验去发现不一致。
func (t *ProductionLibraryTarget) resolveEmbeddingModel(ctx context.Context) {
var model string
if err := t.pool.QueryRow(ctx,
`SELECT coalesce(embedding_model_id,'') FROM knowledge_bases WHERE id=$1 AND tenant_id=$2`,
t.scope.KnowledgeBaseID, t.scope.TenantID).Scan(&model); err == nil {
t.targetEmbeddingModel = model
}
}
func (t *ProductionLibraryTarget) retargetTenantField(row map[string]json.RawMessage) error {
raw, ok := row["tenant_id"]
if !ok {
return errors.New("行缺少 tenant_id,无法确定要写入哪个范围")
}
var current uint64
if err := json.Unmarshal(raw, &current); err != nil {
return errors.New("行 tenant_id 不可解析")
}
if t.snapshotTenant == 0 {
return errors.New("快照来源租户未知,拒绝猜测目标租户")
}
if current != t.snapshotTenant {
return fmt.Errorf("行 tenant_id=%d 不属于快照来源租户 %d", current, t.snapshotTenant)
}
encoded, err := json.Marshal(t.scope.TenantID)
if err != nil {
return errors.New("编码目标租户失败")
}
row["tenant_id"] = encoded
return nil
}
func (t *ProductionLibraryTarget) retargetKnowledgeBaseField(row map[string]json.RawMessage) error {
raw, ok := row["knowledge_base_id"]
if !ok {
return errors.New("行缺少 knowledge_base_id,无法确定要写入哪个知识库")
}
var current string
if err := json.Unmarshal(raw, &current); err != nil || current == "" {
return errors.New("行 knowledge_base_id 不可解析")
}
if t.snapshotKB == "" {
return errors.New("快照来源知识库未知,拒绝猜测目标知识库")
}
if current != t.snapshotKB {
return fmt.Errorf("行 knowledge_base_id=%s 不属于快照来源知识库 %s", current, t.snapshotKB)
}
encoded, err := json.Marshal(t.scope.KnowledgeBaseID)
if err != nil {
return errors.New("编码目标知识库失败")
}
row["knowledge_base_id"] = encoded
return nil
}
// rewriteResourceIdentity 把资源行的定位符与提供方换成目标实例的值。
//
// 只换桶不够:来源行的 scheme(oss/minio…)与对象键里的租户段也是来源实例的,
// 目标实例按它取文件会落到别的提供方或别的租户前缀上(真跑:storage backend
// not found → invalid path → 500 code1007)。
func (t *ProductionLibraryTarget) rewriteResourceIdentity(row map[string]json.RawMessage) error {
raw, ok := row["physical_path"]
if !ok {
return errors.New("资源行缺少定位符")
}
var locator string
if err := json.Unmarshal(raw, &locator); err != nil {
return errors.New("资源定位符不可解析")
}
rewritten, err := retargetLocator(locator, t.storageBackend, t.storageProvider, t.config.ObjectsBucket,
t.snapshotTenant, t.scope.TenantID)
if err != nil {
return err
}
if rewritten != locator {
encoded, err := json.Marshal(rewritten)
if err != nil {
return errors.New("编码目标定位符失败")
}
row["physical_path"] = encoded
}
if t.storageBackend != "" {
if _, ok := row["storage_backend_id"]; ok {
encoded, err := json.Marshal(t.storageBackend)
if err != nil {
return errors.New("编码目标存储后端失败")
}
row["storage_backend_id"] = encoded
}
}
if t.storageProvider != "" {
if _, ok := row["provider"]; ok {
encoded, err := json.Marshal(t.storageProvider)
if err != nil {
return errors.New("编码目标提供方失败")
}
row["provider"] = encoded
}
}
return nil
}
// retargetLocator 把 "storage://<backend>/<scheme>://<bucket>/<key>" 整体换成目标
// 实例的真实取值:后端 id、scheme/提供方、桶名,以及对象键里的租户段。
// 桶名为空表示目标没有配置对象存储(测试夹具),此时不动定位符。
func retargetLocator(locator, backendID, provider, bucket string, sourceTenant, targetTenant uint64) (string, error) {
if bucket == "" || locator == "" {
return locator, nil
}
if locator != strings.TrimSpace(locator) {
return "", errors.New("资源定位符带首尾空白")
}
prefix := ""
rest := locator
if strings.HasPrefix(locator, "storage://") {
trimmed := strings.TrimPrefix(locator, "storage://")
parts := strings.SplitN(trimmed, "/", 2)
if len(parts) != 2 || !validSearchIdentifier(parts[0]) || parts[1] == "" {
return "", errors.New("资源定位符结构不可解析")
}
if backendID != "" {
if !validSearchIdentifier(backendID) {
return "", errors.New("目标存储后端 id 形状不合法")
}
parts[0] = backendID
}
prefix, rest = "storage://"+parts[0]+"/", parts[1]
}
separator := strings.Index(rest, "://")
if separator < 0 {
// 裸对象键没有桶与租户段:落盘按配置桶,改写无意义。
return locator, nil
}
scheme := rest[:separator]
if !validSearchIdentifier(scheme) {
return "", errors.New("资源定位符的提供方不可解析")
}
schemeless := rest[separator+3:]
boundary := strings.Index(schemeless, "/")
if boundary <= 0 || boundary == len(schemeless)-1 {
return "", errors.New("资源定位符缺少对象键")
}
key := schemeless[boundary+1:]
if provider != "" {
if !validSearchIdentifier(provider) {
return "", errors.New("目标提供方形状不合法")
}
scheme = provider
}
return prefix + scheme + "://" + bucket + "/" + retargetKeyTenant(key, sourceTenant, targetTenant), nil
}
// retargetKeyTenant 把对象键里的来源租户段换成目标租户,并丢掉来源实例自己的
// 路径前缀(例如 MINIO_PATH_PREFIX=weknora/),让键与目标实例的写法一致。
// 找不到来源租户段时原样返回:此时键本身不含跨租户引用。
func retargetKeyTenant(key string, sourceTenant, targetTenant uint64) string {
if sourceTenant == 0 || targetTenant == 0 || sourceTenant == targetTenant {
return key
}
source, target := strconv.FormatUint(sourceTenant, 10), strconv.FormatUint(targetTenant, 10)
segments := strings.Split(key, "/")
for index := 0; index < len(segments) && index < 2; index++ {
// 只有"后面还有内容"的段才是路径前缀;整键就是租户号时不该动它。
if index >= len(segments)-1 || segments[index] != source {
continue
}
kept := segments[index:]
kept[0] = target
return strings.Join(kept, "/")
}
return key
}
@@ -0,0 +1,194 @@
package v071
import (
"encoding/json"
"strings"
"testing"
)
// 真实事故的形状:来源 KB 与目标 KB 不同(租户也不同),归档行原样写进目标,
// 目标范围的删除/校验/恢复都看不到它们。这里把"必须改写哪些列"钉死。
func scopeTargetFixture() *ProductionLibraryTarget {
return &ProductionLibraryTarget{
scope: ProductionScope{TenantID: 10001, KnowledgeBaseID: "kb-target"},
config: ProductionLibraryConfig{ObjectsBucket: "kgseal-1029a"},
storageBackend: "backend-target",
storageProvider: "minio",
snapshotTenant: 10000,
snapshotKB: "kb-source",
}
}
func rawRow(t *testing.T, fields map[string]any) map[string]json.RawMessage {
t.Helper()
row := make(map[string]json.RawMessage, len(fields))
for key, value := range fields {
encoded, err := json.Marshal(value)
if err != nil {
t.Fatalf("encode %s: %v", key, err)
}
row[key] = encoded
}
return row
}
func fieldText(t *testing.T, row map[string]json.RawMessage, key string) string {
t.Helper()
var value string
if err := json.Unmarshal(row[key], &value); err != nil {
t.Fatalf("%s not a string: %v", key, err)
}
return value
}
func fieldUint(t *testing.T, row map[string]json.RawMessage, key string) uint64 {
t.Helper()
var value uint64
if err := json.Unmarshal(row[key], &value); err != nil {
t.Fatalf("%s not a number: %v", key, err)
}
return value
}
func TestRetargetRowMapsSourceScopeToTarget(t *testing.T) {
target := scopeTargetFixture()
for _, table := range []string{"knowledges", "chunks", "knowledge_tags", "wiki_pages", "wiki_folders"} {
row := rawRow(t, map[string]any{"id": "row-1", "tenant_id": 10000, "knowledge_base_id": "kb-source"})
if err := target.retargetRow(table, row); err != nil {
t.Fatalf("%s retarget: %v", table, err)
}
if got := fieldUint(t, row, "tenant_id"); got != 10001 {
t.Fatalf("%s tenant_id = %d, want 10001", table, got)
}
if got := fieldText(t, row, "knowledge_base_id"); got != "kb-target" {
t.Fatalf("%s knowledge_base_id = %s, want kb-target", table, got)
}
}
// 共享表只有租户列:不得凭空写一个 knowledge_base_id 出来。
binding := rawRow(t, map[string]any{"id": "bind-1", "tenant_id": 10000, "owner_type": "knowledge", "owner_id": "row-1"})
if err := target.retargetRow("resource_bindings", binding); err != nil {
t.Fatalf("resource_bindings retarget: %v", err)
}
if got := fieldUint(t, binding, "tenant_id"); got != 10001 {
t.Fatalf("binding tenant_id = %d, want 10001", got)
}
if _, ok := binding["knowledge_base_id"]; ok {
t.Fatal("resource_bindings must not gain a knowledge_base_id column")
}
// knowledge_tag_relations 没有范围列:归属走父表,改写必须是空操作。
relation := rawRow(t, map[string]any{"knowledge_id": "row-1", "tag_id": "tag-1"})
if err := target.retargetRow("knowledge_tag_relations", relation); err != nil {
t.Fatalf("knowledge_tag_relations retarget: %v", err)
}
if len(relation) != 2 {
t.Fatalf("knowledge_tag_relations must stay untouched, got %v", relation)
}
}
// 文档行的 embedding 模型必须换成目标知识库绑定的那一个:来源模型 id 在目标
// 实例里通常不存在,留着它检索时的查询向量与库里向量就不是同一套。
func TestRetargetRowUsesTargetEmbeddingModel(t *testing.T) {
target := scopeTargetFixture()
target.targetEmbeddingModel = "model-target"
row := rawRow(t, map[string]any{"id": "doc-1", "tenant_id": 10000, "knowledge_base_id": "kb-source",
"embedding_model_id": "model-source"})
if err := target.retargetRow("knowledges", row); err != nil {
t.Fatalf("retarget: %v", err)
}
if got := fieldText(t, row, "embedding_model_id"); got != "model-target" {
t.Fatalf("embedding_model_id = %s, want model-target", got)
}
// 目标没绑模型时不猜:保持来源值,交给契约校验去发现不一致。
plain := scopeTargetFixture()
row = rawRow(t, map[string]any{"id": "doc-1", "tenant_id": 10000, "knowledge_base_id": "kb-source",
"embedding_model_id": "model-source"})
if err := plain.retargetRow("knowledges", row); err != nil {
t.Fatalf("retarget without target model: %v", err)
}
if got := fieldText(t, row, "embedding_model_id"); got != "model-source" {
t.Fatalf("embedding_model_id = %s, want unchanged", got)
}
}
func TestRetargetRowFailsClosedOutsideSnapshotScope(t *testing.T) {
target := scopeTargetFixture()
cases := map[string]map[string]any{
"foreign tenant": {"id": "x", "tenant_id": 20000, "knowledge_base_id": "kb-source"},
"foreign knowledge": {"id": "x", "tenant_id": 10000, "knowledge_base_id": "kb-other"},
"missing tenant": {"id": "x", "knowledge_base_id": "kb-source"},
"missing knowledge kb": {"id": "x", "tenant_id": 10000},
}
for name, fields := range cases {
if err := target.retargetRow("knowledges", rawRow(t, fields)); err == nil {
t.Fatalf("%s: retarget must fail closed", name)
}
}
// 快照来源未知时不许猜目标范围。
unknown := &ProductionLibraryTarget{scope: ProductionScope{TenantID: 10001, KnowledgeBaseID: "kb-target"}}
if err := unknown.retargetRow("knowledges", rawRow(t, map[string]any{"id": "x", "tenant_id": 10000, "knowledge_base_id": "kb"})); err == nil {
t.Fatal("unknown snapshot scope must fail closed")
}
}
func TestRetargetRowRewritesResourceStorageIdentity(t *testing.T) {
target := scopeTargetFixture()
row := rawRow(t, map[string]any{
"id": "res-1",
"tenant_id": 10000,
"storage_backend_id": "backend-source",
"provider": "oss",
"physical_path": "storage://backend-source/oss://yzh-weknora-test/weknora/10000/res-1/a.png",
})
if err := target.retargetRow("resources", row); err != nil {
t.Fatalf("retarget resource: %v", err)
}
if got := fieldUint(t, row, "tenant_id"); got != 10001 {
t.Fatalf("resource tenant = %d, want 10001", got)
}
want := "storage://backend-target/minio://kgseal-1029a/10001/res-1/a.png"
if got := fieldText(t, row, "physical_path"); got != want {
t.Fatalf("physical_path = %q, want %q", got, want)
}
if got := fieldText(t, row, "provider"); got != "minio" {
t.Fatalf("provider = %q, want minio", got)
}
if got := fieldText(t, row, "storage_backend_id"); got != "backend-target" {
t.Fatalf("storage_backend_id = %q, want backend-target", got)
}
}
func TestRetargetKeyTenantDropsSourcePrefix(t *testing.T) {
cases := []struct{ key, want string }{
{"weknora/10000/exports/a.png", "10001/exports/a.png"},
{"10000/knowledge/file.md", "10001/knowledge/file.md"},
{"other/prefix/a.png", "other/prefix/a.png"},
{"10000", "10000"},
}
for _, tc := range cases {
if got := retargetKeyTenant(tc.key, 10000, 10001); got != tc.want {
t.Fatalf("retargetKeyTenant(%q) = %q, want %q", tc.key, got, tc.want)
}
}
// 同租户不编码:键必须原样返回。
if got := retargetKeyTenant("weknora/10000/a.png", 10000, 10000); got != "weknora/10000/a.png" {
t.Fatalf("same tenant must keep key, got %q", got)
}
}
func TestRetargetLocatorRejectsMalformed(t *testing.T) {
if _, err := retargetLocator(" not-a-locator ", "backend", "minio", "bucket", 10000, 10001); err == nil {
t.Fatal("locator with surrounding spaces must be rejected")
}
if _, err := retargetLocator("storage://backend/oss://", "backend", "minio", "bucket", 10000, 10001); err == nil {
t.Fatal("locator without object key must be rejected")
}
// 没有目标桶(测试夹具)时不改写,保持原值可继续用于采集/校验。
locator := "storage://backend/oss://bucket/key"
if got, err := retargetLocator(locator, "backend", "minio", "", 10000, 10001); err != nil || got != locator {
t.Fatalf("empty bucket must be a no-op, got %q err=%v", got, err)
}
if !strings.Contains(retargetKeyTenant("weknora/10000/a", 10000, 10001), "10001") {
t.Fatal("tenant segment must be replaced")
}
}
@@ -0,0 +1,126 @@
package v071
import (
"context"
"encoding/json"
"errors"
"fmt"
"sort"
"github.com/jackc/pgx/v5/pgxpool"
"knowledge-governance/internal/libraryrelease"
"knowledge-governance/internal/librarysnapshot"
)
// universeFingerprint 是"整实例"的知识域指纹:不限租户/知识库的行数 + 身份集合摘要。
//
// 为什么不能只比目标范围:恢复只按目标范围删除/回放,写到范围外的残留它看不见
// (真实事故:作业报"已恢复到预备份",目标却多出 4 文件 / 38 分块 / 21 资源 /
// 4 绑定)。整实例指纹让"多写了什么"必然暴露,同时也能证明恢复没有误删别的
// 知识库的行。
func universeFingerprint(ctx context.Context, pool *pgxpool.Pool) (map[string]libraryrelease.TableFingerprint, error) {
result := make(map[string]libraryrelease.TableFingerprint, len(productionLibraryTables))
for _, table := range productionLibraryTables {
identity := identityExpr(table)
statement := fmt.Sprintf(
`SELECT count(*), coalesce(md5(string_agg((%s)::text, E'\n' ORDER BY (%s)::text)), '') FROM %s`,
identity, identity, table)
var fingerprint libraryrelease.TableFingerprint
if err := pool.QueryRow(ctx, statement).Scan(&fingerprint.Count, &fingerprint.Digest); err != nil {
return nil, fmt.Errorf("读取整库 %s 指纹失败: %w", table, err)
}
result[table] = fingerprint
}
return result, nil
}
// archivedRowIdentities 读出一份归档部件的行身份(与校验用的身份口径一致)。
func archivedRowIdentities(archive *librarysnapshot.Archive, part librarysnapshot.Part) ([]string, error) {
reader, err := archive.OpenPart(part)
if err != nil {
return nil, err
}
defer reader.Close()
decoder := json.NewDecoder(reader)
token, err := decoder.Token()
if err != nil || token != json.Delim('[') {
return nil, errors.New("归档部件不是行数组")
}
ids := make([]string, 0, 64)
for decoder.More() {
var row map[string]json.RawMessage
if err := decoder.Decode(&row); err != nil {
return nil, err
}
identity := archiveRowIdentity(part.Name, row)
if identity == "" {
continue
}
ids = append(ids, identity)
}
return ids, nil
}
// universeDiff 返回第一处不一致的表;空字符串表示整实例与参照完全一致。
func universeDiff(want, got map[string]libraryrelease.TableFingerprint) string {
tables := make([]string, 0, len(want))
for table := range want {
tables = append(tables, table)
}
sort.Strings(tables)
for _, table := range tables {
current, ok := got[table]
if !ok {
return fmt.Sprintf("%s(目标读不到)", table)
}
if current.Count != want[table].Count {
return fmt.Sprintf("%s 行数 %d,参照 %d", table, current.Count, want[table].Count)
}
if current.Digest != want[table].Digest {
return fmt.Sprintf("%s 身份集合与参照不同(同数量改写也会被这条拦住)", table)
}
}
return ""
}
// assertNoSourceScopeRows 找出"这次要写的这些行,仍然以来源身份躺在目标里"。
//
// 判据按归档里的身份集合逐表核对,而不是"这个租户/知识库有没有行":同一个租户
// 里本来就可能住着别的知识库(共享资源、别的 KB 的绑定),按范围数会把它们误
// 报成残留。这里只认"本次写集合里、仍属于来源租户"的行——它们只可能来自逃出
// 目标范围的写入,而恢复只按目标范围回放,永远删不到它们。
func (t *ProductionLibraryTarget) assertNoSourceScopeRows(ctx context.Context, archive *librarysnapshot.Archive,
manifest librarysnapshot.Manifest) error {
if t.snapshotTenant == 0 {
return nil
}
if t.snapshotTenant == t.scope.TenantID && t.snapshotKB == t.scope.KnowledgeBaseID {
return nil
}
for _, part := range manifest.Parts {
if part.Kind != "database" && part.Kind != "wiki" {
continue
}
if !tableHasTenant(part.Name) {
continue
}
ids, err := archivedRowIdentities(archive, part)
if err != nil {
return err
}
if len(ids) == 0 {
continue
}
statement := fmt.Sprintf(`SELECT count(*) FROM %s WHERE tenant_id=$1 AND id = ANY($2::text[])`, part.Name)
var count int64
if err := t.pool.QueryRow(ctx, statement, t.snapshotTenant, ids).Scan(&count); err != nil {
return fmt.Errorf("检查目标 %s 的来源范围残留失败: %w", part.Name, err)
}
if count > 0 {
return fmt.Errorf("目标 %s 残留 %d 行仍属于来源范围 %d/%s:有写入逃出了目标范围,恢复不会清掉它们",
part.Name, count, t.snapshotTenant, t.snapshotKB)
}
}
return nil
}
@@ -60,8 +60,11 @@ func TestRewriteLocatorSwapsStorageBackendWithBucket(t *testing.T) {
// 行变换用解析到的目标后端与目标桶:写入与校验共用这一份,摘要才不会两边打架。
func TestResourceRowTransformRewritesBackendAndBucket(t *testing.T) {
target := &ProductionLibraryTarget{config: ProductionLibraryConfig{ObjectsBucket: "kgseal-1029a"},
storageBackend: "target-backend"}
row := map[string]json.RawMessage{"physical_path": json.RawMessage(`"storage://source-backend/minio://kgmaint-1028a/10001/k/ab.md"`)}
storageBackend: "target-backend", snapshotTenant: 10001, snapshotKB: "kb",
scope: ProductionScope{TenantID: 10001, KnowledgeBaseID: "kb"}}
row := map[string]json.RawMessage{
"tenant_id": json.RawMessage(`10001`),
"physical_path": json.RawMessage(`"storage://source-backend/minio://kgmaint-1028a/10001/k/ab.md"`)}
if err := target.rowTransform("resources")(row); err != nil {
t.Fatalf("row transform failed: %v", err)
}
@@ -78,8 +81,10 @@ func TestResourceRowTransformRewritesBackendAndBucket(t *testing.T) {
// 只改定位符不改这一列,目标仍是 storage backend not found(真跑 500)。
func TestResourceRowTransformRewritesBackendColumn(t *testing.T) {
target := &ProductionLibraryTarget{config: ProductionLibraryConfig{ObjectsBucket: "kgseal-1029a"},
storageBackend: "target-backend"}
storageBackend: "target-backend", snapshotTenant: 10001, snapshotKB: "kb",
scope: ProductionScope{TenantID: 10001, KnowledgeBaseID: "kb"}}
row := map[string]json.RawMessage{
"tenant_id": json.RawMessage(`10001`),
"physical_path": json.RawMessage(`"storage://source-backend/minio://kgmaint-1028a/1/k/ab.md"`),
"storage_backend_id": json.RawMessage(`"source-backend"`),
}
+91
View File
@@ -0,0 +1,91 @@
// Package sealedsecret holds the one AES-GCM convention this platform uses to
// keep a write-only credential (model key, WeKnora key) inside a database row
// without ever storing the plaintext or a reversible copy elsewhere.
//
// Two rules make it safe to have only one implementation:
// - the master key is an independently provisioned 32-byte 0600 file, never
// in source, settings JSON or database backups;
// - every ciphertext is bound to a caller-supplied AAD (tenant + revision),
// so a sealed value cannot be replayed onto a different row or revision.
package sealedsecret
import (
"crypto/aes"
"crypto/cipher"
"crypto/rand"
"errors"
"fmt"
"os"
)
// ErrUnavailable means the master key is not usable, so sealing and opening are
// both impossible. Callers degrade honestly (they must not pretend it works).
var ErrUnavailable = errors.New("sealed secret storage is unavailable")
// KeySize is the only accepted master-key length (AES-256).
const KeySize = 32
// LoadKeyFile reads the master key. A missing file, a non-regular file, group or
// other permissions, or a wrong length are all ErrUnavailable: a half-configured
// secret store must never be silently substituted by a weaker one.
func LoadKeyFile(path string) ([]byte, error) {
if path == "" {
return nil, ErrUnavailable
}
info, err := os.Lstat(path)
if err != nil || !info.Mode().IsRegular() || info.Mode().Perm() != 0o600 {
return nil, ErrUnavailable
}
raw, err := os.ReadFile(path)
if err != nil || len(raw) != KeySize {
return nil, ErrUnavailable
}
return raw, nil
}
func aead(key []byte) (cipher.AEAD, error) {
if len(key) != KeySize {
return nil, ErrUnavailable
}
block, err := aes.NewCipher(key)
if err != nil {
return nil, ErrUnavailable
}
return cipher.NewGCM(block)
}
// Seal returns nonce||ciphertext bound to aad.
func Seal(key []byte, plaintext, aad string) ([]byte, error) {
sealer, err := aead(key)
if err != nil {
return nil, err
}
nonce := make([]byte, sealer.NonceSize())
if _, err = rand.Read(nonce); err != nil {
return nil, ErrUnavailable
}
return sealer.Seal(nonce, nonce, []byte(plaintext), []byte(aad)), nil
}
// Open reverses Seal. A different aad, a truncated value or a rotated key all
// return ErrUnavailable rather than an empty credential.
func Open(key, sealed []byte, aad string) (string, error) {
opener, err := aead(key)
if err != nil {
return "", err
}
if len(sealed) < opener.NonceSize() {
return "", ErrUnavailable
}
raw, err := opener.Open(nil, sealed[:opener.NonceSize()], sealed[opener.NonceSize():], []byte(aad))
if err != nil {
return "", ErrUnavailable
}
return string(raw), nil
}
// AAD builds the canonical associated data for one row revision. Every caller
// goes through it so two features cannot accidentally share a binding domain.
func AAD(scope, subject string, revision int64) string {
return fmt.Sprintf("%s:%s:%d", scope, subject, revision)
}
@@ -0,0 +1,73 @@
package sealedsecret
import (
"bytes"
"os"
"path/filepath"
"testing"
)
func writeKey(t *testing.T, content []byte, mode os.FileMode) string {
t.Helper()
path := filepath.Join(t.TempDir(), "master.key")
if err := os.WriteFile(path, content, mode); err != nil {
t.Fatalf("write key file: %v", err)
}
if err := os.Chmod(path, mode); err != nil {
t.Fatalf("chmod key file: %v", err)
}
return path
}
func TestSealOpenRoundTripBindsAAD(t *testing.T) {
key := bytes.Repeat([]byte{0x7a}, KeySize)
sealed, err := Seal(key, "sk-live-1234", AAD("weknora-connection", "test", 3))
if err != nil {
t.Fatalf("Seal() error = %v", err)
}
if bytes.Contains(sealed, []byte("sk-live-1234")) {
t.Fatal("ciphertext must not contain the plaintext")
}
plain, err := Open(key, sealed, AAD("weknora-connection", "test", 3))
if err != nil || plain != "sk-live-1234" {
t.Fatalf("Open() = %q, %v", plain, err)
}
if _, err := Open(key, sealed, AAD("weknora-connection", "test", 4)); err == nil {
t.Fatal("a different revision must not open the same ciphertext")
}
if _, err := Open(key, sealed, AAD("weknora-connection", "production", 3)); err == nil {
t.Fatal("a different environment must not open the same ciphertext")
}
}
func TestOpenRejectsTruncatedAndRotatedKey(t *testing.T) {
key := bytes.Repeat([]byte{0x11}, KeySize)
sealed, err := Seal(key, "secret", AAD("qa-ai", "tenant", 1))
if err != nil {
t.Fatalf("Seal() error = %v", err)
}
if _, err := Open(key, sealed[:4], AAD("qa-ai", "tenant", 1)); err == nil {
t.Fatal("truncated ciphertext must not open")
}
rotated := bytes.Repeat([]byte{0x22}, KeySize)
if _, err := Open(rotated, sealed, AAD("qa-ai", "tenant", 1)); err == nil {
t.Fatal("a rotated master key must not open old ciphertext")
}
}
func TestLoadKeyFileRequirements(t *testing.T) {
good := writeKey(t, bytes.Repeat([]byte{0x01}, KeySize), 0o600)
if _, err := LoadKeyFile(good); err != nil {
t.Fatalf("LoadKeyFile(0600, 32B) error = %v", err)
}
cases := map[string]string{
"missing": "",
"short": writeKey(t, bytes.Repeat([]byte{0x01}, 16), 0o600),
"group-visible": writeKey(t, bytes.Repeat([]byte{0x01}, KeySize), 0o640),
}
for name, path := range cases {
if _, err := LoadKeyFile(path); err != ErrUnavailable {
t.Fatalf("%s: LoadKeyFile() error = %v, want ErrUnavailable", name, err)
}
}
}
+12 -1
View File
@@ -13,6 +13,17 @@ import (
// the operator to paste an id. A missing endpoint or credential is an error the
// caller reports honestly rather than a silently empty picker.
func (verifier *Verifier) ListProductionKnowledgeBases(ctx context.Context, baseURL string) ([]weknora.KnowledgeBase, error) {
// The admin-managed connection wins: the page is where operators record the
// endpoint and key, and a second (file) source would silently disagree.
if verifier.config.ConnectionResolver != nil {
if endpoint, apiKey, ok := verifier.config.ConnectionResolver.Resolve(ctx, "production"); ok {
client, err := verifier.clientWithPolicy(endpoint, apiKey, verifier.config.ProductionAllowDockerHTTP, verifier.config.ProductionAllowPrivateHTTP)
if err != nil {
return nil, err
}
return client.ListKnowledgeBases(ctx)
}
}
resolved := firstNonEmpty(baseURL, verifier.config.ProductionBaseURLFallback)
if resolved == "" {
return nil, errors.New("production WeKnora endpoint is not configured")
@@ -21,7 +32,7 @@ func (verifier *Verifier) ListProductionKnowledgeBases(ctx context.Context, base
if apiKey == "" {
return nil, errors.New("production WeKnora API key file is not available")
}
client, err := verifier.client(resolved, apiKey, verifier.config.ProductionAllowDockerHTTP)
client, err := verifier.clientWithPolicy(resolved, apiKey, verifier.config.ProductionAllowDockerHTTP, verifier.config.ProductionAllowPrivateHTTP)
if err != nil {
return nil, err
}
+35 -11
View File
@@ -40,8 +40,19 @@ type Config struct {
ProductionBaseURLFallback string
TestAllowDockerHTTP bool
ProductionAllowDockerHTTP bool
Timeout time.Duration
Now func() time.Time
// ProductionAllowPrivateHTTP mirrors WEKNORA_ALLOW_PRIVATE_HTTP: the
// production instance is reachable only as a same-LAN http endpoint.
ProductionAllowPrivateHTTP bool
TestAllowPrivateHTTP bool
// ConnectionResolver reads the admin-managed connection (endpoint + key)
// from the governance database. When it answers, it wins over the env
// defaults and over the credential file — the page is the place operators
// configure these, and a second source would drift.
ConnectionResolver interface {
Resolve(ctx context.Context, environment string) (string, string, bool)
}
Timeout time.Duration
Now func() time.Time
}
// Verifier asks endpoints what they really hold. It is safe for concurrent use.
@@ -75,12 +86,20 @@ func (verifier *Verifier) VerifyServiceTarget(ctx context.Context, target govern
testBaseURL := firstNonEmpty(target.TestBaseURL, verifier.config.TestBaseURLFallback)
productionKey := verifier.apiKey(verifier.config.ProductionAPIKeyFile)
testKey := verifier.apiKey(verifier.config.TestAPIKeyFile)
if verifier.config.ConnectionResolver != nil {
if endpoint, key, ok := verifier.config.ConnectionResolver.Resolve(ctx, "test"); ok {
testBaseURL, testKey = endpoint, key
}
if endpoint, key, ok := verifier.config.ConnectionResolver.Resolve(ctx, "production"); ok {
productionBaseURL, productionKey = endpoint, key
}
}
verification := governance.ServiceTargetVerification{
Status: governance.ServiceTargetNotConfigured,
Message: "未配置:两项 ID 都留空,没有可核验的值",
TestEndpoint: verifier.probeEndpoint(ctx, testBaseURL, testKey, "测试地址", verifier.config.TestAllowDockerHTTP),
ProductionEndpoint: verifier.probeEndpoint(ctx, productionBaseURL, productionKey, "生产地址", verifier.config.ProductionAllowDockerHTTP),
TestEndpoint: verifier.probeEndpointWithPolicy(ctx, testBaseURL, testKey, "测试地址", verifier.config.TestAllowDockerHTTP, verifier.config.TestAllowPrivateHTTP),
ProductionEndpoint: verifier.probeEndpointWithPolicy(ctx, productionBaseURL, productionKey, "生产地址", verifier.config.ProductionAllowDockerHTTP, verifier.config.ProductionAllowPrivateHTTP),
CheckedAt: verifier.config.Now().UTC(),
}
if target.ProductionKnowledgeBaseID == "" && target.ProductionTenantID == "" {
@@ -96,7 +115,7 @@ func (verifier *Verifier) VerifyServiceTarget(ctx context.Context, target govern
verification.Message = "未核验:没有可用的生产 API Key 文件(PRODUCTION_WEKNORA_API_KEY_FILE),保存未被阻断"
return verification
}
client, err := verifier.client(productionBaseURL, productionKey, verifier.config.ProductionAllowDockerHTTP)
client, err := verifier.clientWithPolicy(productionBaseURL, productionKey, verifier.config.ProductionAllowDockerHTTP, verifier.config.ProductionAllowPrivateHTTP)
if err != nil {
verification.Status = governance.ServiceTargetCannotVerify
verification.Message = "未核验:生产地址不能作为 WeKnora 端点(" + err.Error() + "),保存未被阻断"
@@ -198,11 +217,15 @@ func (verifier *Verifier) verifyTenantOnly(ctx context.Context, client *weknora.
// is available, otherwise a plain /health liveness call. Advisory — the caller
// only reports it.
func (verifier *Verifier) probeEndpoint(ctx context.Context, baseURL, apiKey, label string, allowDockerHTTP bool) governance.ServiceEndpointStatus {
return verifier.probeEndpointWithPolicy(ctx, baseURL, apiKey, label, allowDockerHTTP, false)
}
func (verifier *Verifier) probeEndpointWithPolicy(ctx context.Context, baseURL, apiKey, label string, allowDockerHTTP, allowPrivateHTTP bool) governance.ServiceEndpointStatus {
if strings.TrimSpace(baseURL) == "" {
return governance.ServiceEndpointStatus{Status: governance.ServiceEndpointNotConfigured, Message: label + "未配置"}
}
if apiKey != "" {
if client, err := verifier.client(baseURL, apiKey, allowDockerHTTP); err == nil {
if client, err := verifier.clientWithPolicy(baseURL, apiKey, allowDockerHTTP, allowPrivateHTTP); err == nil {
bases, listErr := client.ListKnowledgeBases(ctx)
switch {
case listErr == nil:
@@ -271,12 +294,13 @@ func (verifier *Verifier) candidateSuffix(ctx context.Context, client *weknora.C
return fmt.Sprintf(";该地址现有 %d 个知识库:%s", len(bases), strings.Join(listed, "、"))
}
func (verifier *Verifier) client(baseURL, apiKey string, allowDockerHTTP bool) (*weknora.Client, error) {
func (verifier *Verifier) clientWithPolicy(baseURL, apiKey string, allowDockerHTTP, allowPrivateHTTP bool) (*weknora.Client, error) {
return weknora.NewClient(weknora.ClientConfig{
BaseURL: baseURL,
APIKey: apiKey,
Timeout: verifier.config.Timeout,
AllowDockerHTTP: allowDockerHTTP,
BaseURL: baseURL,
APIKey: apiKey,
Timeout: verifier.config.Timeout,
AllowDockerHTTP: allowDockerHTTP,
AllowPrivateHTTP: allowPrivateHTTP,
})
}
+454
View File
@@ -0,0 +1,454 @@
// Package weknoraconn owns the two WeKnora connections the console configures:
// endpoint + write-only API key per environment, with a real listing as the only
// proof a configuration is usable.
//
// It deliberately stores no knowledge-base binding. Which base receives 对内 /
// 对外 content stays in routing_kb_bindings / production_kb_bindings, because one
// meaning may only have one home.
package weknoraconn
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"strconv"
"strings"
"time"
postgresstore "knowledge-governance/internal/platform/postgres"
"knowledge-governance/internal/platform/weknora"
"knowledge-governance/internal/sealedsecret"
)
const (
EnvironmentTest = "test"
EnvironmentProduction = "production"
KeyActionKeep = "keep"
KeyActionReplace = "replace"
KeyActionClear = "clear"
// Outcome vocabulary shared by the API, the stored evidence and the page.
OutcomeOK = "ok"
OutcomeUnreachable = "unreachable"
OutcomeKeyInvalid = "key_invalid"
OutcomeKeyForbidden = "key_forbidden"
// VerificationNotConfigured means there is nothing to verify yet.
VerificationNotConfigured = "not_configured"
// proofTTL bounds how long a successful 测试连接 vouches for a candidate.
proofTTL = 15 * time.Minute
)
var (
ErrInvalid = errors.New("weknora connection request is invalid")
ErrNotFound = errors.New("unknown weknora environment")
ErrStaleRevision = errors.New("weknora connection revision is stale")
ErrNotTested = errors.New("weknora connection must be tested before saving")
ErrStorageUnavailable = errors.New("weknora connection secret storage is unavailable")
)
// Store is the persistence the service needs. The postgres store implements it.
type Store interface {
LoadWeKnoraConnection(ctx context.Context, environment string) (postgresstore.WeKnoraConnection, bool, error)
LoadWeKnoraConnections(ctx context.Context) ([]postgresstore.WeKnoraConnection, error)
SaveWeKnoraConnection(ctx context.Context, connection postgresstore.WeKnoraConnection, now time.Time) error
SaveWeKnoraConnectionTest(ctx context.Context, result postgresstore.WeKnoraConnectionTest, now time.Time) error
LoadWeKnoraConnectionTest(ctx context.Context, environment string) (postgresstore.WeKnoraConnectionTest, bool, error)
UpdateWeKnoraConnectionVerification(ctx context.Context, environment, verification string, libraryCount int, checkedAt time.Time, now time.Time) error
}
// KnowledgeBaseLister is the narrow client the service needs; tests inject it.
type KnowledgeBaseLister interface {
ListKnowledgeBases(ctx context.Context) ([]weknora.KnowledgeBase, error)
}
type Config struct {
// MasterKeyFile is the 0600 32-byte master key. Empty or unreadable means
// the page can read state but cannot store a key — reported, never faked.
MasterKeyFile string
// AllowPrivateHTTP follows WEKNORA_ALLOW_PRIVATE_HTTP: same-LAN plain http
// is usable, public http is not.
AllowPrivateHTTP bool
Timeout time.Duration
Now func() time.Time
// NewLister is overridable in tests. Production leaves it nil.
NewLister func(endpoint, apiKey string) (KnowledgeBaseLister, error)
}
type Service struct {
store Store
config Config
}
func New(store Store, config Config) *Service {
if config.Timeout <= 0 {
config.Timeout = 10 * time.Second
}
if config.Now == nil {
config.Now = time.Now
}
return &Service{store: store, config: config}
}
// View is the page projection. It never carries the key or the ciphertext.
type View struct {
Environment string `json:"environment"`
Endpoint string `json:"endpoint"`
KeyConfigured bool `json:"key_configured"`
KeyTail string `json:"key_tail"`
Verification string `json:"verification"`
LibraryCount int `json:"library_count"`
CheckedAt *time.Time `json:"checked_at"`
UpdatedBy string `json:"updated_by"`
Revision int64 `json:"revision"`
StorageReady bool `json:"storage_ready"`
}
// TestInput is one 测试连接 request. An empty endpoint means "the saved one"; an
// empty key_action means keep.
type TestInput struct {
Endpoint string `json:"endpoint"`
APIKey string `json:"api_key"`
KeyAction string `json:"key_action"`
}
type TestResult struct {
Environment string `json:"environment"`
Endpoint string `json:"endpoint"`
Outcome string `json:"outcome"`
LibraryCount int `json:"library_count"`
Message string `json:"message"`
CheckedAt time.Time `json:"checked_at"`
}
type SaveInput struct {
Endpoint string `json:"endpoint"`
APIKey string `json:"api_key"`
KeyAction string `json:"key_action"`
BaseRevision int64 `json:"base_revision"`
}
func ValidEnvironment(value string) bool {
return value == EnvironmentTest || value == EnvironmentProduction
}
// View reads both environments, or one when environment is non-empty. A missing
// row is projected as 未配置 so the page has a stable shape.
func (service *Service) View(ctx context.Context, environment string) ([]View, error) {
stored, err := service.store.LoadWeKnoraConnections(ctx)
if err != nil {
return nil, err
}
byEnvironment := make(map[string]postgresstore.WeKnoraConnection, len(stored))
for _, connection := range stored {
byEnvironment[connection.Environment] = connection
}
environments := []string{EnvironmentTest, EnvironmentProduction}
if environment != "" {
if !ValidEnvironment(environment) {
return nil, ErrNotFound
}
environments = []string{environment}
}
_, masterKeyErr := sealedsecret.LoadKeyFile(service.config.MasterKeyFile)
views := make([]View, 0, len(environments))
for _, name := range environments {
connection := byEnvironment[name]
verification := connection.Verification
if verification == "" {
verification = VerificationNotConfigured
}
views = append(views, View{
Environment: name,
Endpoint: connection.Endpoint,
KeyConfigured: len(connection.KeyCiphertext) > 0,
KeyTail: connection.KeyTail,
Verification: verification,
LibraryCount: connection.LibraryCount,
CheckedAt: connection.CheckedAt,
UpdatedBy: connection.UpdatedBy,
Revision: connection.Revision,
StorageReady: masterKeyErr == nil,
})
}
return views, nil
}
// Test performs the real GET /api/v1/knowledge-bases against the candidate
// endpoint and key, and records the outcome as the evidence a later save needs.
func (service *Service) Test(ctx context.Context, environment string, input TestInput) (TestResult, error) {
if !ValidEnvironment(environment) {
return TestResult{}, ErrNotFound
}
saved, exists, err := service.store.LoadWeKnoraConnection(ctx, environment)
if err != nil {
return TestResult{}, err
}
endpoint := strings.TrimSpace(input.Endpoint)
if endpoint == "" && exists {
endpoint = saved.Endpoint
}
if endpoint == "" {
return TestResult{}, ErrInvalid
}
action := normalizeKeyAction(input.KeyAction)
apiKey, err := service.candidateKey(saved, exists, action, input.APIKey)
if err != nil {
return TestResult{}, err
}
now := service.config.Now().UTC()
result := TestResult{Environment: environment, Endpoint: endpoint, CheckedAt: now}
if !weknora.HTTPAllowed(endpoint, weknora.HTTPPolicy{AllowPrivateHTTP: service.config.AllowPrivateHTTP}) {
result.Outcome = OutcomeUnreachable
result.Message = "地址不可达:只接受 https,或已放行的私网/回环 http 地址"
_ = service.recordTest(ctx, result, apiKey)
return result, nil
}
lister, err := service.newLister(endpoint, apiKey)
if err != nil {
result.Outcome = OutcomeUnreachable
result.Message = "地址不可达:" + err.Error()
_ = service.recordTest(ctx, result, apiKey)
return result, nil
}
probeCtx, cancel := context.WithTimeout(ctx, service.config.Timeout)
defer cancel()
bases, listErr := lister.ListKnowledgeBases(probeCtx)
if listErr != nil {
result.Outcome, result.Message = classifyListError(listErr)
_ = service.recordTest(ctx, result, apiKey)
return result, nil
}
result.Outcome = OutcomeOK
result.LibraryCount = len(bases)
result.Message = "接口可达 · 已读到 " + strconv.Itoa(len(bases)) + " 个知识库"
if recordErr := service.recordTest(ctx, result, apiKey); recordErr != nil {
// The listing worked but the proof did not land: say so now, because the
// save gate will refuse this configuration a moment later.
result.Message += "(测试证据未能落库,保存会被拒绝,请重试)"
}
// The card may only claim 「可达」 for the configuration it actually probed.
if exists && action != KeyActionReplace && endpoint == saved.Endpoint {
_ = service.store.UpdateWeKnoraConnectionVerification(ctx, environment, result.Outcome, result.LibraryCount, result.CheckedAt, now)
}
return result, nil
}
// Save stores a tested configuration. A new key or a new endpoint must have a
// successful test behind it; keeping a tested configuration is not blocked by a
// jittery endpoint.
func (service *Service) Save(ctx context.Context, environment string, input SaveInput, actor string) (View, error) {
if !ValidEnvironment(environment) {
return View{}, ErrNotFound
}
saved, exists, err := service.store.LoadWeKnoraConnection(ctx, environment)
if err != nil {
return View{}, err
}
endpoint := strings.TrimSpace(input.Endpoint)
if endpoint == "" {
return View{}, ErrInvalid
}
action := normalizeKeyAction(input.KeyAction)
if exists && input.BaseRevision != saved.Revision {
return View{}, ErrStaleRevision
}
if exists && endpoint != saved.Endpoint && action != KeyActionReplace {
return View{}, ErrInvalid
}
apiKey, err := service.candidateKey(saved, exists, action, input.APIKey)
if err != nil {
return View{}, err
}
now := service.config.Now().UTC()
revision := int64(1)
if exists {
revision = saved.Revision + 1
}
verification := VerificationNotConfigured
libraryCount := 0
var checkedAt *time.Time
if apiKey != "" {
if !weknora.HTTPAllowed(endpoint, weknora.HTTPPolicy{AllowPrivateHTTP: service.config.AllowPrivateHTTP}) {
return View{}, ErrInvalid
}
if !exists || action == KeyActionReplace || endpoint != saved.Endpoint {
proof, found, proofErr := service.store.LoadWeKnoraConnectionTest(ctx, environment)
if proofErr != nil {
return View{}, proofErr
}
if !found || proof.Outcome != OutcomeOK || proof.Endpoint != endpoint ||
proof.KeyFingerprint != fingerprint(apiKey) || now.Sub(proof.CheckedAt) > proofTTL {
return View{}, ErrNotTested
}
libraryCount = proof.LibraryCount
checkedAt = &proof.CheckedAt
} else {
libraryCount = saved.LibraryCount
checkedAt = saved.CheckedAt
}
verification = OutcomeOK
}
sealed := []byte{}
if apiKey != "" {
masterKey, keyErr := sealedsecret.LoadKeyFile(service.config.MasterKeyFile)
if keyErr != nil {
return View{}, ErrStorageUnavailable
}
sealed, err = sealedsecret.Seal(masterKey, apiKey, sealedsecret.AAD("weknora-connection", environment, revision))
if err != nil {
return View{}, ErrStorageUnavailable
}
}
connection := postgresstore.WeKnoraConnection{
Environment: environment,
Revision: revision,
Endpoint: endpoint,
KeyCiphertext: sealed,
KeyTail: keyTail(apiKey),
Verification: verification,
LibraryCount: libraryCount,
CheckedAt: checkedAt,
UpdatedBy: actor,
}
if err := service.store.SaveWeKnoraConnection(ctx, connection, now); err != nil {
return View{}, err
}
views, err := service.View(ctx, environment)
if err != nil {
return View{}, err
}
return views[0], nil
}
// Resolve returns the endpoint and key the platform should use for one
// environment. It is the single source other features read instead of guessing
// from environment variables.
func (service *Service) Resolve(ctx context.Context, environment string) (string, string, bool) {
if !ValidEnvironment(environment) {
return "", "", false
}
saved, exists, err := service.store.LoadWeKnoraConnection(ctx, environment)
if err != nil || !exists || saved.Endpoint == "" || len(saved.KeyCiphertext) == 0 {
return "", "", false
}
masterKey, keyErr := sealedsecret.LoadKeyFile(service.config.MasterKeyFile)
if keyErr != nil {
return "", "", false
}
apiKey, openErr := sealedsecret.Open(masterKey, saved.KeyCiphertext, sealedsecret.AAD("weknora-connection", environment, saved.Revision))
if openErr != nil {
return "", "", false
}
return saved.Endpoint, apiKey, true
}
// HTTPPolicy exposes the endpoint policy the client uses, so a caller that
// builds its own client stays in agreement with the service.
func (service *Service) HTTPPolicy() weknora.HTTPPolicy {
return weknora.HTTPPolicy{AllowPrivateHTTP: service.config.AllowPrivateHTTP}
}
func (service *Service) candidateKey(saved postgresstore.WeKnoraConnection, exists bool, action, inputKey string) (string, error) {
switch action {
case KeyActionClear:
return "", nil
case KeyActionReplace:
trimmed := strings.TrimSpace(inputKey)
if trimmed == "" {
return "", ErrInvalid
}
return trimmed, nil
default:
if !exists || len(saved.KeyCiphertext) == 0 {
return "", ErrInvalid
}
masterKey, err := sealedsecret.LoadKeyFile(service.config.MasterKeyFile)
if err != nil {
return "", ErrStorageUnavailable
}
apiKey, err := sealedsecret.Open(masterKey, saved.KeyCiphertext, sealedsecret.AAD("weknora-connection", saved.Environment, saved.Revision))
if err != nil {
return "", ErrStorageUnavailable
}
return apiKey, nil
}
}
func (service *Service) newLister(endpoint, apiKey string) (KnowledgeBaseLister, error) {
if service.config.NewLister != nil {
return service.config.NewLister(endpoint, apiKey)
}
return weknora.NewClient(weknora.ClientConfig{
BaseURL: endpoint,
APIKey: apiKey,
Timeout: service.config.Timeout,
AllowPrivateHTTP: service.config.AllowPrivateHTTP,
})
}
func (service *Service) recordTest(ctx context.Context, result TestResult, apiKey string) error {
return service.store.SaveWeKnoraConnectionTest(ctx, postgresstore.WeKnoraConnectionTest{
Environment: result.Environment,
Endpoint: result.Endpoint,
KeyFingerprint: fingerprint(apiKey),
Outcome: result.Outcome,
LibraryCount: result.LibraryCount,
Message: result.Message,
}, result.CheckedAt)
}
// classifyListError maps one failed listing onto the four answers the page
// promises, so 「密钥无效」 and 「密钥无权限」 never collapse into one message.
func classifyListError(err error) (string, string) {
var httpErr *weknora.HTTPError
if errors.As(err, &httpErr) {
switch httpErr.StatusCode {
case 401:
return OutcomeKeyInvalid, "密钥无效:该实例拒绝了这把 API Key(401)"
case 403:
return OutcomeKeyForbidden, "密钥无权限读取知识库列表(403)"
}
}
switch {
case errors.Is(err, context.DeadlineExceeded):
return OutcomeUnreachable, "地址不可达:请求超时"
default:
var netErr interface{ Timeout() bool }
if errors.As(err, &netErr) && netErr.Timeout() {
return OutcomeUnreachable, "地址不可达:请求超时"
}
return OutcomeUnreachable, "地址不可达:连接失败"
}
}
func normalizeKeyAction(action string) string {
switch strings.TrimSpace(action) {
case KeyActionReplace:
return KeyActionReplace
case KeyActionClear:
return KeyActionClear
default:
return KeyActionKeep
}
}
func fingerprint(apiKey string) string {
sum := sha256.Sum256([]byte(apiKey))
return hex.EncodeToString(sum[:])
}
// keyTail is the only part of a key the page may show: the last four characters.
func keyTail(apiKey string) string {
if apiKey == "" {
return ""
}
runes := []rune(apiKey)
if len(runes) <= 4 {
return string(runes)
}
return string(runes[len(runes)-4:])
}
+249
View File
@@ -0,0 +1,249 @@
package weknoraconn
import (
"context"
"errors"
"os"
"path/filepath"
"testing"
"time"
"knowledge-governance/internal/platform/postgres"
"knowledge-governance/internal/platform/weknora"
)
type fakeStore struct {
connections map[string]postgres.WeKnoraConnection
tests map[string]postgres.WeKnoraConnectionTest
}
func newFakeStore() *fakeStore {
return &fakeStore{
connections: map[string]postgres.WeKnoraConnection{},
tests: map[string]postgres.WeKnoraConnectionTest{},
}
}
func (store *fakeStore) LoadWeKnoraConnection(_ context.Context, environment string) (postgres.WeKnoraConnection, bool, error) {
connection, ok := store.connections[environment]
return connection, ok, nil
}
func (store *fakeStore) LoadWeKnoraConnections(context.Context) ([]postgres.WeKnoraConnection, error) {
result := make([]postgres.WeKnoraConnection, 0, len(store.connections))
for _, connection := range store.connections {
result = append(result, connection)
}
return result, nil
}
func (store *fakeStore) SaveWeKnoraConnection(_ context.Context, connection postgres.WeKnoraConnection, _ time.Time) error {
store.connections[connection.Environment] = connection
return nil
}
func (store *fakeStore) SaveWeKnoraConnectionTest(_ context.Context, result postgres.WeKnoraConnectionTest, now time.Time) error {
result.CheckedAt = now
store.tests[result.Environment] = result
return nil
}
func (store *fakeStore) LoadWeKnoraConnectionTest(_ context.Context, environment string) (postgres.WeKnoraConnectionTest, bool, error) {
result, ok := store.tests[environment]
return result, ok, nil
}
func (store *fakeStore) UpdateWeKnoraConnectionVerification(_ context.Context, environment, verification string, libraryCount int, checkedAt time.Time, _ time.Time) error {
connection, ok := store.connections[environment]
if !ok {
return nil
}
connection.Verification = verification
connection.LibraryCount = libraryCount
connection.CheckedAt = &checkedAt
store.connections[environment] = connection
return nil
}
type fakeLister struct {
bases []weknora.KnowledgeBase
err error
}
func (lister fakeLister) ListKnowledgeBases(context.Context) ([]weknora.KnowledgeBase, error) {
return lister.bases, lister.err
}
func newService(t *testing.T, store Store, lister fakeLister) *Service {
t.Helper()
path := filepath.Join(t.TempDir(), "master.key")
if err := os.WriteFile(path, []byte("0123456789abcdef0123456789abcdef"), 0o600); err != nil {
t.Fatalf("write master key: %v", err)
}
if err := os.Chmod(path, 0o600); err != nil {
t.Fatalf("chmod master key: %v", err)
}
now := time.Date(2026, 11, 22, 7, 0, 0, 0, time.UTC)
return New(store, Config{
MasterKeyFile: path,
AllowPrivateHTTP: true,
Timeout: time.Second,
Now: func() time.Time { return now },
NewLister: func(endpoint, apiKey string) (KnowledgeBaseLister, error) {
if endpoint == "" || apiKey == "" {
return nil, errors.New("missing endpoint or key")
}
return lister, nil
},
})
}
func TestTestClassifiesTheFourAnswers(t *testing.T) {
cases := []struct {
name string
err error
wantOutcome string
wantMessage string
}{
{"unauthorized", &weknora.HTTPError{StatusCode: 401}, OutcomeKeyInvalid, "密钥无效"},
{"forbidden", &weknora.HTTPError{StatusCode: 403}, OutcomeKeyForbidden, "密钥无权限"},
{"timeout", context.DeadlineExceeded, OutcomeUnreachable, "地址不可达"},
{"refused", errors.New("dial tcp: connection refused"), OutcomeUnreachable, "地址不可达"},
}
for _, testCase := range cases {
service := newService(t, newFakeStore(), fakeLister{err: testCase.err})
result, err := service.Test(context.Background(), EnvironmentTest, TestInput{
Endpoint: "http://127.0.0.1:8092", APIKey: "sk-test-1234", KeyAction: KeyActionReplace,
})
if err != nil {
t.Fatalf("%s: Test() error = %v", testCase.name, err)
}
if result.Outcome != testCase.wantOutcome {
t.Fatalf("%s: outcome = %q, want %q", testCase.name, result.Outcome, testCase.wantOutcome)
}
if !contains(result.Message, testCase.wantMessage) {
t.Fatalf("%s: message = %q, want it to mention %q", testCase.name, result.Message, testCase.wantMessage)
}
}
}
func TestTestSuccessReportsLibraryCount(t *testing.T) {
service := newService(t, newFakeStore(), fakeLister{bases: []weknora.KnowledgeBase{{ID: "a"}, {ID: "b"}, {ID: "c"}}})
result, err := service.Test(context.Background(), EnvironmentProduction, TestInput{
Endpoint: "http://172.29.151.11:18093", APIKey: "prod-key-9999", KeyAction: KeyActionReplace,
})
if err != nil {
t.Fatalf("Test() error = %v", err)
}
if result.Outcome != OutcomeOK || result.LibraryCount != 3 {
t.Fatalf("result = %+v", result)
}
if !contains(result.Message, "3 个知识库") {
t.Fatalf("message = %q", result.Message)
}
}
func TestSaveRequiresTestProofAndProjectsTailOnly(t *testing.T) {
store := newFakeStore()
service := newService(t, store, fakeLister{bases: []weknora.KnowledgeBase{{ID: "a"}, {ID: "b"}}})
ctx := context.Background()
save := SaveInput{Endpoint: "http://127.0.0.1:8092", APIKey: "sk-live-abcd", KeyAction: KeyActionReplace}
if _, err := service.Save(ctx, EnvironmentTest, save, "admin"); !errors.Is(err, ErrNotTested) {
t.Fatalf("Save() without a test = %v, want ErrNotTested", err)
}
if _, err := service.Test(ctx, EnvironmentTest, TestInput{Endpoint: save.Endpoint, APIKey: save.APIKey, KeyAction: KeyActionReplace}); err != nil {
t.Fatalf("Test() error = %v", err)
}
view, err := service.Save(ctx, EnvironmentTest, save, "admin")
if err != nil {
t.Fatalf("Save() error = %v", err)
}
if !view.KeyConfigured || view.KeyTail != "abcd" || view.Verification != OutcomeOK || view.LibraryCount != 2 {
t.Fatalf("view = %+v", view)
}
if view.Revision != 1 {
t.Fatalf("revision = %d", view.Revision)
}
endpoint, apiKey, ok := service.Resolve(ctx, EnvironmentTest)
if !ok || endpoint != save.Endpoint || apiKey != save.APIKey {
t.Fatalf("Resolve() = %q %q %v", endpoint, apiKey, ok)
}
// The proof is bound to the key: another key must not reuse it.
other := SaveInput{Endpoint: save.Endpoint, APIKey: "sk-live-zzzz", KeyAction: KeyActionReplace, BaseRevision: 1}
if _, err := service.Save(ctx, EnvironmentTest, other, "admin"); !errors.Is(err, ErrNotTested) {
t.Fatalf("Save() with a different untested key = %v, want ErrNotTested", err)
}
}
func TestSaveRejectsEndpointChangeWithoutReenteringKey(t *testing.T) {
store := newFakeStore()
service := newService(t, store, fakeLister{bases: []weknora.KnowledgeBase{{ID: "a"}}})
ctx := context.Background()
first := SaveInput{Endpoint: "http://127.0.0.1:8092", APIKey: "sk-live-abcd", KeyAction: KeyActionReplace}
if _, err := service.Test(ctx, EnvironmentTest, TestInput{Endpoint: first.Endpoint, APIKey: first.APIKey, KeyAction: KeyActionReplace}); err != nil {
t.Fatal(err)
}
if _, err := service.Save(ctx, EnvironmentTest, first, "admin"); err != nil {
t.Fatalf("first Save() error = %v", err)
}
if _, err := service.Save(ctx, EnvironmentTest, SaveInput{Endpoint: "http://127.0.0.1:9092", KeyAction: KeyActionKeep, BaseRevision: 1}, "admin"); !errors.Is(err, ErrInvalid) {
t.Fatalf("endpoint change with keep = %v, want ErrInvalid", err)
}
if _, err := service.Save(ctx, EnvironmentTest, SaveInput{Endpoint: "http://127.0.0.1:8092", KeyAction: KeyActionKeep, BaseRevision: 7}, "admin"); !errors.Is(err, ErrStaleRevision) {
t.Fatalf("stale revision = %v, want ErrStaleRevision", err)
}
// Keeping the tested configuration must not be blocked by proof age.
view, err := service.Save(ctx, EnvironmentTest, SaveInput{Endpoint: "http://127.0.0.1:8092", KeyAction: KeyActionKeep, BaseRevision: 1}, "admin")
if err != nil || view.Verification != OutcomeOK || view.KeyTail != "abcd" {
t.Fatalf("keep Save() = %+v, %v", view, err)
}
}
func TestClearKeyResetsVerification(t *testing.T) {
store := newFakeStore()
service := newService(t, store, fakeLister{bases: []weknora.KnowledgeBase{{ID: "a"}}})
ctx := context.Background()
if _, err := service.Test(ctx, EnvironmentProduction, TestInput{Endpoint: "https://prod.example", APIKey: "prod-key", KeyAction: KeyActionReplace}); err != nil {
t.Fatal(err)
}
if _, err := service.Save(ctx, EnvironmentProduction, SaveInput{Endpoint: "https://prod.example", APIKey: "prod-key", KeyAction: KeyActionReplace}, "admin"); err != nil {
t.Fatal(err)
}
view, err := service.Save(ctx, EnvironmentProduction, SaveInput{Endpoint: "https://prod.example", KeyAction: KeyActionClear, BaseRevision: 1}, "admin")
if err != nil {
t.Fatalf("clear Save() error = %v", err)
}
if view.KeyConfigured || view.KeyTail != "" || view.Verification != VerificationNotConfigured {
t.Fatalf("cleared view = %+v", view)
}
if _, _, ok := service.Resolve(ctx, EnvironmentProduction); ok {
t.Fatal("a cleared connection must not resolve")
}
}
func TestPublicHTTPIsRefusedEvenWhenPrivateHTTPIsAllowed(t *testing.T) {
service := newService(t, newFakeStore(), fakeLister{})
result, err := service.Test(context.Background(), EnvironmentTest, TestInput{Endpoint: "http://weknora.example", APIKey: "k", KeyAction: KeyActionReplace})
if err != nil {
t.Fatalf("Test() error = %v", err)
}
if result.Outcome != OutcomeUnreachable {
t.Fatalf("outcome = %q", result.Outcome)
}
if _, err := service.Save(context.Background(), EnvironmentTest, SaveInput{Endpoint: "http://weknora.example", APIKey: "k", KeyAction: KeyActionReplace}, "admin"); err == nil {
t.Fatal("saving a public http endpoint must fail")
}
}
func contains(value, needle string) bool {
return len(needle) == 0 || (len(value) >= len(needle) && indexOf(value, needle) >= 0)
}
func indexOf(value, needle string) int {
for i := 0; i+len(needle) <= len(value); i++ {
if value[i:i+len(needle)] == needle {
return i
}
}
return -1
}
+10 -3
View File
@@ -1,6 +1,7 @@
import { useCallback, useEffect, useState } from 'react'
import { useOptionalAuth } from './auth'
import { WeKnoraConnections } from './WeKnoraConnections'
import { fetchProductionBindingOptions, fetchProductionBindings, saveProductionBindings } from './productionBindings'
import type { ProductionBindings } from './productionBindings'
import type { KnowledgeBaseOption } from './routingBindings'
@@ -41,9 +42,12 @@ export function ProductionBindingsSection() {
const next = await saveProductionBindings({ internal_kb_id: internalDraft, external_kb_id: externalDraft })
setBindings(next)
const where = `对内 ${next.internal_kb_name || next.internal_kb_id.slice(0, 8) || '未绑定'} · 对外 ${next.external_kb_name || next.external_kb_id.slice(0, 8) || '未绑定'}`
const frozen = next.frozen + next.retargetable > 0
? `;已冻结目标 ${next.frozen} 条(不会被改写),仍可改指向 ${next.retargetable} 条`
: ''
setMessage(next.production_connected
? `发布目标已更新:${where}。之后新发布的“对内/对外”内容会写入这两个生产库(已发布过的条目仍回到原目标)。`
: `发布目标已保存:${where};但当前核对不了生产端(${next.production_message}),下次发布前请确认。`)
? `发布目标已更新:${where}${frozen}。之后新发布的“对内/对外”内容会写入这两个生产库。`
: `发布目标已保存:${where}${frozen};但当前核对不了生产端(${next.production_message}),下次发布前请确认。`)
} catch (cause) { setError(cause instanceof Error ? cause.message : '保存生产知识库绑定失败') }
finally { setSaving(false) }
}
@@ -62,11 +66,14 @@ export function ProductionBindingsSection() {
<option value="">{placeholder}</option>
{options.map((base) => <option key={base.id} value={base.id}>{optionLabel(base)}</option>)}
</select>
: <input value={value} onChange={(event) => onChange(event.target.value.trim())} placeholder={`${placeholder}(输入生产知识库 ID)`} aria-label={placeholder} />
: <select value="" disabled aria-label={placeholder}>
<option value="">读不到生产库列表:先在上方把生产连接测通</option>
</select>
)
return <section className="settings-panel wide" id="settings-production-bindings">
<div className="settings-section-heading"><div><h2>生产知识库绑定</h2><p>发布链路的“源库 → 目标库”:新发布的对内 / 对外内容按这里的绑定写入生产 WeKnora 的对应知识库。留空表示未绑定,按同 ID 语义发布。</p></div><button onClick={() => void load()} disabled={loading}>{loading ? '读取中…' : '重新读取'}</button></div>
<WeKnoraConnections environment="production" title="生产 WeKnora" placeholder="http://172.29.151.11:18093" onSaved={() => void load()} />
{error ? <p className="error-message" role="alert">{error}</p> : null}
{bindings ? <div className="kb-binding-grid">
{renderSide('对内生产知识库', bindings.internal_kb_id, bindings.internal_kb_name)}
+148 -43
View File
@@ -1,6 +1,7 @@
import { useCallback, useEffect, useState } from 'react'
import { useOptionalAuth } from './auth'
import { WeKnoraConnections } from './WeKnoraConnections'
import { fetchRoutingBindingOptions, fetchRoutingBindings, saveRoutingBindings } from './routingBindings'
import type { KnowledgeBaseOption, RoutingBindings } from './routingBindings'
@@ -17,67 +18,171 @@ export function RoutingBindingsSection() {
const [message, setMessage] = useState('')
const load = useCallback(async () => {
setLoading(true); setError(null)
setLoading(true)
setError(null)
try {
const next = await fetchRoutingBindings()
setBindings(next)
setInternalDraft(next.internal_bound_by_ui ? next.internal_kb_id : '')
setExternalDraft(next.external_bound_by_ui ? next.external_kb_id : '')
try { setOptions(await fetchRoutingBindingOptions()) } catch { setOptions([]) }
} catch (cause) { setError(cause instanceof Error ? cause.message : '读取测试知识库绑定失败') }
finally { setLoading(false) }
try {
setOptions(await fetchRoutingBindingOptions())
} catch {
setOptions([])
}
} catch (cause) {
setError(cause instanceof Error ? cause.message : '读取测试知识库绑定失败')
} finally {
setLoading(false)
}
}, [])
useEffect(() => { void load() }, [load])
useEffect(() => {
void load()
}, [load])
const dirty = bindings !== null && (internalDraft !== (bindings.internal_bound_by_ui ? bindings.internal_kb_id : '') || externalDraft !== (bindings.external_bound_by_ui ? bindings.external_kb_id : ''))
const dirty =
bindings !== null &&
(internalDraft !== (bindings.internal_bound_by_ui ? bindings.internal_kb_id : '') ||
externalDraft !== (bindings.external_bound_by_ui ? bindings.external_kb_id : ''))
const sameSelection = internalDraft !== '' && internalDraft === externalDraft
const save = async () => {
setSaving(true); setError(null); setMessage('')
setSaving(true)
setError(null)
setMessage('')
try {
const next = await saveRoutingBindings({ internal_kb_id: internalDraft, external_kb_id: externalDraft })
setBindings(next)
const replan = next.replanned || next.retired_duplicates
? `;同时把 ${next.replanned} 项还没上传的待入库重新指向新库${next.retired_duplicates ? `,${next.retired_duplicates} 项因新库里已有同篇而被退休` : ''}`
: ''
setMessage(`绑定已生效:对内 ${next.internal_kb_name || next.internal_kb_id.slice(0, 8) || '未绑定'} · 对外 ${next.external_kb_name || next.external_kb_id.slice(0, 8) || '未绑定'}。此后“来源与目录”的标记按新绑定路由${replan}。`)
} catch (cause) { setError(cause instanceof Error ? cause.message : '保存测试知识库绑定失败') }
finally { setSaving(false) }
const replan =
next.replanned || next.retired_duplicates
? `;同时把 ${next.replanned} 项还没上传的待入库重新指向新库${next.retired_duplicates ? `,${next.retired_duplicates} 项因新库里已有同篇而被退休` : ''}`
: ''
const fallbacks: string[] = []
if (!next.internal_bound_by_ui)
fallbacks.push(
`对内回退到部署默认库 ${next.internal_fallback_kb_id ? `${next.internal_fallback_kb_id.slice(0, 8)}…` : '(未设置)'}`
)
if (!next.external_bound_by_ui)
fallbacks.push(
`对外回退到部署默认库 ${next.external_fallback_kb_id ? `${next.external_fallback_kb_id.slice(0, 8)}…` : '(未设置)'}`
)
setMessage(
`绑定已生效:对内 ${next.internal_kb_name || next.internal_kb_id.slice(0, 8) || '未绑定'} · 对外 ${next.external_kb_name || next.external_kb_id.slice(0, 8) || '未绑定'}${fallbacks.length ? `;${fallbacks.join(';')}` : ''}。此后“来源与目录”的标记按新绑定路由${replan}。`
)
} catch (cause) {
setError(cause instanceof Error ? cause.message : '保存测试知识库绑定失败')
} finally {
setSaving(false)
}
}
const optionLabel = (base: KnowledgeBaseOption) => `${base.name}(${base.knowledge_count} 文档 · ${base.id.slice(0, 8)}…)`
const renderSide = (label: string, boundID: string, boundName: string, fromEnv: boolean, boundByUI: boolean) => (
const optionLabel = (base: KnowledgeBaseOption) =>
`${base.name}(${base.knowledge_count} 文档 · ${base.id.slice(0, 8)}…)`
const renderSide = (
label: string,
boundID: string,
boundName: string,
fromEnv: boolean,
boundByUI: boolean,
fallbackID = ''
) => (
<div className={`kb-binding-side ${boundID ? 'bound' : 'warn'}`}>
<b>{label}</b>
<span>{boundID ? (boundName || `${boundID.slice(0, 8)}…`) : '未绑定 · 该方向的标记会被拒绝'}</span>
<small>{boundID ? (boundByUI ? (boundName ? `UI 绑定 · ${boundID.slice(0, 8)}…` : 'UI 绑定') : fromEnv ? '来自部署默认值(env)' : '') : '请绑定一个测试知识库'}</small>
<span>{boundID ? boundName || `${boundID.slice(0, 8)}…` : '未绑定 · 该方向的标记会被拒绝'}</span>
<small>
{boundID
? boundByUI
? boundName
? `UI 绑定 · ${boundID.slice(0, 8)}…`
: 'UI 绑定'
: fromEnv
? `回退到部署默认库 ${fallbackID ? `${fallbackID.slice(0, 8)}…` : '(未设置)'}`
: ''
: '请绑定一个测试知识库'}
</small>
</div>
)
const editorFor = (value: string, onChange: (next: string) => void, placeholder: string) => (
options.length > 0
? <select value={value} onChange={(event) => onChange(event.target.value)} aria-label={placeholder}>
<option value="">{placeholder}</option>
{options.map((base) => <option key={base.id} value={base.id}>{optionLabel(base)}</option>)}
</select>
: <input value={value} onChange={(event) => onChange(event.target.value.trim())} placeholder={`${placeholder}(输入知识库 ID)`} aria-label={placeholder} />
)
const editorFor = (value: string, onChange: (next: string) => void, placeholder: string) =>
options.length > 0 ? (
<select value={value} onChange={(event) => onChange(event.target.value)} aria-label={placeholder}>
<option value="">{placeholder}</option>
{options.map((base) => (
<option key={base.id} value={base.id}>
{optionLabel(base)}
</option>
))}
</select>
) : (
<select value="" disabled aria-label={placeholder}>
<option value="">读不到测试库列表:先在上方把测试连接测通</option>
</select>
)
return <section className="settings-panel wide" id="settings-routing-bindings">
<div className="settings-section-heading"><div><h2>测试知识库绑定</h2><p>“来源与目录”里的对内 / 对外标记按这里的绑定分发到对应的测试 WeKnora 知识库;保存后立即生效,无需重启。发布写入的生产知识库见「生产知识库绑定」。</p></div><button onClick={() => void load()} disabled={loading}>{loading ? '读取中…' : '重新读取'}</button></div>
{error ? <p className="error-message" role="alert">{error}</p> : null}
{bindings ? <div className="kb-binding-grid">
{renderSide('对内知识库', bindings.internal_kb_id, bindings.internal_kb_name, bindings.internal_env_fallback, bindings.internal_bound_by_ui)}
{renderSide('对外知识库', bindings.external_kb_id, bindings.external_kb_name, bindings.external_env_fallback, bindings.external_bound_by_ui)}
</div> : null}
{bindings && !bindings.weknora_connected ? <p className="hint">当前读不到 WeKnora 的库列表,只能按 ID 填写;连接恢复后可改为下拉选择。</p> : null}
{isAdmin && bindings ? <div className="kb-binding-editor">
<label>对内绑定 {editorFor(internalDraft, setInternalDraft, '选择对内知识库')}</label>
<label>对外绑定 {editorFor(externalDraft, setExternalDraft, '选择对外知识库')}</label>
<p className="hint">留空 = 取消 UI 绑定,回退到部署默认值;对内与对外必须是不同的库。</p>
{sameSelection ? <p className="error-message">对内与对外必须绑定不同的知识库</p> : null}
<button onClick={() => void save()} disabled={saving || !dirty || sameSelection}>{saving ? '正在校验并保存…' : '保存绑定'}</button>
{message ? <p className="success-message" role="status">{message}</p> : null}
</div> : null}
{!isAdmin ? <p className="hint">仅管理员可修改测试知识库绑定。</p> : null}
</section>
return (
<section className="settings-panel wide" id="settings-routing-bindings">
<div className="settings-section-heading">
<div>
<h2>测试知识库绑定</h2>
<p>
“来源与目录”里的对内 / 对外标记按这里的绑定分发到对应的测试 WeKnora
知识库;保存后立即生效,无需重启。发布写入的生产知识库见「生产知识库绑定」。
</p>
</div>
<button onClick={() => void load()} disabled={loading}>
{loading ? '读取中…' : '重新读取'}
</button>
</div>
<WeKnoraConnections
environment="test"
title="测试 WeKnora"
placeholder="http://127.0.0.1:8092"
onSaved={() => void load()}
/>
{error ? (
<p className="error-message" role="alert">
{error}
</p>
) : null}
{bindings ? (
<div className="kb-binding-grid">
{renderSide(
'对内知识库',
bindings.internal_kb_id,
bindings.internal_kb_name,
bindings.internal_env_fallback,
bindings.internal_bound_by_ui,
bindings.internal_fallback_kb_id
)}
{renderSide(
'对外知识库',
bindings.external_kb_id,
bindings.external_kb_name,
bindings.external_env_fallback,
bindings.external_bound_by_ui,
bindings.external_fallback_kb_id
)}
</div>
) : null}
{bindings && !bindings.weknora_connected ? (
<p className="hint">当前读不到 WeKnora 的库列表:先在上面的连接卡把地址与密钥配好并测通,再回来选库。</p>
) : null}
{isAdmin && bindings ? (
<div className="kb-binding-editor">
<label>对内绑定 {editorFor(internalDraft, setInternalDraft, '选择对内知识库')}</label>
<label>对外绑定 {editorFor(externalDraft, setExternalDraft, '选择对外知识库')}</label>
<p className="hint">留空 = 取消 UI 绑定,回退到部署默认值;对内与对外必须是不同的库。</p>
{sameSelection ? <p className="error-message">对内与对外必须绑定不同的知识库</p> : null}
<button onClick={() => void save()} disabled={saving || !dirty || sameSelection}>
{saving ? '正在校验并保存…' : '保存绑定'}
</button>
{message ? (
<p className="success-message" role="status">
{message}
</p>
) : null}
</div>
) : null}
{!isAdmin ? <p className="hint">仅管理员可修改测试知识库绑定。</p> : null}
</section>
)
}
+5 -3
View File
@@ -424,12 +424,14 @@ describe('SystemSettingsView', () => {
expect(on).toContain('测试 WeKnora')
expect(on).toContain('可达 · 可接单')
expect(on).toContain('延迟 86ms')
expect(on).toContain('已启用 · 可提交生产请求')
expect(on).toContain('生产写入:开启 · 可提交生产请求')
expect(on).toContain('生产写入:开启')
expect(on).toContain('发布目标与凭据:管理员受限配置')
const off = view(false, false)
expect(off).toContain('未启用')
// 连接状态与生产写入是两行:闸门关掉不代表「连接不可达」,两个事实分开说。
expect(off).toContain('连接:可达')
expect(off).toContain('生产写入:关闭')
expect(off).not.toContain('被阻断')
// 卡片状态由真实探测/门禁驱动,写死的“已连接 / 未连接 / 生产映射:未配置”必须消失。
for (const removed of ['本地真实连接', '生产映射:未配置']) expect(off).not.toContain(removed)
// 生产端点可达性来自真实探活,且只说真话:探活不等于验证过接单能力。
@@ -453,7 +455,7 @@ describe('SystemSettingsView', () => {
initialSection="services"
/>
)
expect(blocked).toContain('已启用 · 被阻断')
expect(blocked).toContain('生产写入:开启 · 恢复演练未核验')
// 未配置生产端点时必须显式说「未配置」,不得靠文案含糊过去。
expect(blocked).toContain('端点探测:暂无真实探测结果')
const unconfigured = renderToStaticMarkup(
+23 -26
View File
@@ -119,6 +119,15 @@ export const productionWeKnoraProbe = (monitor: MonitorOverview | null): string
return `端点探测:${label}${detail ? `(${detail})` : ''}`
}
// 生产卡片拆两行:连接是否可达,与生产写入是否开启是两件事 —— 以前把
// 「被阻断」混进连接状态,页面会自相矛盾(端点明明可达却显示被阻断)。
const productionConnectionLine = (monitor: MonitorOverview | null): string => {
const health = findHealth(monitor, '生产 WeKnora')
if (!health) return '连接:暂无探测结果'
if (!health.configured) return '连接:未配置'
return health.liveness === 'up' ? '连接:可达' : '连接:不可达'
}
export function SystemSettingsView({
sources,
connections,
@@ -199,6 +208,12 @@ export function SystemSettingsView({
setDirty(false)
onSave(candidateInput())
}
const productionConnectionLineText = productionConnectionLine(monitor)
const productionWriteLine = !productionEnabled
? '生产写入:关闭'
: productionBackupRestoreVerified
? '生产写入:开启 · 可提交生产请求'
: '生产写入:开启 · 恢复演练未核验'
return (
<main className="page-shell system-settings">
<header className="hero compact">
@@ -278,14 +293,8 @@ export function SystemSettingsView({
</article>
<article className="production">
<b>生产 WeKnora</b>
<span>
{!productionEnabled
? '未启用'
: productionBackupRestoreVerified
? '已启用 · 可提交生产请求'
: '已启用 · 被阻断'}
</span>
<small>{`${productionWeKnoraProbe(monitor)};生产写入:${productionEnabled ? '开启' : '关闭'};发布目标与凭据:管理员受限配置,本页不可回读`}</small>
<span>{productionConnectionLineText}</span>
<small>{`${productionWeKnoraProbe(monitor)};${productionWriteLine};发布目标与凭据:管理员受限配置,本页不可回读`}</small>
</article>
</div>
<div className="safe-setting-grid production-endpoint-form">
@@ -293,27 +302,11 @@ export function SystemSettingsView({
<>
<label>
测试地址
<input
value={testEndpoint}
onChange={(event) => {
setTestEndpoint(event.target.value)
setDirty(true)
}}
placeholder="https://weknora.test.example"
aria-invalid={!testEndpointValid}
/>
<input value={testEndpoint || '未配置'} readOnly disabled />
</label>
<label>
生产地址
<input
value={endpoint}
onChange={(event) => {
setEndpoint(event.target.value)
setDirty(true)
}}
placeholder="https://weknora.prod.example"
aria-invalid={!endpointValid}
/>
<input value={endpoint || '未配置'} readOnly disabled />
</label>
<label>
生产 ID
@@ -376,6 +369,10 @@ export function SystemSettingsView({
原子切换);后端同时对测试地址/生产地址做真实探活,并用生产地址真查这两个 ID(WeKnora
列库接口):端点回答"没有这个知识库"或租户不符会拒绝保存,端点抖动、鉴权失败或未配置生产凭据只警告不拦,核验原话与知识库名在保存后显示。
</small>
<p className="hint">
地址已移到「测试知识库绑定 /
生产知识库绑定」分区顶部的连接卡维护(可在那里测试连接);这里是回退值,只读。
</p>
{serviceSettingsValid ? null : (
<p className="error-message" role="alert">
格式不合法:地址需 http:// 或 https:// 开头的纯地址,不能带用户名密码、? 查询串或 # 片段;生产
+183
View File
@@ -0,0 +1,183 @@
import { useCallback, useEffect, useState } from 'react'
import { PasswordInput } from './PasswordInput'
import { useOptionalAuth } from './auth'
import {
fetchWeKnoraConnections,
keyLabel,
saveWeKnoraConnection,
testWeKnoraConnection,
verificationLabel,
} from './weknoraConnections'
import type { WeKnoraConnection, WeKnoraEnvironment } from './weknoraConnections'
interface Props {
environment: WeKnoraEnvironment
title: string
placeholder: string
onSaved?: () => void
}
// 连接卡:地址 + 写-only 密钥 + 真实列库测试。它只改「连哪台实例、用哪把钥匙」,
// 不碰绑定,也不触发任何同步或发布。
export function WeKnoraConnections({ environment, title, placeholder, onSaved }: Props) {
const auth = useOptionalAuth()
const isAdmin = auth?.identity?.role === 'admin'
const [connection, setConnection] = useState<WeKnoraConnection | null>(null)
const [endpointDraft, setEndpointDraft] = useState('')
const [keyDraft, setKeyDraft] = useState('')
const [testing, setTesting] = useState(false)
const [saving, setSaving] = useState(false)
const [error, setError] = useState<string | null>(null)
const [message, setMessage] = useState('')
const [testedEndpoint, setTestedEndpoint] = useState('')
const load = useCallback(async () => {
try {
const all = await fetchWeKnoraConnections()
const found = all.find((item) => item.environment === environment) ?? null
setConnection(found)
setEndpointDraft(found?.endpoint ?? '')
} catch (cause) {
setError(cause instanceof Error ? cause.message : '读取连接失败')
}
}, [environment])
useEffect(() => {
void load()
}, [load])
const endpoint = endpointDraft.trim()
const keyEntered = keyDraft.trim() !== ''
const endpointChanged = connection !== null && endpoint !== connection.endpoint
const canTest = endpoint !== '' && !testing && !saving
const tested = testedEndpoint !== '' && testedEndpoint === endpoint
const canSave = endpoint !== '' && !saving && !testing && (keyEntered || endpointChanged ? tested : true)
const test = async () => {
setTesting(true)
setError(null)
setMessage('')
try {
const result = await testWeKnoraConnection(environment, {
endpoint,
...(keyEntered
? { api_key: keyDraft.trim(), key_action: 'replace' as const }
: { key_action: 'keep' as const }),
})
setMessage(result.message)
setTestedEndpoint(result.outcome === 'ok' ? endpoint : '')
if (result.outcome !== 'ok') setError(result.message)
} catch (cause) {
setError(cause instanceof Error ? cause.message : '测试连接失败')
} finally {
setTesting(false)
}
}
const save = async (action: 'keep' | 'replace' | 'clear') => {
setSaving(true)
setError(null)
try {
const next = await saveWeKnoraConnection(environment, {
endpoint,
...(action === 'replace' ? { api_key: keyDraft.trim() } : {}),
key_action: action,
base_revision: connection?.revision ?? 0,
})
setConnection(next)
setEndpointDraft(next.endpoint)
setKeyDraft('')
setTestedEndpoint('')
setMessage(`已保存:${keyLabel(next)};最近一次测试:${verificationLabel(next)}。保存未触发任何同步或发布。`)
onSaved?.()
} catch (cause) {
setError(cause instanceof Error ? cause.message : '保存连接失败')
} finally {
setSaving(false)
}
}
const clearKey = () => {
if (!window.confirm('清除后该侧将没有任何可用密钥,发布与列库都会失败。确定清除?')) return
void save('clear')
}
const saveAction = keyEntered ? 'replace' : 'keep'
return (
<div className="weknora-connection-card" id={`settings-weknora-connection-${environment}`}>
<div className="kb-binding-grid">
<div className={`kb-binding-side ${connection?.verification === 'ok' ? 'bound' : 'warn'}`}>
<b>{title}</b>
<span>连接:{verificationLabel(connection)}</span>
<small>
{connection?.endpoint
? `地址:${connection.endpoint} · 密钥:${keyLabel(connection)}`
: '还没有配置地址与密钥'}
</small>
</div>
</div>
{connection && !connection.storage_ready ? (
<p className="hint">密钥存储不可用(主密钥文件缺失或权限不是 0600):可以读取当前状态,但保存会被拒绝。</p>
) : null}
{isAdmin ? (
<div className="kb-binding-editor">
<label>
{title}地址
<input
value={endpointDraft}
onChange={(event) => {
setEndpointDraft(event.target.value)
setTestedEndpoint('')
}}
placeholder={placeholder}
aria-label={`${title}地址`}
/>
</label>
<label>
API Key
<PasswordInput
autoComplete="new-password"
value={keyDraft}
onChange={(event) => {
setKeyDraft(event.target.value)
setTestedEndpoint('')
}}
placeholder={connection?.key_configured ? '留空 = 保留原密钥' : '粘贴 API Key'}
aria-label={`${title} API Key`}
/>
</label>
<p className="hint">
「测试连接」会真实请求该实例的知识库列表,并区分地址不可达 / 密钥无效 / 无权限 /
成功;换了地址或密钥必须先测通才能保存。撤销绑定请把下方目标库留空。
</p>
<div className="kb-binding-actions">
<button onClick={() => void test()} disabled={!canTest}>
{testing ? '正在测试…' : '测试连接'}
</button>
<button onClick={() => void save(saveAction)} disabled={!canSave}>
{saving ? '正在保存…' : '保存连接'}
</button>
{connection?.key_configured ? (
<button className="danger" onClick={clearKey} disabled={saving || testing}>
清除密钥
</button>
) : null}
</div>
{error ? (
<p className="error-message" role="alert">
{error}
</p>
) : null}
{message ? (
<p className="success-message" role="status">
{message}
</p>
) : null}
</div>
) : (
<p className="hint">仅管理员可修改连接与密钥;当前密钥只显示尾 4 位,任何接口都不会回读明文。</p>
)}
</div>
)
}
+4
View File
@@ -11,6 +11,8 @@ export interface ProductionBindings {
production_base_url: string
production_connected: boolean
production_message: string
frozen: number
retargetable: number
}
const record = (value: unknown): value is Record<string, unknown> => Boolean(value) && typeof value === 'object' && !Array.isArray(value)
@@ -25,6 +27,8 @@ function projection(item: Record<string, unknown>): ProductionBindings {
production_base_url: typeof item.production_base_url === 'string' ? item.production_base_url : '',
production_connected: item.production_connected === true,
production_message: typeof item.production_message === 'string' ? item.production_message : '',
frozen: typeof item.frozen === 'number' ? item.frozen : 0,
retargetable: typeof item.retargetable === 'number' ? item.retargetable : 0,
}
}
+4
View File
@@ -5,6 +5,8 @@ export interface RoutingBindings {
external_kb_id: string
internal_env_fallback: boolean
external_env_fallback: boolean
internal_fallback_kb_id: string
external_fallback_kb_id: string
internal_bound_by_ui: boolean
external_bound_by_ui: boolean
internal_kb_name: string
@@ -30,6 +32,8 @@ function bindingProjection(item: Record<string, unknown>): RoutingBindings {
external_kb_id: typeof item.external_kb_id === 'string' ? item.external_kb_id : '',
internal_env_fallback: item.internal_env_fallback === true,
external_env_fallback: item.external_env_fallback === true,
internal_fallback_kb_id: typeof item.internal_fallback_kb_id === 'string' ? item.internal_fallback_kb_id : '',
external_fallback_kb_id: typeof item.external_fallback_kb_id === 'string' ? item.external_fallback_kb_id : '',
internal_bound_by_ui: item.internal_bound_by_ui === true,
external_bound_by_ui: item.external_bound_by_ui === true,
internal_kb_name: typeof item.internal_kb_name === 'string' ? item.internal_kb_name : '',
+26
View File
@@ -882,3 +882,29 @@ button.link {
font-size: 0.8rem;
line-height: 1.5;
}
/* 连接卡:地址 + 写-only 密钥 + 真实列库测试。它嵌在既有的「测试/生产知识库绑定」
分区里,所以只做一个轻量内框,不另起一套分区样式。 */
.weknora-connection-card {
display: grid;
gap: 8px;
margin: 10px 0 16px;
padding: 12px;
border: 1px solid var(--border-base);
border-radius: 10px;
background: var(--bg-soft);
}
.weknora-connection-card .kb-binding-grid {
margin: 0;
grid-template-columns: 1fr;
}
.kb-binding-actions {
display: flex;
gap: 8px;
flex-wrap: wrap;
}
.kb-binding-actions button.danger {
border-color: var(--c-fail);
background: #fff;
color: var(--c-fail);
}
+116
View File
@@ -0,0 +1,116 @@
import { describe, expect, it, vi } from 'vitest'
import {
fetchWeKnoraConnections,
keyLabel,
saveWeKnoraConnection,
testWeKnoraConnection,
verificationLabel,
} from './weknoraConnections'
import type { WeKnoraConnection, WeKnoraVerification } from './weknoraConnections'
const connection: WeKnoraConnection = {
environment: 'production',
endpoint: 'http://172.29.151.11:18093',
key_configured: true,
key_tail: '9f2c',
verification: 'ok',
library_count: 4,
checked_at: '2026-11-22T07:00:00Z',
updated_by: 'admin',
revision: 3,
storage_ready: true,
}
describe('WeKnora connection API', () => {
it('projects only the safe fields and never surfaces a key', async () => {
vi.stubGlobal(
'fetch',
vi.fn().mockResolvedValue(
new Response(
JSON.stringify({
data: [{ ...connection, api_key: 'placeholder-key-value', key_ciphertext: 'AAAA' }],
}),
{ status: 200 }
)
)
)
const [projected] = await fetchWeKnoraConnections()
expect(projected.key_tail).toBe('9f2c')
expect(projected).not.toHaveProperty('api_key')
expect(projected).not.toHaveProperty('key_ciphertext')
vi.unstubAllGlobals()
})
it('renders the four answers distinctly', () => {
const withVerification = (verification: WeKnoraVerification): WeKnoraConnection => ({ ...connection, verification })
expect(verificationLabel({ ...connection, library_count: 4 })).toBe('可达 · 已读到 4 个知识库')
expect(verificationLabel(withVerification('key_invalid'))).toBe('密钥无效')
expect(verificationLabel(withVerification('key_forbidden'))).toBe('密钥无权限读取知识库列表')
expect(verificationLabel(withVerification('unreachable'))).toBe('地址不可达')
expect(verificationLabel(null)).toBe('未配置')
expect(keyLabel(connection)).toBe('已配置 · 尾9f2c')
expect(keyLabel({ ...connection, key_configured: false, key_tail: '' })).toBe('未配置')
})
it('tests through the environment route and keeps the outcome', async () => {
const fetchMock = vi.fn().mockResolvedValue(
new Response(
JSON.stringify({
data: {
environment: 'production',
endpoint: connection.endpoint,
outcome: 'key_forbidden',
library_count: 0,
message: '密钥无权限读取知识库列表(403)',
checked_at: '2026-11-22T07:00:00Z',
},
}),
{ status: 200 }
)
)
vi.stubGlobal('fetch', fetchMock)
const result = await testWeKnoraConnection('production', {
endpoint: connection.endpoint,
api_key: 'sk-live-x',
key_action: 'replace',
})
expect(result.outcome).toBe('key_forbidden')
expect(result.message).toContain('无权限')
expect(fetchMock.mock.calls[0][0]).toBe('/api/v1/weknora-connections/production/test')
expect(JSON.parse(fetchMock.mock.calls[0][1].body)).toMatchObject({ key_action: 'replace' })
vi.unstubAllGlobals()
})
it('saves with the revision and an idempotency key', async () => {
const fetchMock = vi.fn().mockResolvedValue(new Response(JSON.stringify({ data: connection }), { status: 200 }))
vi.stubGlobal('fetch', fetchMock)
const saved = await saveWeKnoraConnection('production', {
endpoint: connection.endpoint,
key_action: 'keep',
base_revision: 2,
})
expect(saved.revision).toBe(3)
const [url, init] = fetchMock.mock.calls[0]
expect(url).toBe('/api/v1/weknora-connections/production')
expect(init.method).toBe('PUT')
expect(init.headers['Idempotency-Key']).toBeTruthy()
expect(JSON.parse(init.body)).toMatchObject({ key_action: 'keep', base_revision: 2 })
vi.unstubAllGlobals()
})
it('surfaces the server reason when a save is refused', async () => {
vi.stubGlobal(
'fetch',
vi
.fn()
.mockResolvedValue(
new Response(JSON.stringify({ error: '请先点「测试连接」并成功,才能保存这一侧的配置' }), { status: 400 })
)
)
await expect(
saveWeKnoraConnection('test', { endpoint: 'http://127.0.0.1:8092', key_action: 'keep', base_revision: 0 })
).rejects.toThrow('测试连接')
vi.unstubAllGlobals()
})
})
+145
View File
@@ -0,0 +1,145 @@
import { newOperationID } from './operationKey'
// 连接状态:地址 + 「有没有配过密钥」+ 上一次真实列库的结论。密钥本身永远不回到
// 页面,只回显尾 4 位。
export type WeKnoraVerification = 'not_configured' | 'ok' | 'unreachable' | 'key_invalid' | 'key_forbidden'
export type WeKnoraTestOutcome = 'ok' | 'unreachable' | 'key_invalid' | 'key_forbidden'
export type WeKnoraEnvironment = 'test' | 'production'
export interface WeKnoraConnection {
environment: WeKnoraEnvironment
endpoint: string
key_configured: boolean
key_tail: string
verification: WeKnoraVerification
library_count: number
checked_at: string | null
updated_by: string
revision: number
storage_ready: boolean
}
export interface WeKnoraConnectionTestResult {
environment: WeKnoraEnvironment
endpoint: string
outcome: WeKnoraTestOutcome
library_count: number
message: string
checked_at: string
}
type KeyAction = 'keep' | 'replace' | 'clear'
const record = (value: unknown): value is Record<string, unknown> =>
Boolean(value) && typeof value === 'object' && !Array.isArray(value)
const verificationOf = (value: unknown): WeKnoraVerification =>
value === 'ok' ||
value === 'unreachable' ||
value === 'key_invalid' ||
value === 'key_forbidden' ||
value === 'not_configured'
? value
: 'not_configured'
function projection(item: Record<string, unknown>): WeKnoraConnection {
const environment = item.environment === 'production' ? 'production' : 'test'
return {
environment,
endpoint: typeof item.endpoint === 'string' ? item.endpoint : '',
key_configured: item.key_configured === true,
key_tail: typeof item.key_tail === 'string' ? item.key_tail : '',
verification: verificationOf(item.verification),
library_count: typeof item.library_count === 'number' ? item.library_count : 0,
checked_at: typeof item.checked_at === 'string' ? item.checked_at : null,
updated_by: typeof item.updated_by === 'string' ? item.updated_by : '',
revision: typeof item.revision === 'number' ? item.revision : 0,
storage_ready: item.storage_ready === true,
}
}
async function reason(response: Response, fallback: string): Promise<string> {
try {
const body = (await response.json()) as unknown
if (record(body) && typeof body.error === 'string' && body.error) return body.error
} catch {
/* keep the status text */
}
return `${fallback}(HTTP ${response.status})`
}
export async function fetchWeKnoraConnections(signal?: AbortSignal): Promise<WeKnoraConnection[]> {
const response = await fetch('/api/v1/weknora-connections', { signal, headers: { Accept: 'application/json' } })
if (!response.ok) throw new Error(await reason(response, '读取 WeKnora 连接失败'))
const value = (await response.json()) as unknown
if (!record(value) || !Array.isArray(value.data)) throw new Error('WeKnora 连接响应无效')
return value.data.filter(record).map(projection)
}
export async function testWeKnoraConnection(
environment: WeKnoraEnvironment,
input: { endpoint: string; api_key?: string; key_action?: KeyAction },
signal?: AbortSignal
): Promise<WeKnoraConnectionTestResult> {
const response = await fetch(`/api/v1/weknora-connections/${environment}/test`, {
method: 'POST',
signal,
headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
body: JSON.stringify(input),
})
if (!response.ok) throw new Error(await reason(response, '测试连接失败'))
const value = (await response.json()) as unknown
if (!record(value) || !record(value.data)) throw new Error('测试连接响应无效')
const data = value.data
return {
environment: data.environment === 'production' ? 'production' : 'test',
endpoint: typeof data.endpoint === 'string' ? data.endpoint : input.endpoint,
outcome:
data.outcome === 'key_invalid' || data.outcome === 'key_forbidden' || data.outcome === 'unreachable'
? data.outcome
: 'ok',
library_count: typeof data.library_count === 'number' ? data.library_count : 0,
message: typeof data.message === 'string' ? data.message : '',
checked_at: typeof data.checked_at === 'string' ? data.checked_at : '',
}
}
export async function saveWeKnoraConnection(
environment: WeKnoraEnvironment,
input: { endpoint: string; api_key?: string; key_action: KeyAction; base_revision: number },
signal?: AbortSignal
): Promise<WeKnoraConnection> {
const response = await fetch(`/api/v1/weknora-connections/${environment}`, {
method: 'PUT',
signal,
headers: { 'Content-Type': 'application/json', Accept: 'application/json', 'Idempotency-Key': newOperationID() },
body: JSON.stringify(input),
})
if (!response.ok) throw new Error(await reason(response, '保存连接失败'))
const value = (await response.json()) as unknown
if (!record(value) || !record(value.data)) throw new Error('保存连接响应无效')
return projection(value.data)
}
// verificationLabel 是页面唯一的口径:四态说清「哪里不通」,成功时带库数量。
export function verificationLabel(connection: WeKnoraConnection | null): string {
if (!connection || !connection.endpoint) return '未配置'
switch (connection.verification) {
case 'ok':
return `可达 · 已读到 ${connection.library_count} 个知识库`
case 'key_invalid':
return '密钥无效'
case 'key_forbidden':
return '密钥无权限读取知识库列表'
case 'unreachable':
return '地址不可达'
default:
// 填了地址但还没测过:说「尚未验证」,不要谎报不可达。
return '尚未验证'
}
}
export function keyLabel(connection: WeKnoraConnection | null): string {
if (!connection || !connection.key_configured) return '未配置'
return `已配置 · 尾${connection.key_tail || '—'}`
}