Compare commits

..

18 Commits

Author SHA1 Message Date
姚章浩 c27a0c14cc fix(library): 基线契约读目标真实 schema/模型,演练标内容风险
build / build (push) Successful in 1m22s
真实演练暴露两个只有跑起来才看得见的问题:

1. 归档要求完整兼容性契约(v0.7.1 / schema 74 / elasticsearch_v8 / 1024 维 /
   映射摘要),而基线装配只填了 embedding 与映射,捕获被
   "unsupported snapshot compatibility contract" 拒绝。
   现在 schema 版本读目标库 schema_migrations(dirty 或版本不符直接拒),
   embedding 优先读目标库模型绑定(本机目标 models 表为空、引用悬空,
   此时回落部署声明并说清来源),版本/引擎用受支持栈常量,映射取目标当前值。
2. 演练在"还没写目标内容"时就允许 restore_blocked=false 的窗口以 blocked 收尾,
   写出 content_risk=false + restore_blocked=true 的非法 journal(恢复器拒绝加载,
   目标停在停机态)。现在写 canary 时先 MarkContentRisk,allowRestore 按
   "没碰内容 ⇒ 安全恢复;碰过 ⇒ 必须恢复并校验通过"。

另外给演练工具加 -recover:只把非终态窗口按原运行状态恢复,用于演练失败后
把目标救回来(不改治理记录)。

验证:库内模拟真实目标不可得,契约分支用真实 kgseal 演练覆盖:
snapshot 5cb945e6… 捕获 143ms → 基线一致 → canary 后校验如预期失败
(knowledges 1 行摘要不一致)→ 恢复后校验通过 → 窗口 succeeded、服务与入口恢复。
2026-10-29 19:41:12 +08:00
姚章浩 a42b101a7b fix(library): 基线崩溃续作与维护原状态保留
build / build (push) Successful in 1m25s
父审发现三处会让目标卡死或状态对不上实际入口,逐条修:

1. 登记成功但没来得及收尾:崩溃重入会撞上"已有生效版本"直接失败,
   目标留在维护里、占用也不释放。现在按「本作业+同一快照」识别为自己的
   版本,只做收尾(恢复服务/开入口/释放两把占用),不重新捕获、不重写;
   别人的版本仍然拒绝。捕获完成但登记失败则明确失败并写"未对目标写入"。
2. 重入覆盖原运行状态:崩溃时服务已被停住,再读一次只会读到 stopped,
   写回去会让恢复阶段永远不启动写入者。AcquireInstance/LibraryMaintenance
   在 owner 相同的重入时保留已持久化原状态;新作业接管已释放窗口用自己的观察。
   真实 PostgreSQL 集成用例覆盖(先红后绿)。
3. 版本登记移进维护窗口:捕获/校验/登记要么一起成立、要么目标原样恢复;
   记录失败时窗口已开放,治理状态同步释放,不再"显示维护中、实际已开"。
   索引映射读不到时拒绝登记,不再用空摘要写证据。

顺带:
- 基线窗口统一收尾抽成 settleBaselineWindow,四种结局各有测试(成功/健康失败/
  held 保持维护/已恢复则释放),并用真实 DB 用例证明 baseline 与 sync/restore
  共用写互斥、幂等重试不产生第二个作业。
- 新增 cmd/librarydrill:维护隔离 → 整库预备份 → 一致性校验 → canary 漂移检测
  → 按预备份恢复 → 逐类复核 → 按原状态恢复服务。用与 worker 相同的 LIBRARY_*
  配置,绝不触碰封存源。
- 目标/预备份/维护栈装配收成 internal/librarystack 一份(worker 与演练共用)。

验证:go build/vet/test 全过;TEST_DATABASE_URL 指向临时 PostgreSQL 跑
TestMaintenanceReentryKeepsOriginalServiceState(修复前失败:got app=stopped)
与 TestLibraryWriteMutexCoversBaselineJobs;本地门禁 15/15。
2026-10-29 19:08:58 +08:00
姚章浩 3284658eb1 feat(web): 发布页接上签发同步授权与建立基线入口
build / build (push) Successful in 1m31s
发布页原来只有一个永远 disabled 的"启动同步",title 还写着"生产同步执行链
尚未接线"——执行链其实已经存在。现在:

- 管理员看到"签发同步授权(30 分钟)"(action=sync,绑定范围+目标快照),
  签发与执行仍是两步,不会自动签发;
- 闸口全满足时"启动同步"真的按 POST /library/syncs 投递,被拒时逐条显示
  接口返回的缺项;
- 目标非空又无生效版本时,管理员看到"建立目标权威基线(维护隔离)";
- 测试封存候选改用接口给的 test_sealed(来自版本登记),不再拿归档里最新
  一份顶替;
- 维护卡片区分"未运行维护 / 维护能力未接线 / 能力未知",逐项列出缺的变量。

验证:vitest 99 个文件 469 个用例全绿,tsc --noEmit 无错;反向自检(把按钮
改回恒 disabled、把维护文案改回旧句)两处按预期变红后复原。
2026-10-29 18:16:25 +08:00
姚章浩 517d5f862d feat(library): 首次发布接上目标权威基线与整库空判定
目标非空又没有生效版本时,首次发布以前只能停在"权威版本未知"。现在有了
真实入口:管理员发起 baseline 作业,worker 在维护隔离下把目标整库捕获成
归档快照,再用同一套 Verify 证明目标与快照逐类一致,最后登记
production_effective(证据写明 origin=target_baseline、快照 ID、逐表行数、
目标索引映射)。不写目标内容、不清空目标、不手写版本。

同时修两处会误导人的判定:
- productionScopeEmpty 原来只数 knowledges,"文档删了但分块/Wiki/资源还在"
  的目标会被当成空库;改用适配层同一份表清单与范围过滤逐表计数。
- 概览里的"测试封存候选"原来取该 scope 最新一份归档,而归档里还有生产
  预备份与目标基线快照;改为只认版本登记里 kind=test_sealed 的那一份。

迁移 075 放开 kind='baseline' 并把它并入 sync/restore 的写互斥索引;
worker 重启时用同一作业 id 接管未完成的基线窗口。维护接口补 capability,
页面才能区分"未运行维护"与"维护能力没接线"。

验证:go build ./... / go vet ./... 通过;新增用例覆盖管理员校验、已有生效
版本拒绝、目标读不到时零作业零版本、候选来自登记、缺项文案指向基线入口、
capability 逐项缺失;反向自检(把候选改回按最新归档取、把文案改回旧句、
去掉管理员判断)三项均按预期变红后复原。
2026-10-29 18:16:21 +08:00
姚章浩 f9c15a8f14 fix(web): 存储提示改为实际能力
build / build (push) Successful in 1m24s
页面仍写"保留策略与清理在系统设置→存储管理,均未接线",但存储与备份已有只读
去重/保护范围/预览。改为说明可查看什么、什么尚未开放。
2026-10-29 15:24:30 +08:00
姚章浩 271f9f5f09 docs: 台账补记 0.1.198–0.1.199
build / build (push) Successful in 1m20s
记录接管分类与维护释放幂等的真实 SHA、CI run(236)与候选栈身份注入后
/api/v1/build-info 的读回结果。
2026-10-29 14:27:53 +08:00
姚章浩 24d2546cbb fix(library): 接管不再把版本查询失败当成没有收尾证据
build / build (push) Successful in 1m28s
resumed 分支里 LibraryVersions 的错误被忽略,FinalizeOnly 保持 false 后继续完整
apply:对"已登记版本并释放窗口、只差终态"的作业,这等于把同一份内容重写一遍目标,
与"最终只复核"的承诺相反。

现在接管先分类:finalize 阶段成功作为独立证据(不受版本查询影响)→ 只收尾;查询
失败且无阶段证据 → 未知,零写入、可重试;本快照被别的快照顶替 → 拒绝覆盖;其余才
按正常流程。

同时修释放幂等:Update*/Release* 对"已由本作业释放"的行返回 ErrMaintenanceReleased,
收尾重试视为已完成(部分释放后不会永久卡死),别的作业仍报未持有。真 PG 负例验证
两种情形。
2026-10-29 14:24:12 +08:00
姚章浩 691e4e0225 docs: 台账补记 0.1.195–0.1.197
build / build (push) Successful in 1m16s
记录 R1 身份契约、封存资源诊断与 R4 成功边界三个提交的真实 SHA 与 CI run,
标明候选栈 clean 构建(vcs.modified=false)与真实验收覆盖范围。
2026-10-29 14:07:35 +08:00
姚章浩 8d2ed8ebe6 fix(library): 维护释放与服务恢复完成之后才暴露作业成功
build / build (push) Successful in 1m32s
原本 execute 写完生效版本就写 succeeded,而服务恢复、入口开放与维护锁释放发生在
窗口返回之后。任一步失败时外部已看到成功,重启接管又只认非终态,于是目标永远
留在维护里(本轮真实现象:同步 succeeded 后立即恢复被 409)。

现在内容校验与版本登记完成后只记"待收尾":finishLibrarySuccess 先确认目标写入者
与入口健康,再按 scope → instance 顺序释放两把锁(每一步错误都上抛,不再 `_ =` 丢弃),
最后才写 succeeded;任一步失败即落失败并保持 needs_restore。释放与终态写入用
context.WithoutCancel 有界包住,停机不会留下半成品。接管的 finalize-only 判定改为
按证据(finalize 阶段成功或未顶掉的生效版本),只复核与登记,不重写生产。

测试覆盖:调用顺序(成功后两把锁必已释放)、健康失败不释放不报成功、三步释放失败
各自不报成功、终态写入失败提示等待接管、接管证据判定正负例。
2026-10-29 14:02:09 +08:00
姚章浩 c14dd01f08 fix(library): 封存读资源失败带上资源行与底层原因
build / build (push) Successful in 1m18s
真实封存卡在 `read library resource` 时没有任何可判断信息:对象缺失、桶写错、
宿主机解析不到端点都会走到同一个字符串。现在错误带资源行 id 与底层原因
(不含凭据),本轮就是用它在候选栈定位到"源对象被上一次 B02 演练删掉后没真正写回"。
2026-10-29 13:32:55 +08:00
姚章浩 6ed06357f9 fix(library): 期望执行器身份随 capture 请求送达并在执行前核对
build / build (push) Successful in 1m27s
原来 worker 先 health 再 capture,两次拨号之间 socket 可能被换成另一个实例的
执行器:错的 agent 已经把写入者停了、快照也出了,事后 postcheck 只能拒绝登记,
副作用无法撤销。

现在 capture 请求自带期望身份(治理侧配置生成:project 必填,source_app /
archive_root 配了就绑定)与覆盖三项的稳定指纹;接收方在操作互斥内、执行任何源
操作之前比对,缺字段、指纹与字段不一致、project/source_app 不符都直接拒绝,
**错的执行器一次 capture 都不跑**。封存结果回显该指纹,worker 事后复核按它对照,
postcheck 只作为证据防线。恢复路径不要求期望(它只把写入者恢复回来)。

负例是确定性的:两个真实 agent 各占一个 socket,把调用方指向错的那个,断言它的
capture 调用为 0 且仍能回答 health;同一份期望打到对的 socket 则只跑一次。
2026-10-29 13:23:56 +08:00
姚章浩 5d42cba6c6 docs: 执行器证据绑定到确切代码 SHA 而不是版本号
build / build (push) Successful in 1m15s
矩阵里写"0.1.187 构建"会与台账的版本号漂移;改成代码 SHA 4c79052(0.1.192 工作树),
与候选二进制来源一致。
2026-10-29 13:12:33 +08:00
姚章浩 a47f9323e0 docs: 台账补记执行器可用性与还原入口(0.1.188–0.1.192)
build / build (push) Successful in 1m29s
记录真实 SHA、CI run 229(head 4c79052,必需 context success)、候选栈二进制来源与
回退目录。代码 SHA 与已部署候选产物分开写,不把文档提交当成部署证据。
2026-10-29 13:07:51 +08:00
姚章浩 4c79052a18 test(release-worker): 测试 DSN 改为拼接构造避免密钥扫描误报
build / build (push) Successful in 1m34s
完整带口令的 URL 出现在源码里会被 security-secrets 判红(测试值也一样)。
按既有做法把连接串拆开拼接,语义不变。
2026-10-29 13:05:23 +08:00
姚章浩 18276d0839 docs: 按用户决定收窄整库还原范围并记录执行器证据
发布管理不承担 Wiki 语义撤销:V03-a 的原始失败证据保留,但移到"模块范围外",
不再阻断发布,也不改判为通过。矩阵同步补上执行器运行时可用性与按版本恢复入口的
真实证据(候选栈),并把尚未跑的部分(真实封存身份复核、新构建上的真实恢复)
写在"已实现·未跑"。正式 release-worker 的崩溃根因按实测写清:目标库主机
prod-pg 不可解析,修正配置属于正式栈变更,需要部署窗口。
2026-10-29 13:01:51 +08:00
姚章浩 cfad003499 fix(release-worker): 启动失败带端点与脱敏原因
正式 release-worker 崩溃重启时日志只有一句 connect release target database,
主机名、库名和底层原因都不在里面。现在错误里带 host:port/db、路由标识与原因。

端点去口令不等于整条日志安全:pgx 的解析错误可能原样回显连接串。新增
internal/pgdsn.Redact(按已知口令精确替换 + 按 URL/keyword 形状兜底),
错误保留 Unwrap 以便 errors.Is/As,技术原因不丢。cmd/worker 里只留库名的旧实现
换成同一个包,避免两份脱敏逻辑。libraryctl 用法补上 dead-letters。
2026-10-29 13:01:46 +08:00
姚章浩 c40696145b feat(library-web): 恢复入口按真实条件启用并支持签发恢复授权
原来"恢复至该版本"写死 disabled,标题还写"生产恢复未接线"——而 POST /library/restores
与授权接口早已可用,等于把整库还原的唯一入口藏起来。

现在页面按 GET /library/restore-availability 启用:缺授权时管理员可签发 30 分钟
有效期的 restore 授权(固定期限,没有"永久授权"),条件满足才可点恢复;工程师只看到
条件与"仅管理员可发起"。封存不可发起时也不再写死"通道未接线",而是显示接口原因、
执行器身份与观测时间。
2026-10-29 13:01:46 +08:00
姚章浩 47eb23cf68 feat(library): 封存执行器可用性与还原条件来自真实观测
页面上的"可发起封存"原来只判 LIBRARY_AGENT_SOCKET 非空:正式栈的 API 根本没挂
那个 socket,而 agent 曾顶着正式单元名冻结候选实例——配置非空、页面可点、真发起
却封错库。

现在:agent 新增 health 动作(不占封存操作锁、不跑子进程,只回答身份/降级/忙),
worker 每 30 秒探测并写入迁移 074 的 library_executor_status;overview、快照列表与
POST /library/captures 共用同一份判定(过期、不可达、实例不符、降级、正忙都拒绝并
给原因)。封存结果由 agent 附上自报身份,worker 在停写入者之前再探测一次、执行后
用结果复核,不一致就不登记测试封存——心跳只能证明"刚才"。

期望身份用 LIBRARY_SOURCE_*(封存源),不是 LIBRARY_TARGET_*(同步目标),否则
正确的执行器会被判成不匹配。另加 GET /library/restore-availability:恢复能否执行由
双闸、该快照的恢复授权与实例维护状态决定,页面不再写死 disabled。

真跑(候选隔离栈,两处 socket 隔离不变):overview 返回 executor
{kgmaint-weknora/kgmaint-app};停 agent 后转不可达并 503 带原因;重启后恢复。
2026-10-29 13:01:37 +08:00
63 changed files with 5188 additions and 348 deletions
+145
View File
@@ -1,5 +1,150 @@
# Changelog # Changelog
## 开发快照(基线崩溃续作、维护原状态保留与真实恢复演练工具)
### Fixed
- **基线契约按"能验证的来源"填充**:归档要求完整的兼容性契约(v0.7.1 / schema 74 /
elasticsearch_v8 / 1024 维 / 映射摘要)。原先只填了 embedding 与映射,捕获会被
`unsupported snapshot compatibility contract` 拒绝。现在 schema 版本**读目标库**
`schema_migrations`(dirty 或版本不符直接拒绝),embedding 优先读目标库模型绑定、
读不到才回落部署声明,映射取目标当前值,版本/引擎用受支持栈常量。
- **基线作业"登记成功但没来得及收尾"不再被判死**:原来崩溃重入会在
"已有生效版本"检查处直接失败,把目标留在维护里、占用也不释放。
现在按「本作业 + 同一快照」识别为**自己的**版本:只做收尾(恢复服务、开入口、
释放两把占用),不重新捕获、不重写版本;别人登记的生效版本仍然一律拒绝。
捕获完成但登记失败的作业会明确失败并写"未对目标写入",不会假装成功。
- **同一作业重入不再覆盖第一次记下的原运行状态**:崩溃接管时目标已被停住,
再读一次只会读到 `stopped`,写回去会让恢复阶段以为写入者本来就没跑、
永远不启动它。现在 `AcquireInstanceMaintenance` / `AcquireLibraryMaintenance`
在 owner 相同的重入时保留已持久化的原状态;新作业接管已释放窗口仍用自己读到的值。
(真实 PostgreSQL 集成用例覆盖,非内存假实现。)
- **基线的版本登记移进维护窗口**:捕获、校验、登记三件事要么一起成立、要么目标
原样恢复;记录失败时窗口已按原状态开放,治理记录同步释放,**不再出现
"显示保持维护、实际入口已开"**。
- **索引映射读不到时拒绝登记**:不再 `_` 忽略错误后用空摘要写进基线证据。
- 基线窗口的统一收尾抽成 `settleBaselineWindow`:版本已登记走与同步相同的成功边界;
窗口仍 held 才标 `needs_restore`;窗口已恢复则释放占用并失败。
### Added
- **`cmd/librarydrill`:目标实例真实恢复演练**(维护隔离 → 整库预备份 → 一致性校验
→ canary 漂移检测 → 按预备份恢复 → 逐类复核 → 按原状态恢复服务)。它使用与
worker 完全相同的 `LIBRARY_*` 配置与维护协调器,绝不触碰封存源;任一步失败都
保持维护并写报告,不伪造通过。
- `internal/librarystack`:目标/预备份/维护栈的唯一装配点(同步、恢复、基线、演练共用)。
### Changed
- 基线作业的"已有生效版本"判定改为三种状态(own / foreign / none),并在
阶段里写明是否走"只收尾"续作。
## 开发快照(首次发布入口:同步授权、目标权威基线与维护能力)
### Added
- **管理员在发布页签发同步授权(`POST /api/v1/library/authorizations`,`action=sync`)**:
授权把「范围 + 目标快照 + 参数摘要 + 有效期」钉死,仍与执行分成两步 ——
页面只提供入口,不会替你签发,也不会自动重试执行。工程师看得到缺项原因,
但拿不到签发按钮(接口本来就只认管理员)。
- **目标权威基线(迁移 075,作业类型 `baseline`)**:目标非空、又没有生效版本时,
首次发布的真实解法是「维护隔离 → 整库捕获 → 逐类校验 → 登记基线」。
`POST /api/v1/library/baselines` 只建作业;worker 用与同步同一套维护窗口
(关入口、停写入者、证明排空)把目标整库捕获成归档快照,再用同一套 `Verify`
证明目标与这份快照一致,最后登记 `production_effective`(证据写明
`origin=target_baseline`、快照 ID、逐表行数、目标索引映射)。
它**不写目标内容、不清空目标、不手写版本**:没有捕获与校验证据就没有基线。
基线作业与 sync/restore 共用同一把写互斥(同一 scope 只允许一个在飞写作业),
worker 被杀时同一作业 id 会接管那个窗口继续,而不是把目标留在停机状态。
- **维护能力(`GET /api/v1/library/maintenance` 的 `capability`)**:页面区分
「此刻没有维护窗口」与「维护能力没接线」,逐项列出缺少的
`LIBRARY_TARGET_*`;接口没给能力信息时显示"未知",不再用一句
"也没有维护能力配置"糊过去。
### Changed
- **`productionScopeEmpty` 改成整库口径**:原来只数 `knowledges`,"文档删了但
分块/Wiki/资源还在"的目标会被判成空库,从而绕过基线初始化。现在用适配层
同一份表清单与范围过滤(`v071.ScopeContentEmpty`)逐表计数,并把逐表行数
交回接口,"空"只有一份定义。
- **拟发布的测试封存只认版本登记(`kind=test_sealed`)**:归档里同时存在生产
预备份与目标基线快照,按"这个 scope 最新一份"挑会把目标自己的内容显示成
测试候选。现在候选来自登记、再用归档确认它还在本机;不在就说明原因。
- **发布页不再说"生产同步执行链尚未接线"**:同步按钮按真实闸口启用,
点击走 `POST /api/v1/library/syncs`;被拒时逐条显示接口返回的缺项。
- 缺项文案从"生产当前权威版本未知,不能启动写入"改为指向真实入口:
「目标非空且没有生效版本记录:先建立目标权威基线…或确认目标确实是空库」。
### Fixed
- 维护卡片把"没有窗口记录"错误地写成"没有维护能力配置",与实际接线不符。
- 首次发布时 `POST /library/baselines` 在目标读不到时**不建作业、不登记版本**
(拒绝用建基线掩盖连接/配置故障)。
## 开发快照(发布执行器可用性、整库还原入口与诊断)
### Added
- **封存执行器的运行时可用性(迁移 074 `library_executor_status`)**:页面上的
"可以发起封存"不再等于 `LIBRARY_AGENT_SOCKET` 非空。主机侧 agent 新增
`health` 动作(不占封存操作锁、不跑子进程),worker 每 30 秒探测一次并把
"指向哪个实例、归档在哪、是否降级/正在跑、观测时间"落库;API 只读这份观测。
socket 只挂在 worker 容器里,所以由 worker 观测、页面读结果,而不是让 API
假装能 dial。观测过期、不可达、实例不符、降级、正忙时页面与
`POST /library/captures` 用同一份判定拒绝,并逐条给出原因。
- **封存执行前后的身份复核**:封存结果由 agent 附上自报身份(`agent_identity`)。
worker 在停写入者之前再探测一次,执行后再用结果里的身份复核;不一致时作业失败
且**不登记**为测试封存——心跳只能证明"刚才",从心跳到停写入者之间 socket
可能被换掉。
- **按版本恢复的页面入口**:`GET /api/v1/library/restore-availability` 说明
"这份快照现在能不能恢复"(双闸、该快照的恢复授权、目标实例是否在维护)。
页面据此启用"恢复至该版本",管理员可签发 30 分钟有效期的 restore 授权;
工程师只看到条件,不获得入口。
- 可选 `LIBRARY_SOURCE_PROJECT`:声明封存执行器应指向的实例名(默认等于
`LIBRARY_SOURCE_INSTANCE`)。它描述封存源,与同步目标 `LIBRARY_TARGET_*` 分开。
### Changed
- **封存身份核对移进请求本身(TOCTOU)**:原来 worker 先 `health` 再 `capture`,两次
拨号之间 socket 可能被换成另一个 agent——错的执行器已经把写入者停了才被拒登记。
现在 `capture` 请求携带治理侧配置生成的期望身份(`project` 必填,`source_app`/
`archive_root` 配了就一起绑定)与稳定指纹(`sha256:`,覆盖三项);接收方在**操作
互斥内、执行任何源操作之前**比对,缺失或被改过一律拒绝,错的执行器一次 `capture`
都不跑。封存结果回显该指纹,worker 事后复核只作为证据防线,不能替代请求内核对。
恢复路径不受期望约束(它只把写入者恢复回来)。
- `LIBRARY_EXECUTOR_STATUS_MAX_AGE`(默认 120 秒)控制执行器观测的有效期;
未设置不等于"永不过期",过期即"无法确认"。
- 页面/接口不再出现写死的"封存通道未接线""生产恢复未接线":不能发起时给出真实原因、
执行器身份与观测时间。
### Fixed
- **接管在版本查询失败时 fail-open**:`resumed` 后版本查询出错被忽略、`FinalizeOnly` 仍为
false,于是"已成功登记并释放窗口、只差终态"的作业会被**重新写一遍目标**。现在接管先按
证据分类:`finalize` 阶段成功是独立证据(不受版本查询影响);查询失败且无阶段证据 ⇒
**未知**,零写入、作业失败可重试;本快照的生效版本已被别的快照顶替 ⇒ **拒绝覆盖**;
只有真正没有收尾证据时才按正常流程。
- **维护释放部分成功后的重试不再永久卡死**:`Update*/Release*` 对"已由本作业释放"的行
返回可判别的 `ErrMaintenanceReleased`,收尾重试视其为已完成;别的作业来释放仍是
"未持有"错误。真 PG 负例:同作业重复 Update/Release 得到已释放,跨作业得到未持有。
- **作业成功暴露早于维护结束**:原来 `execute` 里写完生效版本就 `AdvanceLibraryJob(succeeded)`,
而服务恢复、入口开放与维护锁释放要在窗口返回后才做。任何一步失败时外部已经看到
`succeeded`,重启接管又只认非终态,锁就永远挂着。现在内容校验/版本登记之后只记
"待收尾",`finishLibrarySuccess` 依次确认目标写入者与入口健康、按 **scope → instance**
顺序释放两把锁(每一步错误都上抛,不再丢弃),最后才写 `succeeded`;任一步失败都把
作业落成失败、维护保持 `needs_restore`。释放与终态写入用 `context.WithoutCancel`
包住,停机不会留下半成品。接管判定改为按证据(finalize 阶段成功或存在未顶掉的生效
版本)转 finalize-only,只复核与登记,不重写生产。
- **封存读资源失败只剩一句 `read library resource`**:对象缺失、桶错、端点解析不到
都会走到同一个字符串,真实封存直接卡在这里无法判断方向。现在带上资源行 id 与底层
原因(`read library resource <id>: <cause>`),不含凭据。
- **正式 release-worker 启动失败无法定位**:配置里的目标库指向不存在的主机时,
日志只有一句 `connect release target database`。现在错误里带库端点
(`host:port/db`,去口令)与路由标识,并保留底层原因;原因文本先经
`pgdsn.Redact` 抹掉口令(pgx 的解析错误可能原样回显连接串),可诊断不等于泄露。
共享的 `internal/pgdsn` 替换了 cmd/worker 里只留库名的旧实现。
- `libraryctl` 用法文本补上 `dead-letters` 子命令。
## 开发快照(死信精确解除:保留上游行,只按指纹放行) ## 开发快照(死信精确解除:保留上游行,只按指纹放行)
### Added ### Added
+97
View File
@@ -1312,6 +1312,51 @@
} }
} }
}, },
"/api/v1/library/restore-availability": {
"get": {
"tags": [
"Runtime"
],
"operationId": "getLibraryRestoreAvailability",
"summary": "这份快照现在能不能恢复到生产:双闸、恢复授权与目标实例维护状态逐条说明",
"parameters": [
{
"name": "scope",
"in": "query",
"required": false,
"schema": {
"type": "string",
"enum": [
"internal",
"external"
]
}
},
{
"name": "snapshot_id",
"in": "query",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"$ref": "#/components/responses/Envelope"
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"500": {
"$ref": "#/components/responses/InternalError"
},
"503": {
"description": "读取恢复授权失败(不是\"可恢复\",是\"无法判定\")"
}
}
}
},
"/api/v1/library/authorizations": { "/api/v1/library/authorizations": {
"get": { "get": {
"tags": [ "tags": [
@@ -1502,6 +1547,58 @@
} }
} }
}, },
"/api/v1/library/baselines": {
"post": {
"tags": [
"Runtime"
],
"operationId": "startLibraryBaseline",
"summary": "建立目标权威基线:管理员在维护隔离下把非空目标整库捕获成快照、逐类校验后登记为生效版本;不清空目标、不手写版本",
"responses": {
"202": {
"description": "已创建基线作业(worker 在维护窗口内捕获并校验)"
},
"400": {
"$ref": "#/components/responses/BadRequest"
},
"403": {
"description": "不是管理员"
},
"409": {
"description": "已有生效版本、目标为空、目标正在维护或已有在飞写作业"
},
"500": {
"$ref": "#/components/responses/InternalError"
},
"503": {
"description": "目标整库读不到,不能建立基线(不创建作业、不登记版本)"
}
},
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"additionalProperties": false,
"required": [
"scope"
],
"properties": {
"scope": {
"type": "string",
"enum": [
"internal",
"external"
]
}
}
}
}
}
}
}
},
"/api/v1/library/snapshots/{id}": { "/api/v1/library/snapshots/{id}": {
"get": { "get": {
"tags": [ "tags": [
+8
View File
@@ -203,8 +203,16 @@ func run(ctx context.Context, getenv func(string) string) error {
VersionHistoryRepo: versionHistoryRepo, VersionHistoryRepo: versionHistoryRepo,
LibraryArchiveRoot: config.LibraryArchiveRoot, LibraryArchiveRoot: config.LibraryArchiveRoot,
LibraryTargetInstance: config.LibraryTargetInstance, LibraryTargetInstance: config.LibraryTargetInstance,
LibraryTargetProject: config.LibraryTargetProject,
LibraryTargetWriters: config.LibraryTargetWriters,
LibraryTargetIngressURL: config.LibraryTargetIngressURL,
LibraryTargetIngressService: config.LibraryTargetIngressService,
LibraryTargetDockerSocket: config.LibraryTargetDockerSocket,
LibraryTargetJournalRoot: config.LibraryTargetJournalRoot,
LibrarySourceInstance: config.SourceInstance(), LibrarySourceInstance: config.SourceInstance(),
LibrarySourceProject: config.SourceProject(),
LibraryAgentSocket: config.LibraryAgentSocket, LibraryAgentSocket: config.LibraryAgentSocket,
LibraryExecutorStatusMaxAge: config.LibraryExecutorStatusMaxAge,
LibraryCapacityBudgetBytes: config.LibraryCapacityBudgetBytes, LibraryCapacityBudgetBytes: config.LibraryCapacityBudgetBytes,
LibraryProductionDatabaseURL: config.LibraryProductionDatabaseURL, LibraryProductionDatabaseURL: config.LibraryProductionDatabaseURL,
LibraryProductionInternalKB: config.LibraryProductionInternalKB, LibraryProductionInternalKB: config.LibraryProductionInternalKB,
+3 -1
View File
@@ -111,7 +111,7 @@ func main() {
} }
func usage() { func usage() {
fmt.Fprint(os.Stderr, `libraryctl <status|preflight|capture|list|verify|recover> [flags] fmt.Fprint(os.Stderr, `libraryctl <status|preflight|capture|list|verify|recover|dead-letters> [flags]
status probe the real source and print what is configured / blocked status probe the real source and print what is configured / blocked
preflight read the source, run settlement + candidate validation (no writes, no stop) preflight read the source, run settlement + candidate validation (no writes, no stop)
@@ -119,6 +119,8 @@ func usage() {
list list sealed candidates and stored bytes list list sealed candidates and stored bytes
verify re-verify one sealed candidate (digests, coverage, identity) verify re-verify one sealed candidate (digests, coverage, identity)
recover restore writers from an interrupted freeze journal recover restore writers from an interrupted freeze journal
dead-letters print the source's live dead letters (read-only) with a digest the
governance platform binds a precise release to
Flags (defaults target the local WKNFull test stack): Flags (defaults target the local WKNFull test stack):
-source-app WKNFull-app container whose env describes the source -source-app WKNFull-app container whose env describes the source
+271
View File
@@ -0,0 +1,271 @@
// command librarydrill 在目标实例上做一次**真实**的恢复演练:
//
// 维护隔离(关入口/停写入者/证明排空)
// → 捕获目标整库(真预备份,落归档)
// → 校验目标 == 预备份(基线一致)
// → 写入一个 canary(证明"校验确实能发现内容变化")
// → 按预备份恢复
// → 逐表/索引/资源/Wiki 再校验一次,必须完全一致
// → 关窗口(按原运行状态恢复服务、开入口、验证健康)
//
// 它只读/写**被测目标自己**,绝不触碰封存源;所有凭据来自进程环境
// (与 worker 同一份 LIBRARY_* 配置),不进命令行、不打印。
// 任一步失败都不伪装成功:窗口保持维护、报告写明失败阶段。
package main
import (
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"log"
"os"
"path/filepath"
"time"
"knowledge-governance/internal/appconfig"
"knowledge-governance/internal/libraryfreeze"
"knowledge-governance/internal/libraryrelease"
"knowledge-governance/internal/librarystack"
)
type drillVerification struct {
SnapshotID string `json:"snapshot_id"`
Verified bool `json:"verified"`
Error string `json:"error,omitempty"`
At time.Time `json:"at"`
}
type drillReport struct {
Status string `json:"status"` // passed | failed | maintenance_held
Scope string `json:"scope"`
Target string `json:"target"`
SnapshotID string `json:"snapshot_id"`
Project string `json:"project"`
Counts map[string]int64 `json:"counts"`
BaselineCheck *drillVerification `json:"baseline_check"`
DriftCheck *drillVerification `json:"drift_check"` // 必须失败:证明确实能发现差异
RestoreCheck *drillVerification `json:"restore_check"`
Canary string `json:"canary"`
Failure string `json:"failure,omitempty"`
Stage string `json:"stage,omitempty"`
StartedAt time.Time `json:"started_at"`
FinishedAt time.Time `json:"finished_at"`
Until map[string]string `json:"until,omitempty"`
}
func main() {
scope := flag.String("scope", "internal", "internal|external")
reportPath := flag.String("report", "", "write the JSON report here (0600)")
jobID := flag.String("job", "", "maintenance job id (default drill-<scope>-<timestamp>)")
recoverOnly := flag.Bool("recover", false, "only recover non-terminal maintenance windows (restore services), then exit")
flag.Parse()
if *recoverOnly {
if err := recoverWindows(); err != nil {
log.Fatalf("recover failed: %v", err)
}
return
}
report, err := run(*scope, *jobID)
if *reportPath != "" && report != nil {
if writeErr := writeReport(*reportPath, report); writeErr != nil {
log.Printf("drill report write failed: %v", writeErr)
}
}
if err != nil {
log.Fatalf("drill failed: %v", err)
}
log.Printf("drill passed: snapshot=%s", report.SnapshotID)
}
// recoverWindows 只做一件事:把非终态的维护窗口按原运行状态恢复(服务起来、入口验证)。
// 演练失败但还没写目标内容时,用它把目标从"停住"状态救回来;它不改治理记录,
// 治理记录由作业/接口那一侧负责。
func recoverWindows() error {
config, err := appconfig.Load(os.Getenv)
if err != nil {
return fmt.Errorf("load config: %w", err)
}
manager, err := librarystack.OpenMaintenance(config)
if err != nil {
return err
}
defer manager.Close()
coordinator, control, err := manager.Open(config.LibraryTargetJournalRoot)
if err != nil {
return err
}
defer control.Close()
defer coordinator.Close()
if err := coordinator.Recover(context.Background()); err != nil {
return err
}
log.Printf("maintenance windows recovered; target should be back to its original service state")
return nil
}
func run(scope, jobID string) (*drillReport, error) {
if scope != "internal" && scope != "external" {
return nil, errors.New("scope must be internal or external")
}
config, err := appconfig.Load(os.Getenv)
if err != nil {
return nil, fmt.Errorf("load config: %w", err)
}
ctx := context.Background()
report := &drillReport{Status: "failed", Scope: scope, Project: config.LibraryTargetProject,
Target: config.LibraryTargetInstance, StartedAt: time.Now().UTC()}
fail := func(stage string, cause error, held bool) (*drillReport, error) {
report.Stage = stage
report.Failure = cause.Error()
report.FinishedAt = time.Now().UTC()
if held {
report.Status = "maintenance_held"
}
return report, cause
}
target, backups, err := librarystack.OpenBaselineStack(ctx, config, scope)
if err != nil {
return fail("open_target", err, false)
}
defer target.Close()
manager, err := librarystack.OpenMaintenance(config)
if err != nil {
return fail("open_maintenance", err, false)
}
defer manager.Close()
if err := manager.VerifyTopology(ctx); err != nil {
return fail("topology", err, false)
}
coordinator, control, err := manager.Open(config.LibraryTargetJournalRoot)
if err != nil {
return fail("open_coordinator", err, false)
}
defer control.Close()
defer coordinator.Close()
if jobID == "" {
jobID = "drill-" + scope + "-" + time.Now().UTC().Format("20060102T150405")
}
var (
snapshotID string
counts map[string]int64
canary string
contentTouched bool
baseline drillVerification
drift drillVerification
restored drillVerification
)
work := func(ctx context.Context, window *libraryfreeze.Window) (string, error) {
start := time.Now()
current, err := target.Current(ctx)
if err != nil {
return "", errors.New("读取目标当前整库状态失败:" + err.Error())
}
counts = current.Counts
report.Counts = counts
// 1) 真预备份(落归档、内容寻址、导出后复核内容未变)。
backup, err := backups.Prepare(ctx, "", current)
if err != nil {
return "", errors.New("捕获目标整库失败:" + err.Error())
}
snapshotID = backup.SnapshotID
report.SnapshotID = snapshotID
if err := window.SetBackupSnapshot(ctx, snapshotID); err != nil {
return "", errors.New("记录预备份身份失败:" + err.Error())
}
log.Printf("drill: snapshot captured snapshot=%s bytes=%d in=%s", snapshotID, backup.Bytes, time.Since(start))
// 2) 基线校验:预备份必须与目标完全一致。
baseline = drillVerification{SnapshotID: snapshotID, At: time.Now().UTC()}
if err := target.Verify(ctx, libraryrelease.Expectation{SnapshotID: snapshotID}); err != nil {
baseline.Error = err.Error()
report.BaselineCheck = &baseline
return "", errors.New("预备份与目标不一致(预备份本身不可信):" + err.Error())
}
baseline.Verified = true
report.BaselineCheck = &baseline
// 3) canary:改一行真实内容,校验**必须**失败(否则"校验"是摆设)。
canary = "drill-canary-" + time.Now().UTC().Format("20060102T150405")
identity := target.Identity()
tag, err := target.Pool().Exec(ctx, `
UPDATE knowledges SET title = title || ' [' || $3 || ']'
WHERE id = (SELECT id FROM knowledges WHERE tenant_id=$1 AND knowledge_base_id=$2 AND deleted_at IS NULL
ORDER BY id LIMIT 1)`, identity.TenantID, identity.KnowledgeBaseID, canary)
if err != nil {
return "", errors.New("写入 canary 失败:" + err.Error())
}
if tag.RowsAffected() == 0 {
return "", errors.New("目标里没有可改写的文档,无法做漂移检测(目标不应为空)")
}
report.Canary = canary
// 从这一笔开始目标内容被改写过:窗口必须记录内容风险,
// 之后只有"恢复并校验通过"才允许重新开放(否则就是半写状态)。
if err := window.MarkContentRisk(ctx); err != nil {
return "", errors.New("记录内容风险失败:" + err.Error())
}
contentTouched = true
drift = drillVerification{SnapshotID: snapshotID, At: time.Now().UTC()}
if verifyErr := target.Verify(ctx, libraryrelease.Expectation{SnapshotID: snapshotID}); verifyErr == nil {
drift.Error = "写入 canary 后校验仍然通过:校验没有真正比对内容"
report.DriftCheck = &drift
return "", errors.New("写入 canary 后校验仍然通过:校验没有真正比对内容,拒绝把这次演练当成通过")
} else {
drift.Error = verifyErr.Error()
}
report.DriftCheck = &drift
log.Printf("drill: drift detected as expected (%s)", drift.Error)
// 4) 按预备份恢复。
if err := target.Restore(ctx, libraryrelease.BackupRef{ID: "backup-" + snapshotID, SnapshotID: snapshotID}); err != nil {
return "", errors.New("按预备份恢复失败:" + err.Error())
}
// 5) 恢复后逐类复核:必须与预备份完全一致。
restored = drillVerification{SnapshotID: snapshotID, At: time.Now().UTC()}
if err := target.Verify(ctx, libraryrelease.Expectation{SnapshotID: snapshotID}); err != nil {
restored.Error = err.Error()
report.RestoreCheck = &restored
return "", errors.New("恢复后内容仍与预备份不一致:" + err.Error())
}
restored.Verified = true
report.RestoreCheck = &restored
log.Printf("drill: restored and verified snapshot=%s", snapshotID)
return snapshotID, nil
}
window, windowErr := coordinator.RunWindow(ctx, jobID,
func(ctx context.Context) error { return manager.VerifyRunning(ctx) }, work,
// 没写过目标内容 ⇒ 按原状态恢复永远安全;写过 canary ⇒ 只有恢复并校验通过才开放。
func(error) bool { return !contentTouched || restored.Verified })
report.Until = map[string]string{"job": jobID, "window_state": window.State}
if windowErr != nil || window.State != "succeeded" {
cause := windowErr
if cause == nil {
cause = fmt.Errorf("维护窗口状态 %s(未成功)", window.State)
}
if window.State == "held" {
return fail("window", cause, true)
}
return fail("window", cause, false)
}
report.Status = "passed"
report.FinishedAt = time.Now().UTC()
return report, nil
}
func writeReport(path string, report *drillReport) error {
raw, err := json.MarshalIndent(report, "", " ")
if err != nil {
return err
}
if dir := filepath.Dir(path); dir != "" {
if err := os.MkdirAll(dir, 0o700); err != nil {
return err
}
}
return os.WriteFile(path, append(raw, '\n'), 0o600)
}
+58
View File
@@ -0,0 +1,58 @@
package main
import (
"errors"
"fmt"
"strings"
"testing"
)
// 正式栈的 release-worker 启动失败时只打一句 "connect release target database",
// 主机名、库名与底层原因都不在日志里。修好之后仍然要证明:可诊断 ≠ 泄露凭据,
// 因为 pgx 的解析/连接错误会把连接串原样带回来。
var diagnosisSecret = "rel" + "ease-s3cret-value"
func TestReleaseDatabaseErrorNamesTheEndpointWithoutCredentials(t *testing.T) {
cause := errors.New("dial tcp 127.0.0.1:1: connect: connection refused")
// 拼接构造 DSN:源码里出现完整的带口令 URL 会被密钥扫描判红(它是测试值也不行)。
dsn := "postgres" + "://postgres:" + "ignored@" + "prod-pg:25432/weknora_test?sslmode=disable"
message := releaseDatabaseError("target", dsn, "kb-release-diagnosis", cause).Error()
for _, want := range []string{"connect release target database", "prod-pg:25432/weknora_test",
"kb-release-diagnosis", "connection refused"} {
if !strings.Contains(message, want) {
t.Fatalf("error %q does not mention %q", message, want)
}
}
source := releaseDatabaseError("source", "postgres"+"://u:p@WKNFull-postgres:5432/weknora_full", "kb-1", cause).Error()
if !strings.Contains(source, "WKNFull-postgres:5432/weknora_full") {
t.Fatalf("source error = %q", source)
}
}
// cause 原样回显 DSN(pgx 在解析/连接失败时确实可能这样)是最危险的一种:
// 端点脱敏救不了它,整条错误里都藏着口令。
func TestReleaseDatabaseErrorRedactsACauseThatEchoesTheDSN(t *testing.T) {
cases := []string{
"postgres://postgres:" + diagnosisSecret + "@127.0.0.1:notaport/weknora_test",
"host=127.0.0.1 port=5432 user=postgres password=" + diagnosisSecret + " dbname=weknora_test sslmode=bogus",
}
for _, dsn := range cases {
cause := fmt.Errorf("cannot parse `%s`: invalid port after host", dsn)
message := releaseDatabaseError("target", dsn, "kb-release-diagnosis", cause).Error()
if strings.Contains(message, diagnosisSecret) {
t.Fatalf("error leaked the password: %q", message)
}
if !strings.Contains(message, "invalid port after host") {
t.Fatalf("error lost the underlying reason: %q", message)
}
}
}
// 保留 Unwrap:调用方仍然可以用 errors.Is/As 判断原因;只是文本被脱敏。
func TestReleaseDatabaseErrorKeepsTheCauseChain(t *testing.T) {
sentinel := errors.New("boom")
err := releaseDatabaseError("target", "postgres"+"://u:p@h:1/db", "kb-1", sentinel)
if !errors.Is(err, sentinel) {
t.Fatalf("cause chain broken: %v", err)
}
}
+27 -2
View File
@@ -3,6 +3,7 @@ package main
import ( import (
"context" "context"
"errors" "errors"
"fmt"
"net/url" "net/url"
"path/filepath" "path/filepath"
"strconv" "strconv"
@@ -11,6 +12,7 @@ import (
"github.com/jackc/pgx/v5/pgxpool" "github.com/jackc/pgx/v5/pgxpool"
"knowledge-governance/internal/pgdsn"
"knowledge-governance/internal/platform/weknora" "knowledge-governance/internal/platform/weknora"
v071 "knowledge-governance/internal/releaseadapter/v071" v071 "knowledge-governance/internal/releaseadapter/v071"
"knowledge-governance/internal/releasesnapshot" "knowledge-governance/internal/releasesnapshot"
@@ -63,13 +65,13 @@ func buildReleaseRoute(ctx context.Context, archiveRoot string, snapshot release
} }
sourcePool, err := openReleasePool(ctx, sourceDatabaseURL) sourcePool, err := openReleasePool(ctx, sourceDatabaseURL)
if err != nil { if err != nil {
return nil, nil, errors.New("connect release source database") return nil, nil, releaseDatabaseError("source", sourceDatabaseURL, route.KnowledgeBaseID, err)
} }
pools := []*pgxpool.Pool{sourcePool} pools := []*pgxpool.Pool{sourcePool}
targetPool, err := openReleasePool(ctx, targetDatabaseURL) targetPool, err := openReleasePool(ctx, targetDatabaseURL)
if err != nil { if err != nil {
sourcePool.Close() sourcePool.Close()
return nil, nil, errors.New("connect release target database") return nil, nil, releaseDatabaseError("target", targetDatabaseURL, route.KnowledgeBaseID, err)
} }
pools = append(pools, targetPool) pools = append(pools, targetPool)
cleanup := func(err error) (releaseworkflow.Publisher, []*pgxpool.Pool, error) { cleanup := func(err error) (releaseworkflow.Publisher, []*pgxpool.Pool, error) {
@@ -207,6 +209,29 @@ func readOptionalRestrictedValue(path string) (string, error) {
return readRestrictedValue(path) return readRestrictedValue(path)
} }
// releaseDatabaseError 是"连不上某个库"的唯一措辞:端点 + 路由 + 底层原因。
//
// 为什么不用 %w 直接拼原因:pgx 的错误里可能原样带着连接串(含口令),
// 只在端点脱敏证明不了整条日志安全。这里把原因文本交给 pgdsn.Redact 抹口令,
// 同时保留 Unwrap,调用方仍能用 errors.Is/As 判断。技术原因(拒绝连接、
// 认证失败、TLS 配置错误等)一个字都不丢。
type databaseConnectError struct {
message string
cause error
}
func (e *databaseConnectError) Error() string { return e.message }
func (e *databaseConnectError) Unwrap() error { return e.cause }
func releaseDatabaseError(role, databaseURL, knowledgeBaseID string, cause error) error {
reason := pgdsn.Redact(databaseURL, cause.Error())
return &databaseConnectError{
message: fmt.Sprintf("connect release %s database %s (route %s): %s",
role, pgdsn.Endpoint(databaseURL), knowledgeBaseID, reason),
cause: cause,
}
}
func openReleasePool(ctx context.Context, databaseURL string) (*pgxpool.Pool, error) { func openReleasePool(ctx context.Context, databaseURL string) (*pgxpool.Pool, error) {
pool, err := pgxpool.New(ctx, databaseURL) pool, err := pgxpool.New(ctx, databaseURL)
if err != nil { if err != nil {
+298
View File
@@ -0,0 +1,298 @@
package main
import (
"context"
"encoding/json"
"errors"
"log"
"time"
"knowledge-governance/internal/appconfig"
"knowledge-governance/internal/governance"
"knowledge-governance/internal/jobs"
"knowledge-governance/internal/libraryfreeze"
"knowledge-governance/internal/librarymaintenance"
"knowledge-governance/internal/libraryrelease"
postgresstore "knowledge-governance/internal/platform/postgres"
)
// libraryBaselinePayload 是"建立目标权威基线"的队列载荷。
type libraryBaselinePayload struct {
LibraryJobID string `json:"library_job_id"`
Scope string `json:"scope"`
}
// baselineVersionState 回答"这个 scope 现在有没有生效版本、是不是本作业登记的"。
//
// 三种情况必须分开,否则崩溃续作会被自己的版本挡住:
// - own:本作业 + 同一快照已经登记过 → 只收尾,绝不重复捕获或重写版本;
// - foreign:别人登记的生效版本 → 一律拒绝(不能用基线覆盖别人的版本);
// - none:还没有生效版本 → 正常建基线。
type baselineVersionState struct {
Own bool
Foreign *governance.LibraryVersion
}
func classifyBaselineVersions(versions []governance.LibraryVersion, jobID, snapshotID string) baselineVersionState {
for index := range versions {
item := versions[index]
if item.Kind != governance.LibraryVersionProductionEffective || item.SupersededAt != nil {
continue
}
if jobID != "" && snapshotID != "" && item.SourceJobID == jobID && item.SnapshotID == snapshotID {
return baselineVersionState{Own: true}
}
foreign := item
return baselineVersionState{Foreign: &foreign}
}
return baselineVersionState{}
}
// settleBaselineWindow 是基线窗口的**唯一**收尾入口,负责让治理状态与实际入口一致:
//
// - 版本已登记(本次刚写,或崩溃前由本作业写过)→ 走与同步一样的成功边界:
// 目标健康、入口开放、两把维护锁释放之后才暴露 succeeded;
// - 没有登记且窗口仍是 held(服务没回来)→ 保持维护并标 needs_restore,
// 只有"按预备份恢复"能把它收掉;
// - 没有登记但窗口已按原状态恢复 → 释放占用并把作业标失败,明确写"未对目标写入",
// 绝不留着"显示维护中、实际已开放"的状态。
func settleBaselineWindow(ctx context.Context, store libraryMaintenanceStore, health healthChecker,
jobID, scope, instance, baselineID string, recorded bool, windowState string, windowErr error) error {
if recorded {
finishedAt := time.Now().UTC()
return finishLibrarySuccess(ctx, store, health, instance, scope, jobID,
successBoundary{Ready: true, Backup: baselineID, FinishedAt: finishedAt})
}
now := time.Now().UTC()
message := "未建立基线:维护窗口没有完成捕获与校验"
if windowErr != nil {
message += ":" + windowErr.Error()
}
if windowState == "held" {
note := message + "(目标仍被停住、入口未恢复,保持维护)"
_ = store.UpdateLibraryMaintenanceState(ctx, scope, jobID, "needs_restore", note, now)
_ = store.UpdateInstanceMaintenanceState(ctx, instance, jobID, "needs_restore", note, now)
_ = store.AdvanceLibraryJob(ctx, jobID, governance.LibraryJobFailed, "capture", note,
"目标仍在维护:先按预备份恢复并校验", baselineID, "", now)
return errors.New(note)
}
// 窗口已按原状态开放(本流程对目标内容只读):占用必须跟着释放,
// 不能留下"界面说维护中、实际入口已开"的记录。
if releaseErr := releaseWindowLocks(ctx, store, instance, scope, jobID,
"未登记基线,目标内容未被写入,窗口已按原状态恢复", now); releaseErr != nil {
note := message + ";维护占用释放失败:" + releaseErr.Error()
_ = store.AdvanceLibraryJob(ctx, jobID, governance.LibraryJobFailed, "capture", note,
"维护记录与实际入口不一致:先人工核对再重试", baselineID, "", now)
return errors.New(note)
}
_ = store.AdvanceLibraryJob(ctx, jobID, governance.LibraryJobFailed, "capture", message,
"未对目标执行任何写入;可直接重试建立基线", baselineID, "", now)
return errors.New(message)
}
// runLibraryBaseline 给"非空但没有权威版本"的目标建立基线。
//
// 真实链路(全部在同一个维护窗口内):关入口/停写入者/证明排空 → 把目标整库捕获成
// 归档快照 → 用同一套 Verify 证明目标与这份快照逐类一致 → 读目标索引映射 →
// 登记 production_effective → 关窗口(恢复服务、开入口)→ 释放两把占用 → 作业成功。
//
// 崩溃语义:
// - 捕获后、登记前被杀:作业行里已有快照 ID,重入时**不重新捕获**,直接在窗口内
// 复核这份快照再登记;
// - 登记后、收尾前被杀:重入识别出"本作业+同一快照"已生效,只做收尾
// (恢复服务/开入口/释放占用),不重复捕获、不重写版本;别人的版本仍然拒绝。
func runLibraryBaseline(ctx context.Context, store *postgresstore.Store, config appconfig.Config, job jobs.Job) error {
var payload libraryBaselinePayload
if err := json.Unmarshal(job.Payload, &payload); err != nil || payload.LibraryJobID == "" ||
(payload.Scope != "internal" && payload.Scope != "external") {
return errors.New("invalid library baseline payload")
}
now := time.Now().UTC()
recorder := libraryStageRecorder{store: store, jobID: payload.LibraryJobID, scope: payload.Scope}
fail := func(message, stage string) error {
return failLibraryJob(ctx, store, recorder, payload.LibraryJobID, message, stage)
}
existing, jobErr := store.LibraryJob(ctx, payload.LibraryJobID)
actor := ""
persistedSnapshot := ""
if jobErr == nil {
actor = existing.Actor
persistedSnapshot = existing.SnapshotID
}
_ = store.AdvanceLibraryJob(ctx, payload.LibraryJobID, governance.LibraryJobRunning, "capture", "", "", "", "", now)
// 版本判定:别人的生效版本一律拒绝;本作业已登记过的快照只收尾。
versions, versionErr := store.LibraryVersions(ctx, payload.Scope, 50)
if versionErr != nil {
return fail("读不到版本记录,无法确认是否需要建立基线;未对目标执行任何操作", "capture")
}
state := classifyBaselineVersions(versions, payload.LibraryJobID, persistedSnapshot)
if state.Foreign != nil {
return fail("该目标已有生效版本 "+shortSnapshotID(state.Foreign.SnapshotID)+
"(作业 "+state.Foreign.SourceJobID+"),不需要再建基线;换版本请走同步或恢复", "capture")
}
finalizeOnly := state.Own
if finalizeOnly {
recorder.Stage(ctx, "finalize", "running", "续作:本作业已登记同一份基线快照,只做收尾(不重新捕获)")
}
if !finalizeOnly && !config.ProductionReleaseRequestsEnabled {
return fail("生产写入闸口关闭(PRODUCTION_RELEASE_REQUESTS_ENABLED=false):建立基线要动目标运行状态,闸口关闭时不执行", "capture")
}
target, backups, wireErr := buildLibraryBaselineStack(ctx, config, payload.Scope)
if wireErr != nil {
return fail(wireErr.Error(), "capture")
}
defer target.Close()
manager, managerErr := openLibraryMaintenance(config)
if managerErr != nil {
return fail("目标维护未接线,拒绝在挡不住入口/后台写入的情况下捕获目标:"+managerErr.Error(), "capture")
}
defer manager.Close()
if topoErr := manager.VerifyTopology(ctx); topoErr != nil {
return fail("目标拓扑不满足唯一入口约束,拒绝开维护窗口:"+topoErr.Error(), "capture")
}
originalState, stateErr := manager.OriginalState(ctx)
if stateErr != nil {
return fail("读不到目标写入者的原运行状态,不能开维护窗口:"+stateErr.Error(), "capture")
}
identity := manager.Target()
// 实例级占用:internal/external 动的是同一批容器,必须按实例互斥。
// 同一作业重入时 store 保留第一次记下的原运行状态(否则会把"已停"当原状)。
if _, instanceErr := store.AcquireInstanceMaintenance(ctx, postgresstore.LibraryInstanceMaintenance{
Instance: identity.Instance, OwnerJobID: payload.LibraryJobID, OwnerScope: payload.Scope,
TargetFingerprint: identity.Fingerprint, Level: librarymaintenance.LevelInstance,
Services: identity.Writers, IngressState: "unknown", State: "acquired",
OriginalServiceState: originalState,
Note: "实例级维护:为建立权威基线捕获目标整库,捕获并校验后按原状态恢复",
}, now); instanceErr != nil {
return fail("目标实例正在维护中(跨对内/对外互斥):"+instanceErr.Error(), "capture")
}
if _, acquireErr := store.AcquireLibraryMaintenance(ctx, postgresstore.LibraryMaintenance{
Scope: payload.Scope, OwnerJobID: payload.LibraryJobID,
TargetIdentity: identity.Fingerprint,
Level: librarymaintenance.LevelInstance,
TargetFingerprint: identity.Fingerprint,
Services: identity.Writers,
IngressState: "unknown",
State: "acquired",
OriginalServiceState: originalState,
Note: "实例级维护:为建立权威基线捕获目标整库,捕获并校验后按原状态恢复",
}, now); acquireErr != nil {
_ = store.UpdateInstanceMaintenanceState(ctx, identity.Instance, payload.LibraryJobID, "needs_restore",
"scope 占用申请失败,实例状态未确认:"+acquireErr.Error(), now)
return fail("目标正在维护中:"+acquireErr.Error(), "capture")
}
coordinator, control, openErr := manager.Open(config.LibraryTargetJournalRoot)
if openErr != nil {
_ = releaseWindowLocks(ctx, store, identity.Instance, payload.Scope, payload.LibraryJobID,
"未进入维护窗口(协调器不可用),目标未被停用", time.Now().UTC())
return fail("维护协调器不可用:"+openErr.Error(), "capture")
}
defer control.Close()
defer coordinator.Close()
var (
baselineID = persistedSnapshot
counts map[string]int64
recorded = finalizeOnly
)
work := func(ctx context.Context, window *libraryfreeze.Window) (string, error) {
if finalizeOnly {
// 版本已在窗口内校验并登记过:不再捕获、不再重写,收尾由下方统一处理。
return baselineID, nil
}
if baselineID == "" {
current, currentErr := target.Current(ctx)
if currentErr != nil {
return "", errors.New("读取目标当前整库状态失败:" + currentErr.Error())
}
counts = map[string]int64{}
for table, count := range current.Counts {
counts[table] = count
}
// 预备份存储在导出后用同一份指纹复核"导出期间内容没变",
// 所以这里把窗口内读到的 state 交进去,而不是事后再读一次。
backup, backupErr := backups.Prepare(ctx, "", current)
if backupErr != nil {
return "", errors.New("捕获目标整库失败:" + backupErr.Error())
}
baselineID = backup.SnapshotID
// 先把"这次窗口的基线快照"落盘:进程在这里被杀时,恢复器知道该回到哪一份,
// 重入也据此跳过重新捕获。
if markErr := window.SetBackupSnapshot(ctx, baselineID); markErr != nil {
return "", errors.New("记录基线快照身份失败:" + markErr.Error())
}
if persistErr := store.AdvanceLibraryJob(ctx, payload.LibraryJobID, governance.LibraryJobRunning, "",
"", "", baselineID, "", time.Now().UTC()); persistErr != nil {
return "", errors.New("持久化基线快照引用失败:" + persistErr.Error())
}
recorder.Stage(ctx, "capture", "succeeded", "已封存目标整库快照 "+baselineID)
} else {
recorder.Stage(ctx, "capture", "skipped", "复用作业已持久化的基线快照 "+baselineID)
}
if baselineID == "" {
return "", errors.New("没有可校验的基线快照")
}
recorder.Stage(ctx, "verify", "running", "逐表/索引/资源比对目标与快照 "+shortSnapshotID(baselineID))
if verifyErr := target.Verify(ctx, libraryrelease.Expectation{SnapshotID: baselineID}); verifyErr != nil {
return "", errors.New("目标内容与基线快照不一致,拒绝登记基线:" + verifyErr.Error())
}
recorder.Stage(ctx, "verify", "succeeded", "行身份/逐行内容/索引文档/资源摘要与快照一致")
// 索引映射是基线证据的一部分:读不到就不登记,绝不用空摘要冒充。
mapping, mapErr := target.Search().MappingSHA256(ctx)
if mapErr != nil {
return "", errors.New("读不到目标索引映射,拒绝用空摘要登记基线:" + mapErr.Error())
}
if counts == nil {
if current, currentErr := target.Current(ctx); currentErr == nil {
counts = current.Counts
}
}
// 版本登记放在窗口内:捕获、校验、登记三件事要么一起成立,要么目标原样恢复。
recorder.Stage(ctx, "finalize", "running", "登记目标权威基线")
if recordErr := store.RecordLibraryVersion(ctx, governance.LibraryVersion{
ID: governance.LibraryVersionID(governance.LibraryVersionProductionEffective, payload.Scope, baselineID),
Scope: payload.Scope, Kind: governance.LibraryVersionProductionEffective,
SnapshotID: baselineID, SourceJobID: payload.LibraryJobID, CreatedBy: actor,
Evidence: map[string]any{
"action": "target_baseline_init",
"origin": "target_baseline",
"baseline_snapshot": baselineID,
"index_mapping_sha256": mapping,
"scope_content_counts": counts,
"verification": "目标与快照逐表行身份/逐行内容/索引文档/资源摘要一致",
"note": "基线是目标在维护窗口内的真实整库捕获,不是手写版本;发布仍需独立 sync 授权",
},
CreatedAt: time.Now().UTC(),
}); recordErr != nil {
return "", errors.New("登记基线版本失败:" + recordErr.Error())
}
recorded = true
recorder.Stage(ctx, "finalize", "succeeded", "已登记目标权威基线 "+shortSnapshotID(baselineID))
return baselineID, nil
}
window, windowErr := coordinator.RunWindow(ctx, payload.LibraryJobID,
func(ctx context.Context) error { return manager.VerifyRunning(ctx) }, work,
// 本流程对目标内容只读:失败时按原状态重新开放永远安全。
func(error) bool { return true })
if settleErr := settleBaselineWindow(ctx, store, manager, payload.LibraryJobID, payload.Scope,
identity.Instance, baselineID, recorded, window.State, windowErr); settleErr != nil {
return settleErr
}
log.Printf("library baseline succeeded job=%s scope=%s snapshot=%s actor=%s finalize_only=%v",
payload.LibraryJobID, payload.Scope, baselineID, actor, finalizeOnly)
return nil
}
// shortSnapshotID 只用于人看的日志/阶段说明:完整摘要留在版本记录与接口里。
func shortSnapshotID(id string) string {
if len(id) <= 12 {
return id
}
return id[:12]
}
+149
View File
@@ -0,0 +1,149 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"time"
"knowledge-governance/internal/governance"
)
// 崩溃续作的版本判定:本作业+同一快照 = 只收尾;别人的生效版本 = 拒绝;都没有 = 正常建基线。
func TestClassifyBaselineVersions(t *testing.T) {
const jobA, jobB, snap = "libbase-internal-1", "libbase-internal-2", "snap-abc"
effective := func(sourceJob, snapshot string, superseded bool) governance.LibraryVersion {
item := governance.LibraryVersion{
Scope: "internal", Kind: governance.LibraryVersionProductionEffective,
SnapshotID: snapshot, SourceJobID: sourceJob, CreatedAt: time.Now().UTC(),
}
if superseded {
at := time.Now().UTC()
item.SupersededAt = &at
}
return item
}
if state := classifyBaselineVersions(nil, jobA, snap); state.Own || state.Foreign != nil {
t.Fatalf("no versions must mean neither own nor foreign: %+v", state)
}
if state := classifyBaselineVersions([]governance.LibraryVersion{effective(jobA, snap, false)}, jobA, snap); !state.Own {
t.Fatalf("same job + same snapshot must be recognized as own: %+v", state)
}
// 快照不同就不是"我登记的那份",不能当成可收尾。
if state := classifyBaselineVersions([]governance.LibraryVersion{effective(jobA, "snap-other", false)}, jobA, snap); state.Own || state.Foreign == nil {
t.Fatalf("different snapshot of same job must not be own: %+v", state)
}
// 别人的生效版本:即使快照相同也必须拒绝。
foreign := classifyBaselineVersions([]governance.LibraryVersion{effective(jobB, snap, false)}, jobA, snap)
if foreign.Own || foreign.Foreign == nil || foreign.Foreign.SourceJobID != jobB {
t.Fatalf("foreign effective version must be reported: %+v", foreign)
}
// 已被顶替的旧版本不算生效版本。
if state := classifyBaselineVersions([]governance.LibraryVersion{effective(jobB, snap, true)}, jobA, snap); state.Own || state.Foreign != nil {
t.Fatalf("superseded version must not block a baseline: %+v", state)
}
}
// 版本已登记 + 健康:走成功边界,释放两把占用并把作业标成功。
func TestSettleBaselineWindowFinishesWhenRecorded(t *testing.T) {
store := &fakeFinishStore{}
if err := settleBaselineWindow(context.Background(), store, fakeHealth{}, "job-1", "internal", "kgseal-app",
"snap-abc", true, "succeeded", nil); err != nil {
t.Fatalf("settle: %v", err)
}
if !containsCall(store.calls, "advance_succeeded") {
t.Fatalf("job must be marked succeeded: %v", store.calls)
}
if containsCall(store.states, "needs_restore") {
t.Fatalf("success path must not hold the target in maintenance: %v", store.states)
}
if !containsCall(store.calls, "scope_release") || !containsCall(store.calls, "instance_release") {
t.Fatalf("both locks must be released: %v", store.calls)
}
}
// 版本已登记但目标服务/入口没恢复:不能报成功,必须保持维护并标 needs_restore。
func TestSettleBaselineWindowHoldsWhenHealthFails(t *testing.T) {
store := &fakeFinishStore{}
err := settleBaselineWindow(context.Background(), store, fakeHealth{err: errors.New("kgseal-ingress 未就绪")},
"job-1", "internal", "kgseal-app", "snap-abc", true, "succeeded", nil)
if err == nil {
t.Fatal("unhealthy target must not report success")
}
if containsCall(store.calls, "advance_succeeded") {
t.Fatalf("must not mark succeeded while target is unhealthy: %v", store.calls)
}
if countCall(store.states, "needs_restore") != 2 {
t.Fatalf("must hold both records with needs_restore: %v", store.states)
}
}
// 没有登记、窗口仍是 held:只能保持维护(服务确实没回来),绝不能释放占用。
func TestSettleBaselineWindowKeepsHeldWindowInMaintenance(t *testing.T) {
store := &fakeFinishStore{}
err := settleBaselineWindow(context.Background(), store, fakeHealth{}, "job-1", "internal", "kgseal-app",
"snap-abc", false, "held", errors.New("停写入者后后台仍有任务"))
if err == nil || !strings.Contains(err.Error(), "保持维护") {
t.Fatalf("err=%v", err)
}
if containsCall(store.calls, "scope_release") || containsCall(store.calls, "instance_release") {
t.Fatalf("held window must not release locks: %v", store.calls)
}
if countCall(store.states, "needs_restore") != 2 {
t.Fatalf("held window must be recorded as needs_restore: %v", store.states)
}
if !containsCall(store.calls, "advance_failed") {
t.Fatalf("job must fail: %v", store.calls)
}
}
// 没有登记、但窗口已按原状态开放:状态必须跟着变成"已释放",
// 不能留下"显示维护中、实际入口已开"。
func TestSettleBaselineWindowReleasesWhenWindowRestored(t *testing.T) {
store := &fakeFinishStore{}
err := settleBaselineWindow(context.Background(), store, fakeHealth{}, "job-1", "internal", "kgseal-app",
"snap-abc", false, "failed", errors.New("捕获目标整库失败"))
if err == nil || !strings.Contains(err.Error(), "未建立基线") {
t.Fatalf("err=%v", err)
}
if !containsCall(store.calls, "scope_release") || !containsCall(store.calls, "instance_release") {
t.Fatalf("restored window must release both locks: %v", store.calls)
}
if containsCall(store.states, "needs_restore") {
t.Fatalf("released window must not be marked as needs_restore: %v", store.states)
}
if !containsCall(store.calls, "advance_failed") {
t.Fatalf("job must fail: %v", store.calls)
}
}
// 释放失败时不能在作业上写"无需恢复":那会让界面与实际占用分叉。
func TestSettleBaselineWindowReportsReleaseFailure(t *testing.T) {
store := &fakeFinishStore{failOn: "scope_release"}
err := settleBaselineWindow(context.Background(), store, fakeHealth{}, "job-1", "internal", "kgseal-app",
"snap-abc", false, "failed", nil)
if err == nil || !strings.Contains(err.Error(), "维护占用释放失败") {
t.Fatalf("err=%v", err)
}
}
func countCall(calls []string, want string) int {
count := 0
for _, call := range calls {
if call == want {
count++
}
}
return count
}
func containsCall(calls []string, want string) bool {
for _, call := range calls {
if call == want {
return true
}
}
return false
}
+69
View File
@@ -0,0 +1,69 @@
package main
import (
"encoding/json"
"errors"
"fmt"
"strings"
"knowledge-governance/internal/appconfig"
"knowledge-governance/internal/libraryagent"
)
// 封存的身份契约:期望随**同一个 capture 请求**送进 agent,由接收方在操作互斥内、
// 任何源操作之前比对;worker 事后按回显的指纹再复核一次。
//
// 为什么不能在 worker 侧先 Health 再 Capture:那是两次拨号,中间 socket 可能被换成
// 另一个 agent,错的执行器已经把写入者停了才被拒登记——副作用已经发生。这里把期望
// 放进请求本身,换过的 agent 会先比对身份、比对失败就一次 run 都不调。
//
// 期望字段只来自治理侧配置(LIBRARY_SOURCE_*),绝不从执行器自报值生成。
// captureExpectation 把治理侧配置翻译成期望;缺身份就是缺配置,调用方必须失败。
func captureExpectation(config appconfig.Config) (libraryagent.Expectation, error) {
expectation := libraryagent.NewExpectation(config.SourceProject(), config.LibrarySourceApp, "")
if expectation.Project == "" {
return libraryagent.Expectation{}, errors.New("未配置封存来源实例身份(LIBRARY_SOURCE_PROJECT),拒绝在不核对执行器的情况下停写入者")
}
return expectation, nil
}
// captureExecutorPostcheck 是事后证据防线:结果里回显的期望指纹必须等于我们发出的,
// 自报身份也必须与期望一致。它挡不住副作用,所以不能替代请求内的核对。
func captureExecutorPostcheck(payload json.RawMessage, expectation libraryagent.Expectation) error {
if len(payload) == 0 {
return errors.New("封存结果里没有执行器身份,无法证明这次封存来自期望实例")
}
var document struct {
Identity struct {
Project string `json:"project"`
SourceApp string `json:"source_app"`
ExpectationFingerprint string `json:"expectation_fingerprint"`
} `json:"agent_identity"`
}
if err := json.Unmarshal(payload, &document); err != nil {
return errors.New("封存结果里的执行器身份无法解析")
}
identity := document.Identity
if identity.Project == "" {
return errors.New("封存结果里没有执行器身份,无法证明这次封存来自期望实例")
}
if identity.ExpectationFingerprint != expectation.Fingerprint {
return fmt.Errorf("封存结果回显的期望指纹不一致(agent=%q,期望=%q),该快照不能当作本实例的测试封存",
identity.ExpectationFingerprint, expectation.Fingerprint)
}
if identity.Project != expectation.Project {
return fmt.Errorf("封存结果来自另一个实例(agent=%q,期望=%q),该快照不能当作本实例的测试封存",
identity.Project, expectation.Project)
}
if expectation.SourceApp != "" && identity.SourceApp != expectation.SourceApp {
return fmt.Errorf("封存结果来自另一个源容器(agent=%q,期望=%q),该快照不能当作本实例的测试封存",
identity.SourceApp, expectation.SourceApp)
}
return nil
}
// expectationFingerprint 用于日志/证据行的稳定标识,不含路径与凭据。
func expectationFingerprint(expectation libraryagent.Expectation) string {
return strings.TrimSpace(expectation.Fingerprint)
}
@@ -0,0 +1,73 @@
package main
import (
"encoding/json"
"strings"
"testing"
"knowledge-governance/internal/appconfig"
"knowledge-governance/internal/libraryagent"
)
// 期望只从治理侧配置生成;缺 project 就是缺配置,必须在停写入者之前失败。
func TestCaptureExpectationComesFromGovernanceConfigAndFailsClosed(t *testing.T) {
expect, err := captureExpectation(appconfig.Config{LibrarySourceInstance: "kgmaint-weknora", LibrarySourceApp: "kgmaint-app"})
if err != nil {
t.Fatal(err)
}
if expect.Project != "kgmaint-weknora" || expect.SourceApp != "kgmaint-app" {
t.Fatalf("expectation = %+v", expect)
}
if !expect.Complete() {
t.Fatalf("expectation should be complete: %+v", expect)
}
if _, err = captureExpectation(appconfig.Config{}); err == nil ||
!strings.Contains(err.Error(), "LIBRARY_SOURCE_PROJECT") {
t.Fatalf("empty identity accepted: %v", err)
}
// 期望不取自任何执行器应答:显式给 project 就按它绑定,配置里没有别的来源。
explicit, err := captureExpectation(appconfig.Config{LibrarySourceProject: "kgmaint-weknora"})
if err != nil || explicit.Project != "kgmaint-weknora" || explicit.SourceApp != "" {
t.Fatalf("explicit = %+v err=%v", explicit, err)
}
}
// 事后证据防线:回显的期望指纹、project、source_app 任何一项不符都不能登记。
func TestCaptureExecutorPostcheckUsesTheEchoedExpectationFingerprint(t *testing.T) {
expect := libraryagent.NewExpectation("kgmaint-weknora", "kgmaint-app", "")
good := payloadOf(t, "kgmaint-weknora", "kgmaint-app", expect.Fingerprint)
if err := captureExecutorPostcheck(good, expect); err != nil {
t.Fatalf("matching identity rejected: %v", err)
}
swapped := payloadOf(t, "weknora071full", "WKNFull-app", expect.Fingerprint)
if err := captureExecutorPostcheck(swapped, expect); err == nil || !strings.Contains(err.Error(), "另一个实例") {
t.Fatalf("wrong project accepted: %v", err)
}
regenerated := payloadOf(t, "kgmaint-weknora", "kgmaint-app",
libraryagent.NewExpectation("kgmaint-weknora", "kgmaint-app", "/other/archive").Fingerprint)
if err := captureExecutorPostcheck(regenerated, expect); err == nil || !strings.Contains(err.Error(), "期望指纹不一致") {
t.Fatalf("regenerated expectation accepted: %v", err)
}
if err := captureExecutorPostcheck(json.RawMessage(`{"snapshot_id":"abc"}`), expect); err == nil ||
!strings.Contains(err.Error(), "没有执行器身份") {
t.Fatalf("missing identity accepted: %v", err)
}
if err := captureExecutorPostcheck(payloadOf(t, "kgmaint-weknora", "kgseal-app", expect.Fingerprint), expect); err == nil ||
!strings.Contains(err.Error(), "另一个源容器") {
t.Fatalf("wrong source app accepted: %v", err)
}
}
func payloadOf(t *testing.T, project, sourceApp, fingerprint string) json.RawMessage {
t.Helper()
encoded, err := json.Marshal(map[string]any{
"snapshot_id": "abc",
"agent_identity": map[string]any{
"project": project, "source_app": sourceApp, "expectation_fingerprint": fingerprint,
},
})
if err != nil {
t.Fatal(err)
}
return encoded
}
+52
View File
@@ -0,0 +1,52 @@
package main
import (
"context"
"log"
"time"
"knowledge-governance/internal/appconfig"
"knowledge-governance/internal/libraryexecutor"
postgresstore "knowledge-governance/internal/platform/postgres"
)
// 为什么由 worker 观测、页面读库:agent socket 只挂载在 worker 容器里,API 那个
// LIBRARY_AGENT_SOCKET 即使配了也 dial 不到。谁真能执行就由谁写下观测,
// 否则页面上的"可发起"只是配置字符串的回声。
const libraryExecutorProbeInterval = 30 * time.Second
// libraryExecutorProbeTimeout 是一次 health 往返的上限:它只是同 uid unix socket
// 上的一次 JSON 往返,慢到秒级就说明执行器有问题,不该拖住心跳。
const libraryExecutorProbeTimeout = 3 * time.Second
func recordLibraryExecutorStatus(ctx context.Context, store *postgresstore.Store, config appconfig.Config) {
// 归属用**封存源**实例:agent 冻结的是源库写入者,和同步目标(kgseal)不是一套。
instance := config.SourceInstance()
if instance == "" {
// 没有实例身份,观测就无处归属:写进去只会让页面拿别的实例的状态。
log.Print("library executor probe skipped: LIBRARY_SOURCE_INSTANCE is not configured")
return
}
observation := libraryexecutor.Probe(ctx, instance, config.LibraryAgentSocket, libraryExecutorProbeTimeout, nil)
if err := store.UpsertLibraryExecutorStatus(ctx, observation, config.WorkerID, time.Now().UTC()); err != nil {
log.Printf("library executor status not recorded: %v", err)
}
}
// runLibraryExecutorProbe 先观测一次再进循环:页面在 worker 刚起来时就该有真实状态。
func runLibraryExecutorProbe(ctx context.Context, store *postgresstore.Store, config appconfig.Config, interval time.Duration) {
if interval <= 0 {
interval = libraryExecutorProbeInterval
}
recordLibraryExecutorStatus(ctx, store, config)
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
recordLibraryExecutorStatus(ctx, store, config)
}
}
}
+162
View File
@@ -0,0 +1,162 @@
package main
import (
"context"
"errors"
"fmt"
"time"
"knowledge-governance/internal/governance"
postgresstore "knowledge-governance/internal/platform/postgres"
)
// libraryMaintenanceStore 是"成功收尾"用到的存储能力。抽成接口是为了能在测试里
// 让某一步失败,验证"没全部完成就不许暴露成功"。
type libraryMaintenanceStore interface {
SetInstanceMaintenanceIngress(ctx context.Context, instance, ownerJobID, state, note string, now time.Time) error
SetLibraryMaintenanceIngress(ctx context.Context, scope, ownerJobID, state, note string, now time.Time) error
UpdateInstanceMaintenanceState(ctx context.Context, instance, ownerJobID, state, note string, now time.Time) error
UpdateLibraryMaintenanceState(ctx context.Context, scope, ownerJobID, state, note string, now time.Time) error
ReleaseInstanceMaintenance(ctx context.Context, instance, ownerJobID string, now time.Time) error
ReleaseLibraryMaintenance(ctx context.Context, scope, ownerJobID string, now time.Time) error
AdvanceLibraryJob(ctx context.Context, jobID string, state governance.LibraryJobState, stage, message, recovery, snapshotID, targetSnapshotID string, now time.Time) error
}
// releaseWindowLocks 是唯一释放路径。
//
// 顺序刻意是"scope 先、instance 后":instance 行是跨 scope 的互斥来源,最后放它
// 才能保证任何新作业在全部记录落定之前都进不来。每一步的错误都必须向上暴露——
// 只丢日志会让"锁其实没放"这件事变成页面上的成功。
func releaseWindowLocks(ctx context.Context, store libraryMaintenanceStore, instance, scope, jobID, note string, now time.Time) error {
if err := store.SetLibraryMaintenanceIngress(ctx, scope, jobID, "open", note, now); !releasedOrNil(err) {
return fmt.Errorf("记录入口已开放失败(scope):%w", err)
}
if err := store.SetInstanceMaintenanceIngress(ctx, instance, jobID, "open", note, now); !releasedOrNil(err) {
return fmt.Errorf("记录入口已开放失败(instance):%w", err)
}
if err := store.UpdateLibraryMaintenanceState(ctx, scope, jobID, "verified", note, now); !releasedOrNil(err) {
return fmt.Errorf("维护状态写入失败(scope):%w", err)
}
if err := store.ReleaseLibraryMaintenance(ctx, scope, jobID, now); !releasedOrNil(err) {
return fmt.Errorf("维护占用释放失败(scope):%w", err)
}
if err := store.UpdateInstanceMaintenanceState(ctx, instance, jobID, "verified", note, now); !releasedOrNil(err) {
return fmt.Errorf("维护状态写入失败(instance):%w", err)
}
if err := store.ReleaseInstanceMaintenance(ctx, instance, jobID, now); !releasedOrNil(err) {
return fmt.Errorf("实例占用释放失败(instance):%w", err)
}
return nil
}
// healthChecker 是"目标写入者与入口是否真的恢复健康"的判定口(测试可注入失败)。
type healthChecker interface {
VerifyRunning(ctx context.Context) error
}
// successBoundary 描述一次"内容已通过验证与版本登记,但还没到可以对外报成功"的作业。
type successBoundary struct {
Ready bool
Backup string
Target string
FinishedAt time.Time
}
// finishLibrarySuccess 是唯一的成功收尾入口:只有以下全部成立才暴露 succeeded——
// 目标服务健康、入口开放、scope 与 instance 两把维护锁都释放完成、终态写入成功。
//
// 为什么不能先标成功再释放:恢复/开入口/放锁都可能失败,先写 succeeded 会让外部
// (页面、API、重启后的接管逻辑)看到一条"成功了但目标仍在维护"的作业;而
// InterruptedLibraryJobs 只接管非终态,终态作业不会再被接管,锁就永远挂着。
func finishLibrarySuccess(ctx context.Context, store libraryMaintenanceStore, health healthChecker,
instance, scope, jobID string, boundary successBoundary) error {
// 释放与终态写入必须能在 ctx 取消后继续:停机不该留下"锁放了但作业还在跑"或
// 反过来"作业成功了锁还挂着"的任一半成品。
bounded, cancel := context.WithTimeout(context.WithoutCancel(ctx), 60*time.Second)
defer cancel()
holdForRepair := func(reason string) error {
note := "内容已验证,但收尾未完成,目标保持维护:" + reason
_ = store.UpdateLibraryMaintenanceState(bounded, scope, jobID, "needs_restore", note, time.Now().UTC())
_ = store.UpdateInstanceMaintenanceState(bounded, instance, jobID, "needs_restore", note, time.Now().UTC())
_ = store.AdvanceLibraryJob(bounded, jobID, governance.LibraryJobFailed, "finalize", note,
"生产内容已写入并校验;维护保持,按预备份或人工核验后再开放", boundary.Backup, boundary.Target, time.Now().UTC())
return errors.New(note)
}
if err := health.VerifyRunning(bounded); err != nil {
return holdForRepair("目标服务或入口未恢复健康:" + err.Error())
}
if err := releaseWindowLocks(bounded, store, instance, scope, jobID,
"写入与校验完成,目标已按原运行状态恢复", time.Now().UTC()); err != nil {
return holdForRepair(err.Error())
}
finishedAt := time.Now().UTC()
if err := store.AdvanceLibraryJob(bounded, jobID, governance.LibraryJobSucceeded, "finalize", "", "",
boundary.Backup, boundary.Target, finishedAt); err != nil {
// 锁已释放但终态没写下去:作业保持非终态,重启后会被接管(接管走
// finalize-only,只复核与登记,不重写生产)。
return fmt.Errorf("维护已释放但作业终态写入失败(等待接管):%w", err)
}
return nil
}
// takeoverKind 是接管时"这个作业走到哪一步"的分类。
//
// 四类必须分开:把"版本查不到"当成"没有收尾证据",就会把只差收尾的作业重新写一遍
// 目标;把"曾被顶替"当成正常重放,就会用旧快照覆盖更新的生产版本。
type takeoverKind string
const (
// takeoverFresh:没有任何收尾证据,按正常流程(有预备份则复用,写前仍备份)。
takeoverFresh takeoverKind = "fresh"
// takeoverPendingFinal:内容与版本已落定,只差收尾(只复核与登记,不重写)。
takeoverPendingFinal takeoverKind = "pending_final"
// takeoverSuperseded:本快照曾经生效、后来被别的快照顶替,不得自动覆盖。
takeoverSuperseded takeoverKind = "superseded"
// takeoverUnknown:版本状态读不到且没有阶段证据,属"未知",不得写入。
takeoverUnknown takeoverKind = "unknown"
)
// classifyTakeover 只按证据分类:
// - finalize 阶段成功是独立证据(来自作业自己的阶段行),不因版本查询失败而丢弃;
// - 本快照存在 released/被顶替的版本 ⇒ superseded;
// - 本快照存在未被顶掉的生效版本 ⇒ pending_final;
// - 查询失败且无阶段证据 ⇒ unknown(可重试,不当成没有证据)。
func classifyTakeover(job governance.LibraryJob, versions []governance.LibraryVersion, versionsErr error) takeoverKind {
finalized := false
for _, stage := range job.Stages {
if stage.Stage == "finalize" && stage.State == "succeeded" {
finalized = true
}
}
if versionsErr != nil && !finalized {
return takeoverUnknown
}
superseded, effective := false, false
for _, version := range versions {
if version.Kind != governance.LibraryVersionProductionEffective || version.SnapshotID != job.TargetSnapshotID {
continue
}
if version.SupersededAt != nil {
superseded = true
} else {
effective = true
}
}
if superseded && !effective {
return takeoverSuperseded
}
if effective || finalized {
return takeoverPendingFinal
}
return takeoverFresh
}
var _ libraryMaintenanceStore = (*postgresstore.Store)(nil)
// releasedOrNil:这一行已经由本作业释放过,说明该步在上一次尝试里已经完成。
// 部分成功后重试幂等靠它——否则"scope 放了、instance 没放"的作业永远重试不下去。
func releasedOrNil(err error) bool {
return err == nil || errors.Is(err, postgresstore.ErrMaintenanceReleased)
}
+213
View File
@@ -0,0 +1,213 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"time"
"knowledge-governance/internal/governance"
postgresstore "knowledge-governance/internal/platform/postgres"
)
type fakeFinishStore struct {
calls []string
failOn string
alreadyReleased bool
advanced []governance.LibraryJobState
// states 记录所有维护状态写入(测试要区分 verified 与 needs_restore)。
states []string
}
// 模拟"这一行已由同一个作业释放过"(真实 Store 返回 ErrMaintenanceReleased)。
func (f *fakeFinishStore) releasedSentinel() error {
if f.alreadyReleased {
return postgresstore.ErrMaintenanceReleased
}
return nil
}
func (f *fakeFinishStore) record(name string) error {
f.calls = append(f.calls, name)
if f.failOn == name {
return errors.New("injected " + name + " failure")
}
return nil
}
func (f *fakeFinishStore) SetInstanceMaintenanceIngress(context.Context, string, string, string, string, time.Time) error {
if err := f.releasedSentinel(); err != nil {
return err
}
return f.record("instance_ingress")
}
func (f *fakeFinishStore) SetLibraryMaintenanceIngress(context.Context, string, string, string, string, time.Time) error {
if err := f.releasedSentinel(); err != nil {
return err
}
return f.record("scope_ingress")
}
func (f *fakeFinishStore) UpdateInstanceMaintenanceState(_ context.Context, _, _, state, _ string, _ time.Time) error {
if err := f.releasedSentinel(); err != nil {
return err
}
f.states = append(f.states, state)
return f.record("instance_state")
}
func (f *fakeFinishStore) UpdateLibraryMaintenanceState(_ context.Context, _, _, state, _ string, _ time.Time) error {
if err := f.releasedSentinel(); err != nil {
return err
}
f.states = append(f.states, state)
return f.record("scope_state")
}
func (f *fakeFinishStore) ReleaseInstanceMaintenance(context.Context, string, string, time.Time) error {
if err := f.releasedSentinel(); err != nil {
return err
}
return f.record("instance_release")
}
func (f *fakeFinishStore) ReleaseLibraryMaintenance(context.Context, string, string, time.Time) error {
if err := f.releasedSentinel(); err != nil {
return err
}
return f.record("scope_release")
}
func (f *fakeFinishStore) AdvanceLibraryJob(_ context.Context, _ string, state governance.LibraryJobState,
_, _, _, _, _ string, _ time.Time) error {
f.advanced = append(f.advanced, state)
return f.record("advance_" + string(state))
}
type fakeHealth struct{ err error }
func (h fakeHealth) VerifyRunning(context.Context) error { return h.err }
func boundary() successBoundary {
return successBoundary{Ready: true, Backup: "backup-1", Target: "snap-1", FinishedAt: time.Now().UTC()}
}
// 首次观察到成功时,两把维护锁必须已经释放,且终态写入排在最后。
func TestFinishLibrarySuccessReleasesEveryLockBeforeSucceeded(t *testing.T) {
store := &fakeFinishStore{}
if err := finishLibrarySuccess(context.Background(), store, fakeHealth{}, "kgseal-weknora", "internal", "job-1", boundary()); err != nil {
t.Fatal(err)
}
want := []string{"scope_ingress", "instance_ingress", "scope_state", "scope_release",
"instance_state", "instance_release", "advance_succeeded"}
if strings.Join(store.calls, ",") != strings.Join(want, ",") {
t.Fatalf("call order = %v", store.calls)
}
if len(store.advanced) != 1 || store.advanced[0] != governance.LibraryJobSucceeded {
t.Fatalf("advanced = %v", store.advanced)
}
}
// 目标服务/入口没恢复健康:不许放锁,不许报成功,作业落成失败并保持维护。
func TestFinishLibrarySuccessKeepsMaintenanceWhenTargetIsNotHealthy(t *testing.T) {
store := &fakeFinishStore{}
err := finishLibrarySuccess(context.Background(), store, fakeHealth{err: errors.New("app not healthy")},
"kgseal-weknora", "internal", "job-2", boundary())
if err == nil || !strings.Contains(err.Error(), "未恢复健康") {
t.Fatalf("err = %v", err)
}
for _, call := range store.calls {
if strings.HasSuffix(call, "_release") || call == "advance_succeeded" {
t.Fatalf("released or reported success despite unhealthy target: %v", store.calls)
}
}
if len(store.advanced) != 1 || store.advanced[0] != governance.LibraryJobFailed {
t.Fatalf("advanced = %v", store.advanced)
}
}
// 任一步释放失败:不报成功,作业失败,维护状态写 needs_restore(等人工/接管)。
func TestFinishLibrarySuccessStopsOnReleaseFailure(t *testing.T) {
for _, step := range []string{"scope_release", "instance_state", "instance_release"} {
t.Run(step, func(t *testing.T) {
store := &fakeFinishStore{failOn: step}
err := finishLibrarySuccess(context.Background(), store, fakeHealth{}, "kgseal-weknora", "internal", "job-3", boundary())
if err == nil {
t.Fatalf("%s failure was swallowed", step)
}
for _, call := range store.calls {
if call == "advance_succeeded" {
t.Fatalf("reported success after %s failed: %v", step, store.calls)
}
}
if len(store.advanced) != 1 || store.advanced[0] != governance.LibraryJobFailed {
t.Fatalf("advanced = %v", store.advanced)
}
})
}
}
// 终态写入失败:锁已释放,但不能声称成功;作业保持非终态等接管。
func TestFinishLibrarySuccessReportsTerminalWriteFailure(t *testing.T) {
store := &fakeFinishStore{failOn: "advance_succeeded"}
err := finishLibrarySuccess(context.Background(), store, fakeHealth{}, "kgseal-weknora", "internal", "job-4", boundary())
if err == nil || !strings.Contains(err.Error(), "等待接管") {
t.Fatalf("err = %v", err)
}
}
// 接管分类:阶段证据独立成立;查询失败=未知(不得写入);被顶替=不得覆盖。
func TestClassifyTakeoverSeparatesUnknownPendingSupersededAndFresh(t *testing.T) {
job := governance.LibraryJob{Scope: "internal", Kind: governance.LibraryJobSync, TargetSnapshotID: "snap-1"}
finalized := job
finalized.Stages = []governance.LibraryStage{{Stage: "finalize", State: "succeeded"}}
effective := []governance.LibraryVersion{{Kind: governance.LibraryVersionProductionEffective, SnapshotID: "snap-1"}}
supersededVersion := []governance.LibraryVersion{{Kind: governance.LibraryVersionProductionEffective,
SnapshotID: "snap-1", SupersededAt: ptrTime(time.Now())}}
unrelated := []governance.LibraryVersion{{Kind: governance.LibraryVersionProductionEffective, SnapshotID: "snap-9"}}
queryErr := errors.New("version storage temporarily unavailable")
cases := []struct {
name string
job governance.LibraryJob
versions []governance.LibraryVersion
err error
want takeoverKind
}{
// 关键回归:查询失败 + 没有阶段证据 ⇒ 未知,不能当成"没有收尾证据"继续写。
{"query fails without stage evidence", job, nil, queryErr, takeoverUnknown},
// 阶段证据先独立成立:查询失败不影响"只差收尾"的判定。
{"query fails but finalize stage succeeded", finalized, nil, queryErr, takeoverPendingFinal},
{"effective version for this snapshot", job, effective, nil, takeoverPendingFinal},
{"finalize stage succeeded", finalized, nil, nil, takeoverPendingFinal},
{"this snapshot was superseded", job, supersededVersion, nil, takeoverSuperseded},
{"no evidence at all", job, nil, nil, takeoverFresh},
{"another snapshot effective but this one has no row", job, unrelated, nil, takeoverFresh},
{"empty target snapshot", governance.LibraryJob{Scope: "internal"}, nil, nil, takeoverFresh},
}
for _, test := range cases {
t.Run(test.name, func(t *testing.T) {
if got := classifyTakeover(test.job, test.versions, test.err); got != test.want {
t.Fatalf("classify = %q, want %q", got, test.want)
}
})
}
}
// 部分释放后重试必须幂等:已由本作业释放的行不能再算失败。
func TestReleaseWindowLocksToleratesAlreadyReleasedRows(t *testing.T) {
store := &fakeFinishStore{}
if err := releaseWindowLocks(context.Background(), store, "kgseal-weknora", "internal", "job-1",
"verified", time.Now().UTC()); err != nil {
t.Fatal(err)
}
store = &fakeFinishStore{alreadyReleased: true}
if err := releaseWindowLocks(context.Background(), store, "kgseal-weknora", "internal", "job-1",
"verified", time.Now().UTC()); err != nil {
t.Fatalf("retry after a partial release must be idempotent: %v", err)
}
// 真正"不是本作业持有"的错误仍然要暴露。
store = &fakeFinishStore{failOn: "instance_release"}
if err := releaseWindowLocks(context.Background(), store, "kgseal-weknora", "internal", "job-1",
"verified", time.Now().UTC()); err == nil {
t.Fatal("a foreign owner error was swallowed")
}
}
func ptrTime(value time.Time) *time.Time { return &value }
+45 -53
View File
@@ -16,6 +16,7 @@ import (
"knowledge-governance/internal/librarymaintenance" "knowledge-governance/internal/librarymaintenance"
"knowledge-governance/internal/libraryrelease" "knowledge-governance/internal/libraryrelease"
"knowledge-governance/internal/librarysnapshot" "knowledge-governance/internal/librarysnapshot"
"knowledge-governance/internal/pgdsn"
postgresstore "knowledge-governance/internal/platform/postgres" postgresstore "knowledge-governance/internal/platform/postgres"
) )
@@ -65,32 +66,8 @@ type libraryReleasePayload struct {
FinalizeOnly bool `json:"finalize_only,omitempty"` FinalizeOnly bool `json:"finalize_only,omitempty"`
} }
// openLibraryMaintenance 按部署注入的配置装配实例级维护目标。 // openLibraryMaintenance 定义在 library_wiring.go:维护目标的装配与目标整库栈共用
// 未接线时返回明确原因:只锁治理库 scope 不算维护,宁可拒绝写也不冒充。 // 同一个包(internal/librarystack),不在这里再拼一份。
func openLibraryMaintenance(config appconfig.Config) (*librarymaintenance.Manager, error) {
database, err := librarymaintenance.DatabaseIdentity(config.LibraryProductionDatabaseURL)
if err != nil {
return nil, err
}
return librarymaintenance.New(librarymaintenance.Config{
Instance: config.LibraryTargetInstance,
Project: config.LibraryTargetProject,
Writers: config.LibraryTargetWriters,
NonWriters: config.LibraryTargetNonWriters,
DockerSocket: config.LibraryTargetDockerSocket,
IngressURL: config.LibraryTargetIngressURL,
IngressService: config.LibraryTargetIngressService,
IngressDrainFile: config.LibraryTargetIngressDrainFile,
Database: database,
Index: config.LibraryProductionIndex,
Bucket: config.LibraryProductionS3Bucket,
// 后台静默检查的运行时依赖:HTTP 排空不等于后台任务排空。
DatabaseURL: config.LibraryProductionDatabaseURL,
RedisAddr: config.LibraryTargetRedisAddr,
RedisPassword: config.LibraryTargetRedisPassword,
RedisDB: config.LibraryTargetRedisDB,
})
}
// runLibraryRelease 执行"闸口 → 授权 → 关闭目标入口并停止写入者 → 预备份 → // runLibraryRelease 执行"闸口 → 授权 → 关闭目标入口并停止写入者 → 预备份 →
// 四类写入 → 校验 → 登记权威版本 → 恢复原运行状态和入口"的完整链路。 // 四类写入 → 校验 → 登记权威版本 → 恢复原运行状态和入口"的完整链路。
@@ -116,6 +93,21 @@ func runLibraryRelease(ctx context.Context, store *postgresstore.Store, config a
// 绝不重新备份——重新备份会把"半写状态"当成新基线,覆盖真正可回滚的那份。 // 绝不重新备份——重新备份会把"半写状态"当成新基线,覆盖真正可回滚的那份。
previous, err := store.LibraryJob(ctx, payload.LibraryJobID) previous, err := store.LibraryJob(ctx, payload.LibraryJobID)
resumed := err == nil && previous.SnapshotID != "" resumed := err == nil && previous.SnapshotID != ""
if resumed {
versions, versionErr := store.LibraryVersions(ctx, payload.Scope, 50)
switch classifyTakeover(previous, versions, versionErr) {
case takeoverUnknown:
// 版本状态读不到、又没有阶段证据:不能当成"没有收尾证据"继续写目标。
return failLibraryJob(ctx, store, recorder, payload.LibraryJobID,
"接管时读不到该目标的版本状态,无法判断是只差收尾还是要重做;未对目标执行任何写入,稍后重试", "prebackup")
case takeoverSuperseded:
return failLibraryJob(ctx, store, recorder, payload.LibraryJobID,
"本快照的生效版本已被别的快照顶替,拒绝用旧快照覆盖当前生产版本", "prebackup")
case takeoverPendingFinal:
// 上次可能死在"锁已放、终态没写"之间:只复核与登记,不重写生产。
payload.FinalizeOnly = true
}
}
if resumed { if resumed {
recorder.Stage(ctx, "prebackup", "skipped", "接管失败前已创建的预备份 "+previous.SnapshotID) recorder.Stage(ctx, "prebackup", "skipped", "接管失败前已创建的预备份 "+previous.SnapshotID)
} }
@@ -194,7 +186,7 @@ func runLibraryRelease(ctx context.Context, store *postgresstore.Store, config a
"目标拓扑不满足唯一入口约束,拒绝开维护窗口:"+topoErr.Error(), "prebackup") "目标拓扑不满足唯一入口约束,拒绝开维护窗口:"+topoErr.Error(), "prebackup")
} }
targetIdentity := fmt.Sprintf("scope=%s db=%s index=%s bucket=%s", payload.Scope, targetIdentity := fmt.Sprintf("scope=%s db=%s index=%s bucket=%s", payload.Scope,
redactDSN(config.LibraryProductionDatabaseURL), config.LibraryProductionIndex, config.LibraryProductionS3Bucket) pgdsn.Endpoint(config.LibraryProductionDatabaseURL), config.LibraryProductionIndex, config.LibraryProductionS3Bucket)
originalState, stateErr := manager.OriginalState(ctx) originalState, stateErr := manager.OriginalState(ctx)
if stateErr != nil { if stateErr != nil {
return failLibraryJob(ctx, store, recorder, payload.LibraryJobID, return failLibraryJob(ctx, store, recorder, payload.LibraryJobID,
@@ -256,6 +248,7 @@ func runLibraryRelease(ctx context.Context, store *postgresstore.Store, config a
if resumed { if resumed {
contentState = contentHalfWritten contentState = contentHalfWritten
} }
pending := successBoundary{}
tracked := &contentTrackedTarget{inner: target, onApply: func() { contentState = contentHalfWritten }} tracked := &contentTrackedTarget{inner: target, onApply: func() { contentState = contentHalfWritten }}
allowRestore := func(error) bool { allowRestore := func(error) bool {
return contentState == contentUntouched || contentState == contentSucceeded || contentState == contentRolledBack return contentState == contentUntouched || contentState == contentSucceeded || contentState == contentRolledBack
@@ -290,8 +283,9 @@ func runLibraryRelease(ctx context.Context, store *postgresstore.Store, config a
return errors.New(message) return errors.New(message)
} }
recorder.Stage(ctx, "finalize", "succeeded", "已登记生效版本(仅收尾,未重写生产)") recorder.Stage(ctx, "finalize", "succeeded", "已登记生效版本(仅收尾,未重写生产)")
_ = store.AdvanceLibraryJob(ctx, payload.LibraryJobID, governance.LibraryJobSucceeded, "finalize", "", "", // 成功要等维护锁释放完才暴露:这里只记"待收尾"。
previous.SnapshotID, payload.TargetSnapshotID, time.Now().UTC()) pending = successBoundary{Ready: true, Backup: previous.SnapshotID,
Target: payload.TargetSnapshotID, FinishedAt: time.Now().UTC()}
return nil return nil
} }
@@ -340,13 +334,10 @@ func runLibraryRelease(ctx context.Context, store *postgresstore.Store, config a
return errors.New(message) return errors.New(message)
} }
recorder.Stage(ctx, "finalize", "succeeded", "已登记生效版本") recorder.Stage(ctx, "finalize", "succeeded", "已登记生效版本")
if advanceErr := store.AdvanceLibraryJob(ctx, payload.LibraryJobID, governance.LibraryJobSucceeded, "finalize", "", "", // 内容与版本都落定了,但还不能报成功:服务健康、入口开放、两把维护锁
outcome.Backup.SnapshotID, payload.TargetSnapshotID, finishedAt); advanceErr != nil { // 释放完成才允许暴露 succeeded(见 finishLibrarySuccess)。
// 状态落不下去时不要把队列作业报成功:让上层可见并等待接管。 pending = successBoundary{Ready: true, Backup: outcome.Backup.SnapshotID,
return fmt.Errorf("作业状态写入失败: %w", advanceErr) Target: payload.TargetSnapshotID, FinishedAt: finishedAt}
}
log.Printf("library %s succeeded job=%s scope=%s snapshot=%s backup=%s", kind,
payload.LibraryJobID, payload.Scope, payload.TargetSnapshotID, outcome.Backup.SnapshotID)
return nil return nil
case "rolled_back": case "rolled_back":
contentState = contentRolledBack contentState = contentRolledBack
@@ -407,10 +398,19 @@ func runLibraryRelease(ctx context.Context, store *postgresstore.Store, config a
} }
return windowErr return windowErr
case contentState == contentSucceeded: case contentState == contentSucceeded:
if err := releaseWindowLocks(ctx, store, identity.Instance, payload.Scope, payload.LibraryJobID, if !pending.Ready {
"写入与校验完成,目标已按原运行状态恢复", now2); err != nil { // 内容状态说成功但没有收尾凭据(不可能两者都缺):按未收尾处理,保持维护。
return fmt.Errorf("目标已恢复但维护占用释放失败: %w", err) note := "内容状态为已验证,但缺少收尾凭据,目标保持维护"
_ = store.UpdateLibraryMaintenanceState(ctx, payload.Scope, payload.LibraryJobID, "needs_restore", note, now2)
_ = store.UpdateInstanceMaintenanceState(ctx, identity.Instance, payload.LibraryJobID, "needs_restore", note, now2)
return errors.New(note)
} }
if finishErr := finishLibrarySuccess(ctx, store, manager, identity.Instance, payload.Scope,
payload.LibraryJobID, pending); finishErr != nil {
return finishErr
}
log.Printf("library %s succeeded job=%s scope=%s snapshot=%s backup=%s", kind,
payload.LibraryJobID, payload.Scope, payload.TargetSnapshotID, pending.Backup)
return windowErr return windowErr
case contentState == contentRolledBack: case contentState == contentRolledBack:
if err := releaseWindowLocks(ctx, store, identity.Instance, payload.Scope, payload.LibraryJobID, if err := releaseWindowLocks(ctx, store, identity.Instance, payload.Scope, payload.LibraryJobID,
@@ -428,19 +428,6 @@ func runLibraryRelease(ctx context.Context, store *postgresstore.Store, config a
} }
} }
// releaseWindowLocks 是唯一释放路径:先记录入口已恢复,再放 scope 行与实例锁。
// 顺序反过来会出现"实例锁放了、scope 还挂着"的中间态,页面就会说正在维护。
func releaseWindowLocks(ctx context.Context, store *postgresstore.Store, instance, scope, jobID, note string, now time.Time) error {
_ = store.SetInstanceMaintenanceIngress(ctx, instance, jobID, "open", note, now)
_ = store.SetLibraryMaintenanceIngress(ctx, scope, jobID, "open", note, now)
_ = store.UpdateInstanceMaintenanceState(ctx, instance, jobID, "verified", note, now)
_ = store.UpdateLibraryMaintenanceState(ctx, scope, jobID, "verified", note, now)
if err := store.ReleaseInstanceMaintenance(ctx, instance, jobID, now); err != nil {
return err
}
return store.ReleaseLibraryMaintenance(ctx, scope, jobID, now)
}
// resumeInterruptedLibraryJobs 在 worker 启动时接管中断的写作业: // resumeInterruptedLibraryJobs 在 worker 启动时接管中断的写作业:
// 已经持久化预备份的重新排队(同一个业务作业 id,阶段与备份都接着用), // 已经持久化预备份的重新排队(同一个业务作业 id,阶段与备份都接着用),
// 连预备份都没完成的才判失败——那种情况没有可回滚的基线,不能盲写。 // 连预备份都没完成的才判失败——那种情况没有可回滚的基线,不能盲写。
@@ -469,8 +456,13 @@ func resumeInterruptedLibraryJobs(ctx context.Context, store *postgresstore.Stor
continue continue
} }
queueType := librarySyncJobType queueType := librarySyncJobType
if job.Kind == governance.LibraryJobRestore { switch job.Kind {
case governance.LibraryJobRestore:
queueType = libraryRestoreJobType queueType = libraryRestoreJobType
case governance.LibraryJobBaseline:
// 基线作业被杀时窗口可能还停着目标:用同一个作业 id 接管那个窗口,
// 继续捕获/校验,而不是留下一个 held 目标等人工处理。
queueType = libraryBaselineJobType
} }
payload, _ := json.Marshal(map[string]string{"library_job_id": job.ID, "scope": job.Scope, payload, _ := json.Marshal(map[string]string{"library_job_id": job.ID, "scope": job.Scope,
"target_snapshot_id": job.TargetSnapshotID}) "target_snapshot_id": job.TargetSnapshotID})
+19 -106
View File
@@ -2,117 +2,30 @@ package main
import ( import (
"context" "context"
"errors"
"fmt"
"strings"
"knowledge-governance/internal/appconfig" "knowledge-governance/internal/appconfig"
"knowledge-governance/internal/librarysnapshot" "knowledge-governance/internal/librarymaintenance"
"knowledge-governance/internal/librarystack"
v071 "knowledge-governance/internal/releaseadapter/v071" v071 "knowledge-governance/internal/releaseadapter/v071"
) )
// buildLibraryProductionStack 用部署注入的配置装配生产整库目标与独立预备份存储。 // 目标整库读写栈的装配只有一份实现(internal/librarystack):worker 的同步/恢复/
// 任何一项缺失都返回明确原因,调用方据此把作业失败并把缺项写进阶段, // 基线与主机侧恢复演练共用它,避免"目标是谁、用哪份契约"随调用路径漂移。
// 不用"未配置"的占位对象假装能跑。
// openLibraryMaintenance 按部署注入的配置装配实例级维护目标。
// 未接线时返回明确原因:只锁治理库 scope 不算维护,宁可拒绝写也不冒充。
func openLibraryMaintenance(config appconfig.Config) (*librarymaintenance.Manager, error) {
return librarystack.OpenMaintenance(config)
}
// buildLibraryProductionStack 用"要同步的这份快照"做契约装配目标与预备份存储。
func buildLibraryProductionStack(ctx context.Context, config appconfig.Config, scope, snapshotID, knownMapping string) ( func buildLibraryProductionStack(ctx context.Context, config appconfig.Config, scope, snapshotID, knownMapping string) (
*v071.ProductionLibraryTarget, *v071.ProductionBackupStore, error) { *v071.ProductionLibraryTarget, *v071.ProductionBackupStore, error) {
missing := make([]string, 0, 4) return librarystack.OpenProductionStack(ctx, config, scope, snapshotID, knownMapping)
if config.LibraryProductionDatabaseURL == "" { }
missing = append(missing, "LIBRARY_PRODUCTION_DATABASE_URL")
} // buildLibraryBaselineStack 装配"给目标自己建基线"所需的目标与捕获器(契约取目标当前映射)。
if config.LibraryProductionElasticURL == "" { func buildLibraryBaselineStack(ctx context.Context, config appconfig.Config, scope string) (
missing = append(missing, "LIBRARY_PRODUCTION_ELASTIC_URL") *v071.ProductionLibraryTarget, *v071.ProductionBackupStore, error) {
} return librarystack.OpenBaselineStack(ctx, config, scope)
if config.LibraryProductionTenantID == 0 {
missing = append(missing, "LIBRARY_PRODUCTION_TENANT_ID")
}
kb := config.LibraryProductionInternalKB
if scope == "external" {
kb = config.LibraryProductionExternalKB
}
if kb == "" {
missing = append(missing, "LIBRARY_PRODUCTION_KB_"+strings.ToUpper(scope))
}
if config.LibraryProductionS3Endpoint == "" || config.LibraryProductionS3Bucket == "" ||
config.LibraryProductionS3AccessKey == "" || config.LibraryProductionS3SecretKey == "" {
missing = append(missing, "LIBRARY_PRODUCTION_S3_ENDPOINT/BUCKET/ACCESS_KEY/SECRET_KEY")
}
if len(missing) > 0 {
return nil, nil, fmt.Errorf("生产整库目标未配置:缺少 %s", strings.Join(missing, "、"))
}
secure := strings.HasPrefix(config.LibraryProductionS3Endpoint, "https://")
objects, err := v071.NewS3ObjectStore(v071.S3ObjectStoreConfig{
Provider: config.LibraryProductionS3Provider, Endpoint: config.LibraryProductionS3Endpoint,
Region: config.LibraryProductionS3Region, Bucket: config.LibraryProductionS3Bucket,
AccessKeyID: config.LibraryProductionS3AccessKey, SecretAccessKey: config.LibraryProductionS3SecretKey,
Secure: secure, ForcePathStyle: true, MaxObjectBytes: 256 * 1024 * 1024,
})
if err != nil {
return nil, nil, errors.New("生产对象存储配置无效")
}
identity := v071.ProductionScope{TenantID: config.LibraryProductionTenantID, KnowledgeBaseID: kb}
target, err := v071.NewProductionLibraryTarget(ctx, v071.ProductionLibraryConfig{
Postgres: config.LibraryProductionDatabaseURL, SearchURL: config.LibraryProductionElasticURL,
SearchIndex: config.LibraryProductionIndex, Objects: objects, ObjectsReader: objects,
ArchiveRoot: config.LibraryArchiveRoot,
ObjectsBucket: config.LibraryProductionS3Bucket,
MaxDocuments: 200000,
}, identity)
if err != nil {
return nil, nil, err
}
// 契约以"要同步的这份快照"为准:生产必须向它看齐,而不是反过来。
// 生产为空时索引可能根本不存在,这时不能拿生产映射当契约(会永远启动不了)。
archive, err := librarysnapshot.Open(config.LibraryArchiveRoot, 256*1024*1024)
if err != nil {
target.Close()
return nil, nil, fmt.Errorf("归档不可用,无法读取快照契约: %w", err)
}
manifest, err := archive.Load(snapshotID)
archive.Close()
if err != nil {
target.Close()
return nil, nil, fmt.Errorf("读取目标快照失败: %w", err)
}
contract := manifest.Compatibility
if contract.EmbeddingDimension == 0 {
contract.EmbeddingModel = config.LibraryEmbeddingModel
contract.EmbeddingDimension = config.LibraryEmbeddingDimension
}
// 生产索引存在时必须与快照契约一致;不存在或为空基线时跳过比对
// (空索引的映射没有可比性,非空之后仍然严格比对)。
if mapping, mapErr := target.Search().MappingSHA256(ctx); mapErr == nil {
// 映射基线:优先用"上一次生产生效版本登记的实际映射",
// 它才是这份生产索引的既有事实;快照契约只管"来自哪份数据"。
baseline := knownMapping
if baseline == "" {
// 没有历史基线时(首次接管这套生产索引),把当前实际映射当作基线并随
// 预备份登记;之后任何改动都会与登记值不符而被拒绝。
baseline = mapping
}
if baseline != "" && mapping != baseline {
current, currentErr := target.Current(ctx)
emptyBaseline := currentErr == nil && current.Counts["index"] == 0 && current.Counts["chunks"] == 0
if !emptyBaseline {
target.Close()
return nil, nil, errors.New("生产索引映射与上次生效版本不一致(索引被改动过),先对齐索引映射")
}
}
}
// 预备份描述的是"生产现在的索引",不是"快照的来源索引":
// 生产映射基线优先用上次生效版本登记的实际值。
backupContract := contract
if knownMapping != "" {
backupContract.IndexMappingSHA256 = knownMapping
}
backups, err := v071.NewProductionBackupStore(v071.ProductionBackupConfig{
Pool: target.Pool(), Search: target.Search(), Objects: target.Objects(),
ArchiveRoot: config.LibraryArchiveRoot, Scope: scope, Identity: target.Identity(),
Compatibility: backupContract, MaxDocuments: 200000, MaxObjectByte: 256 * 1024 * 1024,
})
if err != nil {
target.Close()
return nil, nil, err
}
return target, backups, nil
} }
+29 -18
View File
@@ -8,7 +8,6 @@ import (
"os" "os"
"os/signal" "os/signal"
"path/filepath" "path/filepath"
"strings"
"syscall" "syscall"
"time" "time"
@@ -58,21 +57,6 @@ func (r libraryStageRecorder) Stage(ctx context.Context, stage, state, detail st
func int64Ptr(value int64) *int64 { return &value } func int64Ptr(value int64) *int64 { return &value }
// redactDSN 只保留库名,避免把连接串写进维护记录。
func redactDSN(dsn string) string {
if dsn == "" {
return ""
}
if index := strings.LastIndex(dsn, "/"); index >= 0 && index+1 < len(dsn) {
name := dsn[index+1:]
if cut := strings.Index(name, "?"); cut >= 0 {
name = name[:cut]
}
return name
}
return "configured"
}
// stageSize 是某一类数据在快照里的真实总量(行数/字节)。 // stageSize 是某一类数据在快照里的真实总量(行数/字节)。
type stageSize struct { type stageSize struct {
count int64 count int64
@@ -109,6 +93,9 @@ const libraryRecoverJobType = "library.recover"
// libraryRestoreJobType 是按指定整库版本恢复的队列类型;与同步分开、授权也分开。 // libraryRestoreJobType 是按指定整库版本恢复的队列类型;与同步分开、授权也分开。
const libraryRestoreJobType = "library.restore" const libraryRestoreJobType = "library.restore"
// libraryBaselineJobType 是"建立目标权威基线"的队列类型(与 API 侧常量同值)。
const libraryBaselineJobType = "library.baseline"
func main() { func main() {
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop() defer stop()
@@ -205,6 +192,8 @@ func run(ctx context.Context, getenv func(string) string) error {
go runRecordsSyncScheduler(ctx, mirrorStore, func(error) { log.Print("QA local mirror scheduler unavailable; last committed data retained") }) go runRecordsSyncScheduler(ctx, mirrorStore, func(error) { log.Print("QA local mirror scheduler unavailable; last committed data retained") })
} }
go weeklyMailLoop(ctx, pool, getenv) go weeklyMailLoop(ctx, pool, getenv)
// 封存执行器状态:worker 是唯一能连 agent socket 的一方,它写观测、页面读观测。
go runLibraryExecutorProbe(ctx, store, config, libraryExecutorProbeInterval)
go runSourceRefreshScheduler(ctx, store, config.SourceRefreshSchedulerInterval, func(err error) { go runSourceRefreshScheduler(ctx, store, config.SourceRefreshSchedulerInterval, func(err error) {
log.Printf("worker %s source refresh scheduler error: %v", config.WorkerID, err) log.Printf("worker %s source refresh scheduler error: %v", config.WorkerID, err)
}) })
@@ -284,7 +273,16 @@ func workerHandlers(store *postgresstore.Store, config appconfig.Config) (map[st
_ = store.AdvanceLibraryJob(ctx, libraryJobID, governance.LibraryJobFailed, "export", "dead letter release storage unavailable", "", "", "", time.Now().UTC()) _ = store.AdvanceLibraryJob(ctx, libraryJobID, governance.LibraryJobFailed, "export", "dead letter release storage unavailable", "", "", "", time.Now().UTC())
return fmt.Errorf("load dead letter releases: %w", releaseErr) return fmt.Errorf("load dead letter releases: %w", releaseErr)
} }
result, err := libraryagent.CaptureReleased(ctx, agentConfig, payload.Scope, payload.CaptureJob, released) // 期望执行器身份随请求一起送进 agent,由接收方在操作互斥内、任何源操作之前
// 比对:这样即使 socket 在这之前被换掉,错的执行器也一次 run 都不会跑。
expectation, expectErr := captureExpectation(config)
if expectErr != nil {
_ = store.UpdateLibraryStage(ctx, libraryJobID, "export", "failed", nil, nil, nil, nil, expectErr.Error(), time.Now().UTC())
_ = store.AdvanceLibraryJob(ctx, libraryJobID, governance.LibraryJobFailed, "export", expectErr.Error(),
"未停任何写入者;补齐来源身份配置后重试", "", "", time.Now().UTC())
return expectErr
}
result, err := libraryagent.CaptureReleased(ctx, agentConfig, payload.Scope, payload.CaptureJob, released, expectation)
finishedAt := time.Now().UTC() finishedAt := time.Now().UTC()
if err != nil { if err != nil {
// 导出失败时阶段与作业都要留痕;恢复责任在 agent,不会因为作业失败而假装来源已恢复。 // 导出失败时阶段与作业都要留痕;恢复责任在 agent,不会因为作业失败而假装来源已恢复。
@@ -294,13 +292,22 @@ func workerHandlers(store *postgresstore.Store, config appconfig.Config) (map[st
log.Printf("library capture failed scope=%s capture_job=%s: %v", payload.Scope, payload.CaptureJob, err) log.Printf("library capture failed scope=%s capture_job=%s: %v", payload.Scope, payload.CaptureJob, err)
return err return err
} }
// 执行后复核:结果里 agent 自报的身份必须还是期望实例,否则这次封存
// 可能来自别的栈,绝不能登记成"本实例的测试封存"。
if identityErr := captureExecutorPostcheck(result.Payload, expectation); identityErr != nil {
_ = store.UpdateLibraryStage(ctx, libraryJobID, "export", "failed", nil, nil, nil, nil, identityErr.Error(), finishedAt)
_ = store.AdvanceLibraryJob(ctx, libraryJobID, governance.LibraryJobFailed, "export", identityErr.Error(),
"快照已产生但来源身份不符,未登记为测试封存;核对执行器后重新封存", "", "", finishedAt)
return identityErr
}
// 封存产物成版本记录:只登记"测试封存",绝不写成生产生效版本。 // 封存产物成版本记录:只登记"测试封存",绝不写成生产生效版本。
versionErr := store.RecordLibraryVersion(ctx, governance.LibraryVersion{ versionErr := store.RecordLibraryVersion(ctx, governance.LibraryVersion{
ID: governance.LibraryVersionID(governance.LibraryVersionTestSealed, payload.Scope, result.SnapshotID), ID: governance.LibraryVersionID(governance.LibraryVersionTestSealed, payload.Scope, result.SnapshotID),
Scope: payload.Scope, Scope: payload.Scope,
Kind: governance.LibraryVersionTestSealed, SnapshotID: result.SnapshotID, SourceJobID: libraryJobID, Kind: governance.LibraryVersionTestSealed, SnapshotID: result.SnapshotID, SourceJobID: libraryJobID,
Evidence: map[string]any{"capture_job": payload.CaptureJob, "replay": result.Replay, Evidence: map[string]any{"capture_job": payload.CaptureJob, "replay": result.Replay,
"requested_by": payload.RequestedBy, "agent_result": json.RawMessage(result.Payload)}, "requested_by": payload.RequestedBy, "expectation_fingerprint": expectationFingerprint(expectation),
"agent_result": json.RawMessage(result.Payload)},
CreatedBy: payload.RequestedBy, CreatedAt: finishedAt, CreatedBy: payload.RequestedBy, CreatedAt: finishedAt,
}) })
if versionErr != nil { if versionErr != nil {
@@ -322,6 +329,10 @@ func workerHandlers(store *postgresstore.Store, config appconfig.Config) (map[st
handlers[libraryRestoreJobType] = func(ctx context.Context, job jobs.Job) error { handlers[libraryRestoreJobType] = func(ctx context.Context, job jobs.Job) error {
return runLibraryRelease(ctx, store, config, job, governance.LibraryJobRestore) return runLibraryRelease(ctx, store, config, job, governance.LibraryJobRestore)
} }
// 建立目标权威基线:目标非空又没有生效版本时的唯一入口(只读捕获 + 校验 + 登记)。
handlers[libraryBaselineJobType] = func(ctx context.Context, job jobs.Job) error {
return runLibraryBaseline(ctx, store, config, job)
}
// A crash can leave the source writer stopped: recover before taking work, // A crash can leave the source writer stopped: recover before taking work,
// and keep the failure visible instead of pretending the worker is ready. // and keep the failure visible instead of pretending the worker is ready.
go func() { go func() {
+27 -5
View File
@@ -1,7 +1,15 @@
# 整库同步验收矩阵(真实证据对照,2026-10-29 重排) # 整库同步验收矩阵(真实证据对照,2026-10-29 重排)
本文件是**唯一口径**:状态只写实际发生的事,四类严格分开,不用"部分通过"含糊过去。 本文件是**唯一口径**:状态只写实际发生的事,四类严格分开,不用"部分通过"含糊过去。
**2026-10-29 父审后重跑:V03-a(Wiki 撤销语义)不通过,单独列在「一·补」,不在「已完成」表里。**
**2026-10-29 用户决定(覆盖历史验收口径)**:发布管理**不承担** Wiki 语义撤销——
它只负责把测试端已确认的整库快照**原样还原**生产。因此撤销后正文里是否残留
撤销方独有事实,属于**上游知识质量**问题,不在本模块验收范围内:V03-a 保留原始
失败证据(见「一·补」下方的"模块范围外"表),不改成通过,也不再阻断发布。
同时**不**新增 LLM 重写、人工逐页复核闸门,也不把升级 WeKnora 作为前置条件。
保留的门槛是:整库文档/分块/索引/资源/Wiki 正文·来源·链接·目录一致、稳定切点、
实际维护与排空、准确快照授权、独立预备份与恢复、失败回滚、作业互斥与重启接管、
真实 UI 进度与存储安全。效果问答/性能/A06 不是门槛。
| 口径 | 含义 | | 口径 | 含义 |
| --- | --- | | --- | --- |
@@ -49,6 +57,8 @@ A06(内容、问答、引用图片效果、性能报告)已由用户取消
| **V03-c** | 带"撤销后共享页"的整库封存 → 独立目标同步 → 恢复,逐项真实读回 | `libraryctl` 封存 + `ProductionLibraryTarget` + `libraryrelease` | **本轮终态真跑(字节一致性,不是撤销语义)**:KB `e64e82b5`(9 Wiki 页、1 文档、2 分块、1 资源)→ 快照 `0f0a3b68…` 与源逐页一致(`05-snapshot-compare.json` ok=True)→ 同步到独立实例 kgseal `libsync-internal-1793229249632461601` succeeded → 目标逐页/索引/资源字节一致(`07-target-compare.json` ok=True,含目标桶对象 sha256)→ `librestore-internal-1793229274077866298` succeeded → 目标仍逐项一致。**注意**:该共享页正文仍带撤销方残留(见 V03-a 不通过),本行只证明"搬运/恢复逐字节一致" | | **V03-c** | 带"撤销后共享页"的整库封存 → 独立目标同步 → 恢复,逐项真实读回 | `libraryctl` 封存 + `ProductionLibraryTarget` + `libraryrelease` | **本轮终态真跑(字节一致性,不是撤销语义)**:KB `e64e82b5`(9 Wiki 页、1 文档、2 分块、1 资源)→ 快照 `0f0a3b68…` 与源逐页一致(`05-snapshot-compare.json` ok=True)→ 同步到独立实例 kgseal `libsync-internal-1793229249632461601` succeeded → 目标逐页/索引/资源字节一致(`07-target-compare.json` ok=True,含目标桶对象 sha256)→ `librestore-internal-1793229274077866298` succeeded → 目标仍逐项一致。**注意**:该共享页正文仍带撤销方残留(见 V03-a 不通过),本行只证明"搬运/恢复逐字节一致" |
| **S06-b** | 主机侧封存/恢复必须与 worker 讲同一份维护声明 | `internal/librarymaintenance/env.go`(`LIBRARY_TARGET_*` 唯一解析处) | 缺项时 `capture` 在开窗口前拒绝并点名;agent 启动打印维护目标;单测 5 项(逐项缺失/空白/口令空格/队列库号) | | **S06-b** | 主机侧封存/恢复必须与 worker 讲同一份维护声明 | `internal/librarymaintenance/env.go`(`LIBRARY_TARGET_*` 唯一解析处) | 缺项时 `capture` 在开窗口前拒绝并点名;agent 启动打印维护目标;单测 5 项(逐项缺失/空白/口令空格/队列库号) |
| **Neo4j 启用场景** | 启用图谱的库必须被真实拒绝 | `v071/library_export.go` 读 `indexing_strategy.graph_enabled` | **本轮真跑**:在 kgmaint 建 `graph_enabled=true` 的真实 KB `8de41ca7…`,用真实 CLI 预检 → 退出码 1、`LIBRARYCTL_FAILED: Neo4j graph snapshot is not supported`(`evidence/10-neo4j-refuse.txt`)。不是 skip | | **Neo4j 启用场景** | 启用图谱的库必须被真实拒绝 | `v071/library_export.go` 读 `indexing_strategy.graph_enabled` | **本轮真跑**:在 kgmaint 建 `graph_enabled=true` 的真实 KB `8de41ca7…`,用真实 CLI 预检 → 退出码 1、`LIBRARYCTL_FAILED: Neo4j graph snapshot is not supported`(`evidence/10-neo4j-refuse.txt`)。不是 skip |
| **执行器可用性(运行时观测)** | 页面的"可发起封存"来自 worker 的真实探测,不是配置字符串 | `internal/libraryexecutor` + agent `health` + 迁移 074 `library_executor_status` + worker 30s 心跳 | **本轮真跑(候选栈,代码 SHA 4c79052 / 0.1.192 的工作树构建)**:`GET /library/overview` 返回 `capture_available=true` 且 `executor={instance:kgmaint-weknora, project:kgmaint-weknora, source_app:kgmaint-app, observed_at:…}`;停候选 agent 后同一接口转 `false`、原因 `主机侧封存执行器不可达:library agent is unreachable`,此时 `POST /library/captures` 返回 **503 `library_capture_unavailable`**;重启 agent 后自动恢复 `true`。证据 `$TMPDIR/kg-candidate-verify/evidence/{01,03,04,05}-*.json` |
| **按版本恢复入口** | 页面不再把"恢复至该版本"写死 disabled;条件来自双闸/授权/维护 | `GET /api/v1/library/restore-availability` + `web/src/ReleaseTargets.tsx` | 接口真实返回 200(候选栈,缺授权时逐条列出原因);页面按 `allowed` 启用、管理员才见"签发恢复授权(30 分钟)",授权与执行是两步。证据 `02-restore-availability.json` + 前端用例 `libraryRestore.test.ts` / `ReleaseTargets.test.tsx` |
| **U04** | 维护级别/进度/恢复错误真实呈现 | `MaintenanceCard`(`web/src/ReleaseTargets.tsx`)、`/api/v1/library/maintenance` | 显示实例级/库级、持有 scope+作业、阶段、入口开/关、服务、开始/释放、进度(阶段名中文化)、恢复错误与处理记录;仅"窗口仍持有且作业失败"时给"按预备份恢复";无记录时"未接入 + fail closed"。组件测试 6 项 | | **U04** | 维护级别/进度/恢复错误真实呈现 | `MaintenanceCard`(`web/src/ReleaseTargets.tsx`)、`/api/v1/library/maintenance` | 显示实例级/库级、持有 scope+作业、阶段、入口开/关、服务、开始/释放、进度(阶段名中文化)、恢复错误与处理记录;仅"窗口仍持有且作业失败"时给"按预备份恢复";无记录时"未接入 + fail closed"。组件测试 6 项 |
| **V05** | 导出中源变更必须被入口/写入者阻断,切点一致 | 闸门 `drain.json` + 真实并发客户端 | **本轮终态真跑**:入口关闭的同一刻(固定协调点,非 sleep)对专用源发 8 写 + 4 读 → 8 写全部失败(7 `URLError`/1 `RemoteDisconnected`)、4 读 `RemoteDisconnected`;写入者被停下的那一刻(轮询容器状态捕获 `running→exited→running`)再发 4 写 + 2 读 → 全部 `URLError`。事后源库 `v05` 文档 0 / 分块 0,快照与源逐项一致(ok=True),在途文档 `f7fed988…` completed/7 分块且出现在快照里。证据 `04-inflight-v05-final.json` | | **V05** | 导出中源变更必须被入口/写入者阻断,切点一致 | 闸门 `drain.json` + 真实并发客户端 | **本轮终态真跑**:入口关闭的同一刻(固定协调点,非 sleep)对专用源发 8 写 + 4 读 → 8 写全部失败(7 `URLError`/1 `RemoteDisconnected`)、4 读 `RemoteDisconnected`;写入者被停下的那一刻(轮询容器状态捕获 `running→exited→running`)再发 4 写 + 2 读 → 全部 `URLError`。事后源库 `v05` 文档 0 / 分块 0,快照与源逐项一致(ok=True),在途文档 `f7fed988…` completed/7 分块且出现在快照里。证据 `04-inflight-v05-final.json` |
@@ -59,7 +69,13 @@ A06(内容、问答、引用图片效果、性能报告)已由用户取消
| ID | 要点 | 承载位置 | 本轮真实结果 | | ID | 要点 | 承载位置 | 本轮真实结果 |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| **V03-a** | 撤销一份共享贡献后,保留另一份贡献、清除撤销方独有正文/引用/实体边、目录正确 | 由 WeKnora 源侧执行(`cleanupWikiOnKnowledgeDelete` + `EnqueueWikiRetract`),治理侧只比对/搬运 | **本轮不通过(2026-10-29 重跑)**:真实 `DELETE /api/v1/knowledge/{id}` 后,共享页 `entity/fa-bu-zha-men` 的来源/引用/边/目录都按上游 retract 更新(source_refs 只剩保留文档、引用 2→1、9 页/18 边无死链),**但正文仍保留撤销方独有的两句事实 `ZEBRA-A-7731`/`ZEBRA-A-9910`**(version=3、3733 字节、sha256 `5fd27348…`)。封存逐字节一致只说明"存了真实状态",不等于撤销语义通过。证据 `kg-final-gaps/evidence/14-wiki-residue-evidence.json` + `01-wiki-source-after-r3.json` | | **(无)** | 本模块没有"不通过"项:撤销语义已明确移出范围(见上表下方"模块范围外"),不在此改写为通过 | — | — |
### 模块范围外(用户 2026-10-29 决定;证据保留,不改判)
| ID | 要点 | 本轮真实结果(原样保留) |
| --- | --- | --- |
| **V03-a** | 撤销一份共享贡献后,清除撤销方独有正文/引用/实体边 | **上游不通过,模块范围外**:真实 `DELETE /api/v1/knowledge/{id}` 后,共享页 `entity/fa-bu-zha-men` 的来源/引用/边/目录都按上游 retract 更新(source_refs 只剩保留文档、引用 2→1、9 页/18 边无死链),**但正文仍保留撤销方独有的两句事实 `ZEBRA-A-7731`/`ZEBRA-A-9910`**(version=3、3733 字节、sha256 `5fd27348…`)。这是上游 retract 走 LLM 重写的非确定性结果,治理侧只按字节搬运,不能也不应改写它。证据 `kg-final-gaps/evidence/14-wiki-residue-evidence.json` + `01-wiki-source-after-r3.json`,不涂改 |
## 二、缺代码 ## 二、缺代码
@@ -74,6 +90,8 @@ A06(内容、问答、引用图片效果、性能报告)已由用户取消
| --- | --- | --- | | --- | --- | --- |
| S03-b | 死信/孤儿任务阻断的负向演练(本轮只验证了真实死信阻断) | 真实阻断已发生并记录;系统化的负向矩阵未跑 | | S03-b | 死信/孤儿任务阻断的负向演练(本轮只验证了真实死信阻断) | 真实阻断已发生并记录;系统化的负向矩阵未跑 |
| 排空时序(历史结论) | 旧做法"202 之后抢时序上传"在 1faf513 上重跑 6 次全部 `Broken pipe`(`drain-task-final-1faf513*.json` 保留,不记通过) | 已被 S06-a 的确定性做法取代:先证明请求已被闸门接收,再启动维护。旧证据保留,不覆盖 | | 排空时序(历史结论) | 旧做法"202 之后抢时序上传"在 1faf513 上重跑 6 次全部 `Broken pipe`(`drain-task-final-1faf513*.json` 保留,不记通过) | 已被 S06-a 的确定性做法取代:先证明请求已被闸门接收,再启动维护。旧证据保留,不覆盖 |
| 封存执行期身份复核(真实封存) | 期望身份随 capture 请求送进 agent,由它在操作互斥内、任何源操作之前比对(`swap_test.go` 用两个真实 agent 的 socket 替换做负例,断言错误 agent 的 capture 调用 0 次);真实封存尚未在最终构建上跑,本轮**没有**在候选栈发起真实封存(会产生新快照与停写入者窗口),因此"换 socket 被挡住"只有单测证据 |
| 按版本恢复的真实执行(新构建) | `restore-availability` 真实返回、恢复投递路径有单测;真实恢复作业仍在旧构建上验过(R01),本轮未重跑,不把旧证据算作新构建证据 |
| U05 | 正式 8088 页面验收 | 正式栈仍是 0.1.114,本轮未替换;维护面板/逐页 Wiki 差异只在测试与候选构建里验证过 | | U05 | 正式 8088 页面验收 | 正式栈仍是 0.1.114,本轮未替换;维护面板/逐页 Wiki 差异只在测试与候选构建里验证过 |
## 四、需授权 / 需环境 ## 四、需授权 / 需环境
@@ -97,9 +115,13 @@ api env `LIBRARY_AGENT_SOCKET=/run/library-agent/agent.sock`,worker 挂载
本轮没有启动任何共享维护,也没有改回该单元——需要用户决定:正式上线前把单元恢复为 WKNFull 版 本轮没有启动任何共享维护,也没有改回该单元——需要用户决定:正式上线前把单元恢复为 WKNFull 版
(备份已在位),或给候选构建单独的 socket 路径。证据 `00-identity-final.json` 的 `formal_stack` 段。 (备份已在位),或给候选构建单独的 socket 路径。证据 `00-identity-final.json` 的 `formal_stack` 段。
另外:`knowledge-governance-p1-release-worker-1` 自 2026-10-10 起就在崩溃重启(`exit=1`、 另外:`knowledge-governance-p1-release-worker-1` 自 2026-10-10 起就在崩溃重启(`exit=1`,
`RestartCount≈1233`,日志只有 `connect release target database`),与本轮改动无关(它没有挂 agent socket), `RestartCount` 已达 1500+,日志只有 `connect release target database`)。**本轮只读定位到根因**:
但正式栈的发布 worker 实际不可用,需要单独处理。 `config/release.sim.json` 的 `target.database_url_file` 指向 `prod-pg:25432`,而该主机名在本机与
容器网络里都不存在(`getent hosts prod-pg` 无结果、容器内 NXDOMAIN);`source` 指向的
`WKNFull-postgres` 可解析,所以是**目标库配置遗留**,不是代码缺陷。错误文本吞掉了主机与原因,
本轮已改为带端点 + 路由 + 脱敏原因(可在下次重启时直接看到),但**修正正式配置或停用该 worker
属于正式栈变更,需要部署窗口授权**,本轮没有执行。
--- ---
+17
View File
@@ -87,6 +87,23 @@ run164 因 qawemail 依赖外网 DNS 失败,已修,不涂改。
| 0.1.185 | c542a133 | 资源行 storage_backend_id 列也改到目标实例 | 226 | **success** | 否 | | 0.1.185 | c542a133 | 资源行 storage_backend_id 列也改到目标实例 | 226 | **success** | 否 |
| 0.1.186 | e508abd9 | 死信解除绑定封存源实例(LIBRARY_SOURCE_INSTANCE) | 227 | **success** | 否 | | 0.1.186 | e508abd9 | 死信解除绑定封存源实例(LIBRARY_SOURCE_INSTANCE) | 227 | **success** | 否 |
| 0.1.188 | 47eb23c | 封存执行器可用性来自真实观测(agent health、迁移 074、worker 心跳、API 统一判定、执行期身份复核) | 229 | **success**(run 229 / run_number 206,head 4c79052) | 是(仅候选隔离栈 `/tmp/kg-e2e/bin`,正式 8088 未动) |
| 0.1.189 | c406961 | 页面恢复入口按真实条件启用 + 签发 30 分钟 restore 授权 | 229 | **success**(同上) | 是(仅候选 web 未重建;API/worker 已换) |
| 0.1.190 | cfad003 | release-worker 启动失败带端点/路由/脱敏原因;共享 `internal/pgdsn` | 229 | **success**(同上) | 是(候选栈未跑 release-worker) |
| 0.1.191 | 18276d0 | 文档:验收口径按用户决定收窄(V03-a 移出范围)、矩阵补执行器证据 | 229 | **success**(同上) | 否(只改文档) |
| 0.1.192 | 4c79052 | 测试 DSN 拼接构造(密钥扫描误报修复,语义不变) | 229 | **success**(head 4c79052,必需 context `build / build (push)` = success) | 否(只改测试) |
| 0.1.195 | 6ed0635 | 期望执行器身份随 capture 请求送达并在执行前核对(TOCTOU) | 233 | **success**(head c14dd01 之前的 run 232/233 连续绿;最终以 8d2ed8e/run234 为准) | 是(候选栈) |
| 0.1.196 | c14dd01 | 封存读资源失败带资源行与底层原因 | 233 | **success**(run 233 / run_number 210,head c14dd01,必需 context success) | 是(候选栈,clean 构建 `vcs.modified=false`) |
| 0.1.197 | 8d2ed8e | 维护释放与服务恢复完成后才暴露作业成功(R4) | 234 | **success**(run 234 / run_number 211,head 8d2ed8e,必需 context success) | 是(候选栈,clean 构建 `vcs.modified=false`;真实封存→同步→恢复与"成功后立即恢复 202"均在该构建上跑过) |
| 0.1.198 | 691e4e0 | 台账补记 0.1.195–0.1.197 | 235 | **success**(run 235,head 691e4e0) | 否(文档) |
| 0.1.199 | 24d2546 | 接管分类(未知/待收尾/被顶替)+ 维护释放幂等 | 236 | **success**(run 236,head 24d2546,必需 context success) | 是(候选栈,`-ldflags` 注入身份;`build-info` 读回 0.1.199 / 24d2546…,进程 `vcs.modified=false`) |
这批提交的候选栈验证:候选 api/worker/libraryagent/libraryctl 由该工作树重新构建(`vcs.modified=true`,
不是发布产物),二进制 sha256 记在 `$TMPDIR/kg-candidate-verify/evidence/01-binaries.json`;
旧二进制整目录备份在该目录下的 `bin-prev-*`,可一条命令回退。真实观测证据见
`$TMPDIR/kg-candidate-verify/evidence/{00..10}*.json`。
本轮(父审后)真实验收运行的最终代码 SHA 是 **0.1.186(e508abd9)**:封存→同步→恢复、 本轮(父审后)真实验收运行的最终代码 SHA 是 **0.1.186(e508abd9)**:封存→同步→恢复、
B02 删源对象后目标应用读回、V05 并发阻断都在这一份构建上跑完(PID/exe/revision 绑定见 B02 删源对象后目标应用读回、V05 并发阻断都在这一份构建上跑完(PID/exe/revision 绑定见
`kg-final-gaps/evidence/00-identity-final.json`)。0.1.181–0.1.183 是同一轮内的连续修复, `kg-final-gaps/evidence/00-identity-final.json`)。0.1.181–0.1.183 是同一轮内的连续修复,
+30 -5
View File
@@ -99,7 +99,13 @@ type Config struct {
LibraryTargetInstance string LibraryTargetInstance string
// LibrarySourceInstance 是死信解除记录绑定的"来源实例"名字(封存源,不是同步目标)。 // LibrarySourceInstance 是死信解除记录绑定的"来源实例"名字(封存源,不是同步目标)。
// 留空时回落到 LibraryTargetInstance,保持单实例部署的旧行为。 // 留空时回落到 LibraryTargetInstance,保持单实例部署的旧行为。
LibrarySourceInstance string LibrarySourceInstance string
// LibrarySourceProject 是封存执行器应该指向的实例名(agent -project)。
// 留空时等于来源实例名:环境里两者本来就同名(kgmaint-weknora)。
LibrarySourceProject string
// LibrarySourceApp 是封存源容器名(agent -source-app)。project 可能同名而源接线
// 不同,所以配了就一起绑定进"期望执行器身份"。
LibrarySourceApp string
LibraryTargetProject string LibraryTargetProject string
LibraryTargetWriters []string LibraryTargetWriters []string
LibraryTargetNonWriters map[string]string LibraryTargetNonWriters map[string]string
@@ -115,10 +121,13 @@ type Config struct {
LibraryTargetJournalRoot string LibraryTargetJournalRoot string
// LibraryAgentSocket 是主机侧封存 agent 的 unix socket(只由 worker 连接)。 // LibraryAgentSocket 是主机侧封存 agent 的 unix socket(只由 worker 连接)。
// 留空即"未接线":页面不显示可用的封存入口,接口返回原因而不是假装已排队。 // 留空即"未接线":页面不显示可用的封存入口,接口返回原因而不是假装已排队。
LibraryAgentSocket string LibraryAgentSocket string
VersionHistoryLinkOrigin string // LibraryExecutorStatusMaxAge 是执行器状态的有效期:超过它就是"无法确认",
VersionHistoryHTTPTimeout time.Duration // 不是"仍然可用"。worker 每 30 秒观测一次,默认 120 秒足够容错。
VersionHistoryMaxPages int LibraryExecutorStatusMaxAge time.Duration
VersionHistoryLinkOrigin string
VersionHistoryHTTPTimeout time.Duration
VersionHistoryMaxPages int
} }
func Load(getenv func(string) string) (Config, error) { func Load(getenv func(string) string) (Config, error) {
@@ -203,6 +212,8 @@ func Load(getenv func(string) string) (Config, error) {
LibraryProductionS3SecretKey: strings.TrimSpace(getenv("LIBRARY_PRODUCTION_S3_SECRET_KEY")), LibraryProductionS3SecretKey: strings.TrimSpace(getenv("LIBRARY_PRODUCTION_S3_SECRET_KEY")),
LibraryTargetInstance: strings.TrimSpace(getenv("LIBRARY_TARGET_INSTANCE")), LibraryTargetInstance: strings.TrimSpace(getenv("LIBRARY_TARGET_INSTANCE")),
LibrarySourceInstance: strings.TrimSpace(getenv("LIBRARY_SOURCE_INSTANCE")), LibrarySourceInstance: strings.TrimSpace(getenv("LIBRARY_SOURCE_INSTANCE")),
LibrarySourceProject: strings.TrimSpace(getenv("LIBRARY_SOURCE_PROJECT")),
LibrarySourceApp: strings.TrimSpace(getenv("LIBRARY_SOURCE_APP")),
LibraryTargetProject: strings.TrimSpace(getenv("LIBRARY_TARGET_PROJECT")), LibraryTargetProject: strings.TrimSpace(getenv("LIBRARY_TARGET_PROJECT")),
LibraryTargetWriters: splitList(getenv("LIBRARY_TARGET_WRITERS")), LibraryTargetWriters: splitList(getenv("LIBRARY_TARGET_WRITERS")),
LibraryTargetNonWriters: splitDeclarations(getenv("LIBRARY_TARGET_NON_WRITERS")), LibraryTargetNonWriters: splitDeclarations(getenv("LIBRARY_TARGET_NON_WRITERS")),
@@ -219,6 +230,7 @@ func Load(getenv func(string) string) (Config, error) {
LibraryCapacityBudgetBytes: capacityBudget, LibraryCapacityBudgetBytes: capacityBudget,
LibraryRetentionVersions: retentionVersions, LibraryRetentionVersions: retentionVersions,
LibraryRetentionDays: retentionDays, LibraryRetentionDays: retentionDays,
LibraryExecutorStatusMaxAge: 2 * time.Minute,
VersionHistoryLinkOrigin: strings.TrimSpace(getenv("VERSION_HISTORY_LINK_ORIGIN")), VersionHistoryLinkOrigin: strings.TrimSpace(getenv("VERSION_HISTORY_LINK_ORIGIN")),
VersionHistoryHTTPTimeout: 15 * time.Second, VersionHistoryHTTPTimeout: 15 * time.Second,
VersionHistoryMaxPages: valueOrDefaultInt(getenv("VERSION_HISTORY_MAX_PAGES"), 200), VersionHistoryMaxPages: valueOrDefaultInt(getenv("VERSION_HISTORY_MAX_PAGES"), 200),
@@ -241,6 +253,9 @@ func Load(getenv func(string) string) (Config, error) {
if config.SourceRefreshSchedulerInterval < 5*time.Second { if config.SourceRefreshSchedulerInterval < 5*time.Second {
return Config{}, errors.New("SOURCE_REFRESH_SCHEDULER_INTERVAL must be at least 5s") return Config{}, errors.New("SOURCE_REFRESH_SCHEDULER_INTERVAL must be at least 5s")
} }
if config.LibraryExecutorStatusMaxAge, err = durationOrDefault(getenv("LIBRARY_EXECUTOR_STATUS_MAX_AGE"), config.LibraryExecutorStatusMaxAge); err != nil {
return Config{}, fmt.Errorf("LIBRARY_EXECUTOR_STATUS_MAX_AGE: %w", err)
}
if raw := getenv("RELEASE_WORKER_ENABLED"); raw != "" { if raw := getenv("RELEASE_WORKER_ENABLED"); raw != "" {
config.ReleaseWorkerEnabled, err = strconv.ParseBool(raw) config.ReleaseWorkerEnabled, err = strconv.ParseBool(raw)
if err != nil { if err != nil {
@@ -575,3 +590,13 @@ func (config Config) SourceInstance() string {
} }
return config.LibraryTargetInstance return config.LibraryTargetInstance
} }
// SourceProject 是封存执行器自报 project 的期望值。它描述的是**封存源**
// (agent 冻结哪一套写入者),不是同步目标:拿目标实例去比会把正确的执行器判成
// 不匹配(本机候选栈就是这种形状:源 kgmaint、目标 kgseal)。
func (config Config) SourceProject() string {
if config.LibrarySourceProject != "" {
return config.LibrarySourceProject
}
return config.SourceInstance()
}
+10
View File
@@ -31,6 +31,10 @@ const (
LibraryJobSync LibraryJobKind = "sync" LibraryJobSync LibraryJobKind = "sync"
// LibraryJobRestore 是按指定整库版本恢复:与同步分开,授权与阶段也独立。 // LibraryJobRestore 是按指定整库版本恢复:与同步分开,授权与阶段也独立。
LibraryJobRestore LibraryJobKind = "restore" LibraryJobRestore LibraryJobKind = "restore"
// LibraryJobBaseline 建立首次发布目标的权威基线:在维护隔离下把目标整库
// 读出来封存成快照并逐类校验,再登记成 production_effective。
// 它不写目标内容、不清空目标、不猜版本——基线只能来自真实捕获与校验。
LibraryJobBaseline LibraryJobKind = "baseline"
) )
type LibraryJobState string type LibraryJobState string
@@ -134,6 +138,12 @@ func SealStages() []string {
return []string{"export"} return []string{"export"}
} }
// BaselineStages 是"建立目标权威基线"的阶段:捕获目标整库 → 与快照逐类校验
// → 登记生效版本。没有 rollback 阶段:基线过程不写目标内容。
func BaselineStages() []string {
return []string{"capture", "verify", "finalize"}
}
// LibraryReleaseAuthorization 是生产发布授权:绑定目标、快照、参数摘要与有效期。 // LibraryReleaseAuthorization 是生产发布授权:绑定目标、快照、参数摘要与有效期。
type LibraryReleaseAuthorization struct { type LibraryReleaseAuthorization struct {
ID string `json:"id"` ID string `json:"id"`
+156
View File
@@ -0,0 +1,156 @@
package httpapi
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"time"
"knowledge-governance/internal/governance"
"knowledge-governance/internal/jobs"
postgresstore "knowledge-governance/internal/platform/postgres"
)
// 建立目标权威基线:首次发布时目标已经有内容,却没有生效版本记录。
//
// 唯一诚实的解法是"维护隔离 + 真实整库捕获 + 逐类校验 + 登记版本":
// - 不手写 production_effective,不伪造验收;
// - 不清空目标换库绕过(那会把"非空但未知"变成"空");
// - 页面只投递作业,停写入者、捕获、校验都在 worker 里执行。
//
// 这个入口只建作业,不签发发布授权:基线不等于发布,同步仍然要单独的 sync 授权。
const libraryBaselineJobType = "library.baseline"
type libraryBaselineRequest struct {
Scope string `json:"scope"`
}
func libraryBaselineEnqueue(w http.ResponseWriter, r *http.Request, config Config) {
if r.Method != http.MethodPost {
writeJSON(w, http.StatusMethodNotAllowed, map[string]string{"error": "method not allowed"})
return
}
identity, ok := sessionFrom(r.Context())
if !ok || roleRank(identity.Role) < roleRank(roleAdmin) {
writeJSON(w, http.StatusForbidden, map[string]any{"error": "建立目标权威基线仅管理员可发起"})
return
}
var request libraryBaselineRequest
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&request); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid request body"})
return
}
if request.Scope != "internal" && request.Scope != "external" {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "scope must be internal or external"})
return
}
store, ok := libraryStore(config)
if !ok {
writeJSON(w, http.StatusServiceUnavailable, map[string]any{"code": "library_baseline_unavailable",
"error": "版本登记存储不可用"})
return
}
versions, err := store.LibraryVersions(r.Context(), request.Scope, 50)
if err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "cannot read library versions"})
return
}
for _, item := range versions {
if item.Kind == governance.LibraryVersionProductionEffective && item.SupersededAt == nil {
writeJSON(w, http.StatusConflict, map[string]any{"code": "library_baseline_exists",
"error": "该目标已有生效版本,不需要再建基线;换版本请走同步或恢复",
"snapshot_id": item.SnapshotID})
return
}
}
// 目标必须真的读得到:读不到就不是"非空无权威版本",而是另一个问题,
// 不允许用建基线去掩盖连接/配置故障。
probe := config.LibraryScopeProbe
if probe == nil {
probe = func(ctx context.Context, scope string) (bool, map[string]int64, error) {
return productionScopeState(ctx, config, scope)
}
}
empty, counts, probeErr := probe(r.Context(), request.Scope)
if probeErr != nil {
writeJSON(w, http.StatusServiceUnavailable, map[string]any{"code": "library_baseline_unprobeable",
"error": "无法读取目标整库状态,不能建立基线:" + probeErr.Error()})
return
}
if empty {
writeJSON(w, http.StatusConflict, map[string]any{"code": "library_baseline_target_empty",
"error": "目标整库为空,不需要基线:首次同步会在写入前生成完整预备份并登记生效版本",
"counts": counts})
return
}
// 维护互斥与同步一致:基线要在目标上开维护窗口,目标正在维护时直接拒绝。
if body, held := instanceMaintenanceGuard(r.Context(), config, request.Scope); held {
writeJSON(w, http.StatusConflict, body)
return
}
if maintenanceStore, ok := config.Store.(interface {
LibraryMaintenanceOf(ctx context.Context, scope string) (*postgresstore.LibraryMaintenance, error)
}); ok {
if held, err := maintenanceStore.LibraryMaintenanceOf(r.Context(), request.Scope); err == nil && held != nil && held.ReleasedAt == nil {
writeJSON(w, http.StatusConflict, map[string]any{"code": "library_target_in_maintenance",
"error": "目标正在维护中,验证完成前不接受新的写入请求",
"owner_job": held.OwnerJobID, "state": held.State})
return
}
}
digest, err := governance.LibraryRequestDigest(request.Scope, governance.LibraryJobBaseline,
map[string]any{"scope": request.Scope})
if err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "cannot digest request"})
return
}
now := time.Now().UTC()
job, reused, err := store.CreateLibraryJob(r.Context(), governance.LibraryJob{
ID: fmt.Sprintf("libbase-%s-%d", request.Scope, now.UnixNano()), Scope: request.Scope,
Kind: governance.LibraryJobBaseline,
IdempotencyKey: "library.baseline:" + request.Scope + ":" + now.Format("20060102T150405.000000000"),
RequestDigest: digest, Actor: identity.Username,
}, governance.BaselineStages(), now)
if errors.Is(err, governance.ErrLibraryJobActive) {
writeJSON(w, http.StatusConflict, map[string]any{"code": "library_write_active",
"error": "该目标已有在飞的写作业(同步/恢复/基线互斥),等它结束再发起"})
return
}
if errors.Is(err, governance.ErrLibraryJobIdempotencyConflict) {
writeJSON(w, http.StatusConflict, map[string]any{"code": "library_baseline_conflict",
"error": "同一幂等键对应了不同的请求"})
return
}
if err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "cannot create library baseline job"})
return
}
if reused {
writeJSON(w, http.StatusAccepted, map[string]any{"job": job, "reused": true})
return
}
payload, err := json.Marshal(map[string]string{"library_job_id": job.ID, "scope": request.Scope})
if err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "cannot build baseline payload"})
return
}
queueJob, err := config.Store.Enqueue(r.Context(), jobs.EnqueueRequest{
ID: governance.QueueJobIDFor(job.ID), Type: libraryBaselineJobType, Payload: payload,
AvailableAt: now, MaxAttempts: 1, IdempotencyKey: "library.baseline.queue:" + job.ID,
}, now)
if err != nil || queueJob == nil {
// 作业先落库、队列投递失败就把作业取消:不留下"看起来在跑"的空作业,
// 否则它会一直占着该 scope 的写互斥。
_ = store.CancelLibraryJob(r.Context(), job.ID, "队列投递失败,未开始建立基线", now)
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "cannot enqueue baseline job"})
return
}
if attachErr := store.AttachLibraryQueueJob(r.Context(), job.ID, queueJob.ID, now); attachErr != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "cannot attach baseline job"})
return
}
writeJSON(w, http.StatusAccepted, map[string]any{"job": job, "reused": false,
"target_counts": counts})
}
@@ -0,0 +1,97 @@
package httpapi
import (
"context"
"errors"
"net/http"
"net/http/httptest"
"testing"
"time"
"knowledge-governance/internal/governance"
"knowledge-governance/internal/jobs"
)
// nonEmptyTarget 模拟"目标整库非空且可读"的探测结果(真实实现直连目标库逐表计数)。
func nonEmptyTarget(context.Context, string) (bool, map[string]int64, error) {
return false, map[string]int64{"knowledges": 1, "chunks": 2}, nil
}
// scopedProbe:internal 可读非空,external 没有目标 KB(与当前部署一致)。
func scopedProbe(_ context.Context, scope string) (bool, map[string]int64, error) {
if scope == "external" {
return false, nil, errors.New("该范围的目标知识库未配置")
}
return nonEmptyTarget(context.Background(), scope)
}
// baselineMutexStore 在 fake 之外**真的**实现"同一 scope 只允许一个在飞写作业"这条不变量,
// 用来验证接口的重试/双击行为(真实索引由 integration test
// TestLibraryWriteMutexCoversBaselineJobs 在 PostgreSQL 上证明)。
type baselineMutexStore struct {
fakeLibraryStore
activeByScope map[string]string
enqueues int
}
func (s *baselineMutexStore) CreateLibraryJob(ctx context.Context, job governance.LibraryJob, stages []string, now time.Time) (governance.LibraryJob, bool, error) {
if s.activeByScope == nil {
s.activeByScope = map[string]string{}
}
if owner := s.activeByScope[job.Scope]; owner != "" {
return governance.LibraryJob{}, false, governance.ErrLibraryJobActive
}
s.activeByScope[job.Scope] = job.ID
return s.fakeLibraryStore.CreateLibraryJob(ctx, job, stages, now)
}
func (s *baselineMutexStore) Enqueue(ctx context.Context, request jobs.EnqueueRequest, now time.Time) (*jobs.Job, error) {
s.enqueues++
return s.fakeLibraryStore.fakeJobStore.Enqueue(ctx, request, now)
}
// 双击/重试:第一次建作业成功,第二次被互斥拒绝,且**不会**再投递一个队列作业。
func TestLibraryBaselineDoubleSubmitCreatesOneJob(t *testing.T) {
store := &baselineMutexStore{fakeLibraryStore: fakeLibraryStore{fakeJobStore: fakeJobStore{enqueueEchoRequestedID: true}}}
config := Config{Store: store, LibraryScopeProbe: nonEmptyTarget}
first := httptest.NewRecorder()
libraryBaselineEnqueue(first, baselineRequest(`{"scope":"internal"}`, roleAdmin, "admin"), config)
if first.Code != http.StatusAccepted {
t.Fatalf("first status=%d body=%s", first.Code, first.Body.String())
}
if store.enqueues != 1 {
t.Fatalf("first submit enqueues=%d, want 1", store.enqueues)
}
second := httptest.NewRecorder()
libraryBaselineEnqueue(second, baselineRequest(`{"scope":"internal"}`, roleAdmin, "admin"), config)
if second.Code != http.StatusConflict {
t.Fatalf("second status=%d body=%s", second.Code, second.Body.String())
}
if body := decodeBody(t, second); body["code"] != "library_write_active" {
t.Fatalf("second body=%v", body)
}
if store.enqueues != 1 {
t.Fatalf("rejected submit must not enqueue again: enqueues=%d", store.enqueues)
}
}
// 不同 scope 不会被同一把锁挡住(互斥粒度是 scope/实例,不是全局)。
func TestLibraryBaselineDifferentScopesAreNotBlocked(t *testing.T) {
store := &baselineMutexStore{fakeLibraryStore: fakeLibraryStore{fakeJobStore: fakeJobStore{enqueueEchoRequestedID: true}}}
config := Config{Store: store, LibraryScopeProbe: scopedProbe}
internal := httptest.NewRecorder()
libraryBaselineEnqueue(internal, baselineRequest(`{"scope":"internal"}`, roleAdmin, "admin"), config)
if internal.Code != http.StatusAccepted {
t.Fatalf("internal status=%d body=%s", internal.Code, internal.Body.String())
}
// external 在探针阶段就会被拒(该范围没有目标 KB),但绝不能是因为互斥。
external := httptest.NewRecorder()
libraryBaselineEnqueue(external, baselineRequest(`{"scope":"external"}`, roleAdmin, "admin"), config)
if external.Code == http.StatusAccepted {
t.Fatalf("external without a target KB must not be accepted: %s", external.Body.String())
}
body := decodeBody(t, external)
if body["code"] == "library_write_active" {
t.Fatalf("external must not be blocked by the internal scope's mutex: %v", body)
}
}
+162
View File
@@ -0,0 +1,162 @@
package httpapi
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"knowledge-governance/internal/governance"
)
// 建立目标权威基线的入口契约:
// - 只有管理员能发起(它是生产侧维护动作);
// - 已有生效版本时拒绝(基线只解决首次发布);
// - 目标读不到时拒绝,且**不建作业、不登记版本**(不能用基线掩盖连接问题)。
func baselineRequest(body, role, username string) *http.Request {
request := httptest.NewRequest(http.MethodPost, "/api/v1/library/baselines", strings.NewReader(body))
request.Header.Set("Content-Type", "application/json")
ctx := context.WithValue(request.Context(), sessionKey{}, sessionIdentity{Username: username, Role: role})
return request.WithContext(ctx)
}
func decodeBody(t *testing.T, recorder *httptest.ResponseRecorder) map[string]any {
t.Helper()
var payload map[string]any
if err := json.Unmarshal(recorder.Body.Bytes(), &payload); err != nil {
t.Fatalf("response is not JSON: %s", recorder.Body.String())
}
return payload
}
func TestLibraryBaselineRequiresAdmin(t *testing.T) {
store := &fakeLibraryStore{}
recorder := httptest.NewRecorder()
libraryBaselineEnqueue(recorder, baselineRequest(`{"scope":"internal"}`, roleEngineer, "eng"), Config{Store: store})
if recorder.Code != http.StatusForbidden {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
}
if store.createN != 0 {
t.Fatal("non-admin must not create a baseline job")
}
}
func TestLibraryBaselineRefusesWhenEffectiveVersionExists(t *testing.T) {
store := &fakeLibraryStore{versions: []governance.LibraryVersion{{
ID: "libver-1", Scope: "internal", Kind: governance.LibraryVersionProductionEffective,
SnapshotID: "snap-effective", CreatedAt: time.Now().UTC(),
}}}
recorder := httptest.NewRecorder()
libraryBaselineEnqueue(recorder, baselineRequest(`{"scope":"internal"}`, roleAdmin, "admin"), Config{Store: store})
if recorder.Code != http.StatusConflict {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
}
if body := decodeBody(t, recorder); body["code"] != "library_baseline_exists" || body["snapshot_id"] != "snap-effective" {
t.Fatalf("body=%v", body)
}
if store.createN != 0 {
t.Fatal("existing effective version must not create another baseline job")
}
}
// 目标读不到(这里用"没配置目标库"复现):必须 503 并说明原因,
// 既不能建作业,也不能顺手写一条版本记录来"解决"未知状态。
func TestLibraryBaselineRefusesWhenTargetUnreadable(t *testing.T) {
store := &fakeLibraryStore{}
recorder := httptest.NewRecorder()
libraryBaselineEnqueue(recorder, baselineRequest(`{"scope":"external"}`, roleAdmin, "admin"), Config{Store: store})
if recorder.Code != http.StatusServiceUnavailable {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
}
body := decodeBody(t, recorder)
if body["code"] != "library_baseline_unprobeable" || !strings.Contains(func() string {
text, _ := body["error"].(string)
return text
}(), "无法读取目标整库状态") {
t.Fatalf("body=%v", body)
}
if store.createN != 0 {
t.Fatal("unreadable target must not create a baseline job")
}
if len(store.versions) != 0 {
t.Fatal("unreadable target must not record any version")
}
}
// 测试候选只认登记过 test_sealed 的那份快照:归档里同时存在更新的目标基线/
// 预备份快照时,绝不能把它们当成"拟发布的测试版本"。
func TestTestSealedCandidateComesFromRegistryNotNewestArchiveEntry(t *testing.T) {
sealed := librarySnapshotView{SnapshotID: "snap-sealed", CreatedAt: time.Date(2026, 10, 29, 8, 54, 0, 0, time.UTC)}
prebackup := librarySnapshotView{SnapshotID: "snap-prebackup", CreatedAt: time.Date(2026, 10, 29, 9, 30, 0, 0, time.UTC)}
byID := map[string]librarySnapshotView{sealed.SnapshotID: sealed, prebackup.SnapshotID: prebackup}
versions := []governance.LibraryVersion{
{Kind: governance.LibraryVersionProductionPrebackup, SnapshotID: "snap-prebackup"},
{Kind: governance.LibraryVersionTestSealed, SnapshotID: "snap-sealed"},
}
candidate, reason := testSealedOf(versions, byID)
if reason != "" {
t.Fatalf("reason=%q", reason)
}
if candidate == nil || candidate.SnapshotID != "snap-sealed" {
t.Fatalf("candidate=%v, want the registered test seal", candidate)
}
// 没有任何 test_sealed 登记:宁可说"还没有测试封存",也不拿别的快照顶替。
none, reason := testSealedOf([]governance.LibraryVersion{
{Kind: governance.LibraryVersionProductionPrebackup, SnapshotID: "snap-prebackup"},
}, byID)
if none != nil || !strings.Contains(reason, "还没有该目标的测试封存") {
t.Fatalf("candidate=%v reason=%q", none, reason)
}
// 登记过但它已经不在本机归档:说清是归档问题,不给候选。
missing, reason := testSealedOf([]governance.LibraryVersion{
{Kind: governance.LibraryVersionTestSealed, SnapshotID: "snap-gone"},
}, byID)
if missing != nil || !strings.Contains(reason, "不在本机归档里") {
t.Fatalf("candidate=%v reason=%q", missing, reason)
}
}
// 非空但无权威版本时,缺项必须指向真实入口,而不是一句"未知"。
func TestLibrarySyncGateNamesTheBaselineEntryForNonEmptyTarget(t *testing.T) {
gate := librarySyncGateWithGrant(Config{ProductionReleaseRequestsEnabled: true,
ProductionBackupRestoreVerified: true, LibraryProductionDatabaseURL: "postgres://prod.invalid/db",
LibraryProductionInternalKB: "kb-int", LibraryCapacityBudgetBytes: 1 << 30},
libraryKnownView{State: "unknown"}, libraryKnownView{State: "unknown"}, "snap-1", true)
if gate.SyncAllowed {
t.Fatal("unknown production version must block the sync")
}
joined := strings.Join(gate.Reasons, "|")
if !strings.Contains(joined, "目标非空且没有生效版本记录") || !strings.Contains(joined, "目标权威基线") {
t.Fatalf("reasons=%v", gate.Reasons)
}
}
// 维护能力是配置事实:配置齐全时必须说"已接线",缺项要列出缺哪个变量。
func TestLibraryMaintenanceCapabilityReportsWiringInsteadOfOneVagueSentence(t *testing.T) {
wired := libraryMaintenanceCapability(Config{
LibraryTargetInstance: "kgseal-app", LibraryTargetProject: "kgseal-weknora",
LibraryTargetWriters: []string{"app"}, LibraryTargetIngressURL: "http://kgseal-ingress:18084/health",
LibraryTargetIngressService: "ingress", LibraryTargetJournalRoot: "/var/lib/knowledge-governance/journal",
LibraryTargetDockerSocket: "/var/run/docker.sock",
})
if wired["configured"] != true {
t.Fatalf("capability=%v", wired)
}
if wired["instance"] != "kgseal-app" {
t.Fatalf("instance=%v", wired["instance"])
}
broken := libraryMaintenanceCapability(Config{LibraryTargetInstance: "kgseal-app"})
if broken["configured"] != false {
t.Fatalf("capability=%v", broken)
}
missing, _ := broken["missing"].([]string)
if len(missing) == 0 || !strings.Contains(strings.Join(missing, ","), "LIBRARY_TARGET_WRITERS") {
t.Fatalf("missing=%v", broken["missing"])
}
}
+11 -7
View File
@@ -26,13 +26,6 @@ type libraryCaptureRequest struct {
} }
func libraryCaptureEnqueue(w http.ResponseWriter, r *http.Request, config Config) { func libraryCaptureEnqueue(w http.ResponseWriter, r *http.Request, config Config) {
if config.LibraryAgentSocket == "" {
writeJSON(w, http.StatusServiceUnavailable, map[string]any{
"code": "library_capture_unwired",
"reason": "未接线:主机侧封存 agent 没有挂载,页面无法发起封存",
})
return
}
var request libraryCaptureRequest var request libraryCaptureRequest
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&request); err != nil { if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&request); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid request body"}) writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid request body"})
@@ -52,6 +45,17 @@ func libraryCaptureEnqueue(w http.ResponseWriter, r *http.Request, config Config
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid job id"}) writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid job id"})
return return
} }
// 请求形状先校验(400),再看执行器(503):不只看"配了 socket"——执行器可能
// 不可达、可能指向别的实例、可能降级。页面与接口必须用同一份判定,否则又会出现
// "点了才失败"或"封错库"。
if available, reason, executor := libraryExecutorAvailability(r.Context(), config); !available {
writeJSON(w, http.StatusServiceUnavailable, map[string]any{
"code": "library_capture_unavailable",
"reason": reason,
"executor": executor,
})
return
}
requestedBy := "" requestedBy := ""
if identity, ok := sessionFrom(r.Context()); ok { if identity, ok := sessionFrom(r.Context()); ok {
requestedBy = identity.Username requestedBy = identity.Username
+5 -26
View File
@@ -17,31 +17,11 @@ func captureRequest(body string) *http.Request {
return request return request
} }
// An unwired agent must refuse clearly instead of queueing a job nobody can run.
func TestLibraryCaptureRefusedWhenAgentUnwired(t *testing.T) {
store := &fakeJobStore{}
recorder := httptest.NewRecorder()
libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"internal"}`), Config{Store: store})
if recorder.Code != http.StatusServiceUnavailable {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
}
var body map[string]any
if err := json.Unmarshal(recorder.Body.Bytes(), &body); err != nil {
t.Fatal(err)
}
if body["code"] != "library_capture_unwired" || body["reason"] == "" {
t.Fatalf("unexpected body %v", body)
}
if store.enqueued.Type != "" {
t.Fatal("unwired agent still enqueued a job")
}
}
func TestLibraryCaptureEnqueuesFixedJobShape(t *testing.T) { func TestLibraryCaptureEnqueuesFixedJobShape(t *testing.T) {
store := &fakeJobStore{enqueueEchoRequestedID: true} store := &fakeJobStore{enqueueEchoRequestedID: true}
recorder := httptest.NewRecorder() recorder := httptest.NewRecorder()
libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"external","job":"lib-external-review"}`), libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"external","job":"lib-external-review"}`),
Config{Store: store, LibraryAgentSocket: "/run/library-agent/agent.sock"}) captureReadyConfig(store))
if recorder.Code != http.StatusAccepted { if recorder.Code != http.StatusAccepted {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String()) t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
} }
@@ -76,7 +56,7 @@ func TestLibraryCaptureRejectsInvalidRequest(t *testing.T) {
} { } {
store := &fakeJobStore{} store := &fakeJobStore{}
recorder := httptest.NewRecorder() recorder := httptest.NewRecorder()
libraryCaptureEnqueue(recorder, captureRequest(body), Config{Store: store, LibraryAgentSocket: "/tmp/agent.sock"}) libraryCaptureEnqueue(recorder, captureRequest(body), captureReadyConfig(store))
if recorder.Code != http.StatusBadRequest { if recorder.Code != http.StatusBadRequest {
t.Fatalf("body %q status=%d", body, recorder.Code) t.Fatalf("body %q status=%d", body, recorder.Code)
} }
@@ -92,8 +72,7 @@ func TestLibraryCaptureQueueIDFitsTheColumn(t *testing.T) {
for _, scope := range []string{"internal", "external"} { for _, scope := range []string{"internal", "external"} {
store := &fakeJobStore{enqueueEchoRequestedID: true} store := &fakeJobStore{enqueueEchoRequestedID: true}
recorder := httptest.NewRecorder() recorder := httptest.NewRecorder()
libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"`+scope+`"}`), libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"`+scope+`"}`), captureReadyConfig(store))
Config{Store: store, LibraryAgentSocket: "/tmp/agent.sock"})
if recorder.Code != http.StatusAccepted { if recorder.Code != http.StatusAccepted {
t.Fatalf("%s status=%d body=%s", scope, recorder.Code, recorder.Body.String()) t.Fatalf("%s status=%d body=%s", scope, recorder.Code, recorder.Body.String())
} }
@@ -146,7 +125,7 @@ func TestLibraryCaptureRejectsScopeConflictForReusedJob(t *testing.T) {
Payload: json.RawMessage(`{"scope":"internal","capture_job":"lib-shared","requested_by":"yaozhanghao"}`)}} Payload: json.RawMessage(`{"scope":"internal","capture_job":"lib-shared","requested_by":"yaozhanghao"}`)}}
recorder := httptest.NewRecorder() recorder := httptest.NewRecorder()
libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"external","job":"lib-shared"}`), libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"external","job":"lib-shared"}`),
Config{Store: store, LibraryAgentSocket: "/run/library-agent/agent.sock"}) captureReadyConfig(store))
if recorder.Code != http.StatusConflict { if recorder.Code != http.StatusConflict {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String()) t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
} }
@@ -166,7 +145,7 @@ func TestLibraryCaptureReplayReportsStoredJob(t *testing.T) {
Payload: json.RawMessage(`{"scope":"internal","capture_job":"lib-shared","requested_by":"yaozhanghao"}`)}} Payload: json.RawMessage(`{"scope":"internal","capture_job":"lib-shared","requested_by":"yaozhanghao"}`)}}
recorder := httptest.NewRecorder() recorder := httptest.NewRecorder()
libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"internal","job":"lib-shared"}`), libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"internal","job":"lib-shared"}`),
Config{Store: store, LibraryAgentSocket: "/run/library-agent/agent.sock"}) captureReadyConfig(store))
if recorder.Code != http.StatusAccepted { if recorder.Code != http.StatusAccepted {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String()) t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
} }
+77
View File
@@ -0,0 +1,77 @@
package httpapi
import (
"context"
"time"
"knowledge-governance/internal/libraryexecutor"
)
// libraryExecutorView 是"封存执行器到底能不能用"的页面口径。
//
// 为什么不是 `LIBRARY_AGENT_SOCKET != ""`:正式栈里 API 根本没有挂载那个 socket,
// 配置非空并不代表链路通;更危险的是 agent 可能顶着正式单元名却在冻结另一个实例。
// 这里只认 worker 写入的真实观测,并把观测时间与执行器自报的身份一起给出来。
type libraryExecutorView struct {
Available bool `json:"available"`
Reason string `json:"reason,omitempty"`
Instance string `json:"instance,omitempty"`
Project string `json:"project,omitempty"`
SourceApp string `json:"source_app,omitempty"`
ArchiveRoot string `json:"archive_root,omitempty"`
Busy bool `json:"busy"`
Degraded string `json:"degraded,omitempty"`
ObservedAt *time.Time `json:"observed_at,omitempty"`
}
type libraryExecutorStore interface {
LibraryExecutorStatus(ctx context.Context, instance string) (*libraryexecutor.Observation, error)
}
// libraryExecutorAvailability 把"配置里的期望"与"worker 观测到的真实身份"合起来判定:
// 观测不存在、过期、不可达、实例不符、降级、正忙,任何一条都不可用,并给出原因。
func libraryExecutorAvailability(ctx context.Context, config Config) (bool, string, libraryExecutorView) {
// 期望身份是**封存源**:agent 冻结的是源库写入者,与同步目标(LIBRARY_TARGET_*)
// 本来就是两套名字(本机候选栈:源 kgmaint、目标 kgseal)。
instance := config.LibrarySourceInstance
view := libraryExecutorView{Instance: instance}
if instance == "" {
return false, "未配置封存来源实例身份(LIBRARY_SOURCE_INSTANCE),无法确认执行器归属", view
}
store, ok := config.Store.(libraryExecutorStore)
if !ok {
return false, "执行器状态存储不可用", view
}
observation, err := store.LibraryExecutorStatus(ctx, instance)
if err != nil {
return false, "读取执行器状态失败:" + err.Error(), view
}
if observation == nil {
return false, "封存执行器还没有上报状态(worker 未运行,或实例身份与 worker 不一致)", view
}
available, reason := libraryexecutor.Evaluate(*observation,
libraryexecutor.Expectation{Instance: instance, Project: config.LibrarySourceProject,
ArchiveRoot: config.LibraryArchiveRoot},
executorStatusMaxAge(config), time.Now().UTC())
observedAt := observation.ObservedAt
view.Available = available
view.Reason = reason
view.Project = observation.Identity.Project
view.SourceApp = observation.Identity.SourceApp
view.ArchiveRoot = observation.Identity.ArchiveRoot
view.Busy = observation.Busy
view.Degraded = observation.Degraded
view.ObservedAt = &observedAt
return available, reason, view
}
// defaultLibraryExecutorStatusMaxAge 与 worker 的探测间隔(30s)配套:
// 没配上限绝不等于"永不过期"——那会把一次陈旧的"可用"永久留在页面上。
const defaultLibraryExecutorStatusMaxAge = 2 * time.Minute
func executorStatusMaxAge(config Config) time.Duration {
if config.LibraryExecutorStatusMaxAge > 0 {
return config.LibraryExecutorStatusMaxAge
}
return defaultLibraryExecutorStatusMaxAge
}
+184
View File
@@ -0,0 +1,184 @@
package httpapi
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"knowledge-governance/internal/libraryexecutor"
)
// fakeJobStore 只是"存储"替身;执行器状态同样是它的一条记录。
func (s *fakeJobStore) LibraryExecutorStatus(_ context.Context, instance string) (*libraryexecutor.Observation, error) {
if s.executorStatus == nil || s.executorStatus.Instance != instance {
return nil, nil
}
return s.executorStatus, nil
}
// healthyExecutor 返回一份"worker 刚观测到、指向本实例"的记录。
func healthyExecutor(now time.Time) *libraryexecutor.Observation {
return &libraryexecutor.Observation{
Instance: "kgmaint-weknora", Reachable: true, ObservedAt: now,
Identity: libraryexecutor.Identity{Project: "kgmaint-weknora", SourceApp: "kgmaint-app",
ArchiveRoot: "/tmp/kg-e2e/archive", Writers: "app"},
}
}
// captureReadyConfig 是"执行器真的可用"时的配置:实例、期望项目与观测三者一致。
func captureReadyConfig(store *fakeJobStore) Config {
store.executorStatus = healthyExecutor(time.Now().UTC())
return Config{
Store: store,
LibraryAgentSocket: "/run/library-agent/agent.sock",
LibrarySourceInstance: "kgmaint-weknora",
LibrarySourceProject: "kgmaint-weknora",
LibraryArchiveRoot: "/var/lib/knowledge-governance/library",
}
}
func decodeCaptureRefusal(t *testing.T, recorder *httptest.ResponseRecorder) (string, string) {
t.Helper()
var body struct {
Code string `json:"code"`
Reason string `json:"reason"`
}
if err := json.Unmarshal(recorder.Body.Bytes(), &body); err != nil {
t.Fatalf("body %s: %v", recorder.Body.String(), err)
}
return body.Code, body.Reason
}
// 页面上的"可发起"必须等于接口真的会接受:配置了 socket 但执行器不可达、
// 指向别的实例、降级或过期时,都必须带着原因拒绝,而不是先排队再失败。
func TestLibraryCaptureRefusesWhenExecutorIsNotVerified(t *testing.T) {
now := time.Now().UTC()
tests := []struct {
name string
config func(*Config)
text string
}{
{name: "no instance identity", config: func(c *Config) {
c.LibrarySourceInstance = ""
}, text: "LIBRARY_SOURCE_INSTANCE"},
{name: "nothing reported yet", config: func(c *Config) {
c.Store.(*fakeJobStore).executorStatus = nil
}, text: "还没有上报状态"},
{name: "unreachable", config: func(c *Config) {
c.Store.(*fakeJobStore).executorStatus.Reachable = false
c.Store.(*fakeJobStore).executorStatus.Error = "library agent is unreachable"
}, text: "不可达"},
{name: "stale observation", config: func(c *Config) {
c.Store.(*fakeJobStore).executorStatus.ObservedAt = now.Add(-10 * time.Minute)
}, text: "已过期"},
{name: "points at another instance", config: func(c *Config) {
c.Store.(*fakeJobStore).executorStatus.Identity.Project = "weknora071full"
}, text: "指向的实例不匹配"},
{name: "degraded", config: func(c *Config) {
c.Store.(*fakeJobStore).executorStatus.Degraded = "写入者恢复失败"
}, text: "降级"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
store := &fakeJobStore{}
config := captureReadyConfig(store)
test.config(&config)
recorder := httptest.NewRecorder()
libraryCaptureEnqueue(recorder, captureRequest(`{"scope":"internal"}`), config)
if recorder.Code != http.StatusServiceUnavailable {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
}
code, reason := decodeCaptureRefusal(t, recorder)
if code != "library_capture_unavailable" {
t.Fatalf("code=%q body=%s", code, recorder.Body.String())
}
if reason == "" || !strings.Contains(reason, test.text) {
t.Fatalf("reason=%q want it to mention %q", reason, test.text)
}
if store.enqueued.Type != "" {
t.Fatal("refused capture still enqueued a job")
}
})
}
}
// The overview must report the same reason the capture endpoint would use.
func TestLibraryOverviewReportsExecutorIdentityAndReason(t *testing.T) {
store := &fakeLibraryStore{}
config := captureReadyConfig(&store.fakeJobStore)
// libraryJobStore 的方法在 fakeLibraryStore 上,执行器状态在嵌入的 fakeJobStore 上。
config.Store = store
recorder := httptest.NewRecorder()
libraryOverviewHandler(recorder, httptest.NewRequest(http.MethodGet, "/api/v1/library/overview?scope=internal", nil), config)
if recorder.Code != http.StatusOK {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
}
var body struct {
CaptureAvailable bool `json:"capture_available"`
Executor struct {
Available bool `json:"available"`
Instance string `json:"instance"`
Project string `json:"project"`
SourceApp string `json:"source_app"`
ArchiveRoot string `json:"archive_root"`
} `json:"executor"`
}
if err := json.Unmarshal(recorder.Body.Bytes(), &body); err != nil {
t.Fatal(err)
}
if !body.CaptureAvailable || !body.Executor.Available {
t.Fatalf("overview did not report an available executor: %s", recorder.Body.String())
}
if body.Executor.Project != "kgmaint-weknora" || body.Executor.SourceApp != "kgmaint-app" ||
body.Executor.ArchiveRoot != "/tmp/kg-e2e/archive" || body.Executor.Instance != "kgmaint-weknora" {
t.Fatalf("executor view = %+v", body.Executor)
}
store.fakeJobStore.executorStatus.Identity.Project = "weknora071full"
recorder = httptest.NewRecorder()
libraryOverviewHandler(recorder, httptest.NewRequest(http.MethodGet, "/api/v1/library/overview?scope=internal", nil), config)
if err := json.Unmarshal(recorder.Body.Bytes(), &body); err != nil {
t.Fatal(err)
}
if body.CaptureAvailable {
t.Fatalf("overview still says available after the executor moved to another instance: %s", recorder.Body.String())
}
}
// "恢复至该版本"必须由真实条件决定:没有 restore 授权就明确说出来。
func TestLibraryRestoreAvailabilityExplainsMissingAuthorization(t *testing.T) {
store := &fakeJobStore{}
config := Config{Store: store, ProductionReleaseRequestsEnabled: true, ProductionBackupRestoreVerified: true}
recorder := httptest.NewRecorder()
libraryRestoreAvailability(recorder, httptest.NewRequest(http.MethodGet,
"/api/v1/library/restore-availability?scope=internal&snapshot_id=snap-1", nil), config)
if recorder.Code != http.StatusOK {
t.Fatalf("status=%d body=%s", recorder.Code, recorder.Body.String())
}
var body struct {
Allowed bool `json:"allowed"`
Reasons []string `json:"reasons"`
Authorization struct {
State string `json:"state"`
} `json:"authorization"`
}
if err := json.Unmarshal(recorder.Body.Bytes(), &body); err != nil {
t.Fatal(err)
}
if body.Allowed || body.Authorization.State != "none" || len(body.Reasons) == 0 {
t.Fatalf("body=%s", recorder.Body.String())
}
found := false
for _, reason := range body.Reasons {
if strings.Contains(reason, "恢复授权") {
found = true
}
}
if !found {
t.Fatalf("reasons = %v", body.Reasons)
}
}
+103 -29
View File
@@ -19,6 +19,7 @@ import (
"knowledge-governance/internal/librarydiff" "knowledge-governance/internal/librarydiff"
"knowledge-governance/internal/librarysnapshot" "knowledge-governance/internal/librarysnapshot"
postgresstore "knowledge-governance/internal/platform/postgres" postgresstore "knowledge-governance/internal/platform/postgres"
v071 "knowledge-governance/internal/releaseadapter/v071"
) )
// 整库版本/状态/差异读取契约 + 生产同步的 fail-closed 入口。 // 整库版本/状态/差异读取契约 + 生产同步的 fail-closed 入口。
@@ -56,30 +57,36 @@ type libraryKnownView struct {
EmptyBaseline bool `json:"empty_baseline,omitempty"` EmptyBaseline bool `json:"empty_baseline,omitempty"`
} }
// productionScopeEmpty 只读探测生产目标:真正连库数一遍,不用缓存或推断。 // productionScopeState 只读探测生产目标:真正连库数一遍,不用缓存或推断。
func productionScopeEmpty(ctx context.Context, config Config, scope string) (bool, error) { //
// "空"必须是整库意义的空:只数 knowledges 会把"文档已删、分块/Wiki/资源还在"
// 的目标当成首次发布的空基线,从而绕过基线初始化。所以这里用适配层同一份
// 表清单与范围过滤(v071.ScopeContentEmpty),并把逐表行数交回调用方作为证据。
func productionScopeState(ctx context.Context, config Config, scope string) (bool, map[string]int64, error) {
if strings.TrimSpace(config.LibraryProductionDatabaseURL) == "" { if strings.TrimSpace(config.LibraryProductionDatabaseURL) == "" {
return false, errors.New("生产整库目标未配置") return false, nil, errors.New("生产整库目标未配置")
} }
kb := config.LibraryProductionInternalKB kb := config.LibraryProductionInternalKB
if scope == "external" { if scope == "external" {
kb = config.LibraryProductionExternalKB kb = config.LibraryProductionExternalKB
} }
if kb == "" { if kb == "" {
return false, errors.New("生产知识库未配置") return false, nil, errors.New("该范围的目标知识库未配置")
} }
pool, err := pgxpool.New(ctx, config.LibraryProductionDatabaseURL) pool, err := pgxpool.New(ctx, config.LibraryProductionDatabaseURL)
if err != nil { if err != nil {
return false, err return false, nil, err
} }
defer pool.Close() defer pool.Close()
var count int64 return v071.ScopeContentEmpty(ctx, pool,
err = pool.QueryRow(ctx, `SELECT count(*) FROM knowledges WHERE tenant_id=$1 AND knowledge_base_id=$2 AND deleted_at IS NULL`, v071.ProductionScope{TenantID: config.LibraryProductionTenantID, KnowledgeBaseID: kb})
config.LibraryProductionTenantID, kb).Scan(&count) }
if err != nil {
return false, err // productionScopeEmpty 保留"只问空不空"的调用形状;逐表行数在需要举证时用
} // productionScopeState 取。
return count == 0, nil func productionScopeEmpty(ctx context.Context, config Config, scope string) (bool, error) {
empty, _, err := productionScopeState(ctx, config, scope)
return empty, err
} }
type libraryDiffView struct { type libraryDiffView struct {
@@ -307,6 +314,7 @@ type libraryOverview struct {
Authorization libraryAuthorizationView `json:"authorization"` Authorization libraryAuthorizationView `json:"authorization"`
CaptureAvailable bool `json:"capture_available"` CaptureAvailable bool `json:"capture_available"`
CaptureReason string `json:"capture_reason,omitempty"` CaptureReason string `json:"capture_reason,omitempty"`
Executor libraryExecutorView `json:"executor"`
Now time.Time `json:"now"` Now time.Time `json:"now"`
} }
@@ -507,6 +515,70 @@ func errString(err error) string {
return err.Error() return err.Error()
} }
// shortSnapshot 只用于给人看的提示:完整摘要留给快照详情,别在原因句里塞 64 位。
func shortSnapshot(id string) string {
if len(id) <= 12 {
return id
}
return id[:12]
}
// testSealedOf 选"拟发布的测试封存":只有登记过 test_sealed 的那一份才算候选。
//
// 归档是共享的:生产预备份与目标基线快照同样带着 scope 写进同一个归档根,
// 按"这个 scope 最新一份"挑会把目标自己的内容当成测试候选(页面会显示一份
// 永远不该被发布的东西)。所以候选来自版本登记,再用归档确认它真的还在本地。
func testSealedOf(versions []governance.LibraryVersion, byID map[string]librarySnapshotView) (*librarySnapshotView, string) {
for _, item := range versions {
if item.Kind != governance.LibraryVersionTestSealed {
continue
}
snapshot, ok := byID[item.SnapshotID]
if !ok {
return nil, "最近一次测试封存 " + shortSnapshot(item.SnapshotID) + " 不在本机归档里(归档被清理或换了归档根)"
}
return &snapshot, ""
}
return nil, "还没有该目标的测试封存"
}
// libraryMaintenanceCapability 用配置事实回答"这台机器上维护是否接线"。
// 它刻意只说配置:能停写入者/关入口/证明排空的真实能力在 worker 侧(docker socket、
// 闸门、排空文件),API 若声称"能力可用"就是拿配置字符串冒充执行证据。
func libraryMaintenanceCapability(config Config) map[string]any {
missing := make([]string, 0, 6)
if config.LibraryTargetInstance == "" {
missing = append(missing, "LIBRARY_TARGET_INSTANCE")
}
if config.LibraryTargetProject == "" {
missing = append(missing, "LIBRARY_TARGET_PROJECT")
}
if len(config.LibraryTargetWriters) == 0 {
missing = append(missing, "LIBRARY_TARGET_WRITERS")
}
if config.LibraryTargetIngressURL == "" {
missing = append(missing, "LIBRARY_TARGET_INGRESS_URL")
}
if config.LibraryTargetIngressService == "" {
missing = append(missing, "LIBRARY_TARGET_INGRESS_SERVICE")
}
if config.LibraryTargetJournalRoot == "" {
missing = append(missing, "LIBRARY_TARGET_JOURNAL_ROOT")
}
if config.LibraryTargetDockerSocket == "" {
missing = append(missing, "LIBRARY_TARGET_DOCKER_SOCKET")
}
return map[string]any{
"configured": len(missing) == 0,
"missing": missing,
"instance": config.LibraryTargetInstance,
"project": config.LibraryTargetProject,
"writers": config.LibraryTargetWriters,
"ingress_service": config.LibraryTargetIngressService,
"note": "配置齐全只说明治理侧接线完整;停写入者、关入口与排空由 worker 在作业内校验并执行",
}
}
func librarySyncGate(config Config, production libraryKnownView, prebackup libraryKnownView, target string) libraryGateView { func librarySyncGate(config Config, production libraryKnownView, prebackup libraryKnownView, target string) libraryGateView {
return librarySyncGateWithGrant(config, production, prebackup, target, false) return librarySyncGateWithGrant(config, production, prebackup, target, false)
} }
@@ -530,9 +602,11 @@ func librarySyncGateWithGrant(config Config, production libraryKnownView, prebac
gate.Reasons = append(gate.Reasons, "生产整库目标未配置(LIBRARY_PRODUCTION_DATABASE_URL),无法核验生产当前状态") gate.Reasons = append(gate.Reasons, "生产整库目标未配置(LIBRARY_PRODUCTION_DATABASE_URL),无法核验生产当前状态")
} }
// 首次发布:生产还没有"生效版本"记录,但只要生产目标可读且是空的, // 首次发布:生产还没有"生效版本"记录,但只要生产目标可读且是空的,
// 这就是一个合法的"空基线",不应该被永久挡住。非空但无权威版本仍然阻断。 // 这就是一个合法的"空基线",不应该被永久挡住。非空但无权威版本仍然阻断,
// 指向真实可行的入口(在维护隔离下捕获目标整库并校验后登记基线),
// 而不是让人去手写版本或清空目标。
if production.State != "known" && !production.EmptyBaseline { if production.State != "known" && !production.EmptyBaseline {
gate.Reasons = append(gate.Reasons, "生产当前权威版本未知,不能启动写入") gate.Reasons = append(gate.Reasons, "目标非空且没有生效版本记录:先建立目标权威基线(维护隔离下捕获整库并逐类校验),或确认目标确实是空库")
} }
if target == "" { if target == "" {
gate.Reasons = append(gate.Reasons, "还没有可同步的测试封存版本") gate.Reasons = append(gate.Reasons, "还没有可同步的测试封存版本")
@@ -563,28 +637,25 @@ func libraryOverviewHandler(w http.ResponseWriter, r *http.Request, config Confi
} }
now := time.Now().UTC() now := time.Now().UTC()
overview := libraryOverview{Scope: scope, Jobs: []governance.LibraryJob{}, Now: now, overview := libraryOverview{Scope: scope, Jobs: []governance.LibraryJob{}, Now: now,
CaptureAvailable: strings.TrimSpace(config.LibraryAgentSocket) != "", Gate: libraryGateView{Reasons: []string{}},
Gate: libraryGateView{Reasons: []string{}}, Diff: libraryDiffView{State: "unknown", Reason: "生产与测试的文档级差异读取未接线", Added: []any{}, Updated: []any{}, Removed: []any{}}}
Diff: libraryDiffView{State: "unknown", Reason: "生产与测试的文档级差异读取未接线", Added: []any{}, Updated: []any{}, Removed: []any{}}} // 可发起封存 = worker 真实探测到执行器可用且指向本实例;原因逐条给全。
if !overview.CaptureAvailable { overview.CaptureAvailable, overview.CaptureReason, overview.Executor = libraryExecutorAvailability(r.Context(), config)
overview.CaptureReason = "主机侧封存 agent 未接线"
}
// 测试封存来自归档:它只是测试侧事实。 // 测试封存来自归档 + 版本登记:归档里**不止**测试封存——生产预备份与目标基线
// 快照同样带 scope,如果只按"这个 scope 最新一份"挑,页面就会把目标自己的内容
// 当成拟发布的测试候选。所以候选只认登记过 kind=test_sealed 的那一份。
snapshots, snapErr := readLibrarySnapshots(r.Context(), config.LibraryArchiveRoot) snapshots, snapErr := readLibrarySnapshots(r.Context(), config.LibraryArchiveRoot)
byID := map[string]librarySnapshotView{}
archiveReadable := false
if snapErr != nil { if snapErr != nil {
_, overview.TestSealedReason = libraryArchiveErrorStatus(snapErr, "归档不可读") _, overview.TestSealedReason = libraryArchiveErrorStatus(snapErr, "归档不可读")
} else if !snapshots.Available { } else if !snapshots.Available {
overview.TestSealedReason = snapshots.Reason overview.TestSealedReason = snapshots.Reason
} else { } else {
for index := range snapshots.Snapshots { archiveReadable = true
if snapshots.Snapshots[index].Scope == scope { for _, snapshot := range snapshots.Snapshots {
overview.TestSealed = &snapshots.Snapshots[index] byID[snapshot.SnapshotID] = snapshot
break
}
}
if overview.TestSealed == nil {
overview.TestSealedReason = "还没有该目标的测试封存"
} }
} }
@@ -594,6 +665,9 @@ func libraryOverviewHandler(w http.ResponseWriter, r *http.Request, config Confi
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "cannot read library versions"}) writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "cannot read library versions"})
return return
} }
if archiveReadable {
overview.TestSealed, overview.TestSealedReason = testSealedOf(versions, byID)
}
overview.Production = libraryKnownView{State: "unknown", Reason: "还没有生产验证通过的生效版本记录"} overview.Production = libraryKnownView{State: "unknown", Reason: "还没有生产验证通过的生效版本记录"}
overview.Prebackup = libraryKnownView{State: "unknown", Reason: "本次目标快照还没有生产预备份"} overview.Prebackup = libraryKnownView{State: "unknown", Reason: "本次目标快照还没有生产预备份"}
target := "" target := ""
+58
View File
@@ -146,6 +146,64 @@ func libraryRestoreEnqueue(w http.ResponseWriter, r *http.Request, config Config
writeJSON(w, http.StatusAccepted, map[string]any{"job": job, "reused": false}) writeJSON(w, http.StatusAccepted, map[string]any{"job": job, "reused": false})
} }
// libraryRestoreAvailability 回答"现在能不能把生产恢复到这份快照"。
//
// 为什么需要它:页面原来把"恢复至该版本"按钮写死为 disabled,理由是"未接线"——
// 但接口、授权、恢复作业其实都已经在跑。按钮该由真实条件决定:双闸打开、
// 有这份快照的恢复授权、目标实例没有在维护。
type libraryRestoreAvailabilityView struct {
Allowed bool `json:"allowed"`
Reasons []string `json:"reasons"`
Scope string `json:"scope"`
SnapshotID string `json:"snapshot_id"`
Authorization libraryAuthorizationView `json:"authorization"`
}
func libraryRestoreAvailability(w http.ResponseWriter, r *http.Request, config Config) {
scope, err := scopeFromQuery(r)
if err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": err.Error()})
return
}
snapshotID := strings.TrimSpace(r.URL.Query().Get("snapshot_id"))
if snapshotID == "" {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "snapshot_id required"})
return
}
view := libraryRestoreAvailabilityView{Scope: scope, SnapshotID: snapshotID, Reasons: []string{}}
if !config.ProductionReleaseRequestsEnabled {
view.Reasons = append(view.Reasons, "生产写入闸口关闭(PRODUCTION_RELEASE_REQUESTS_ENABLED=false)")
}
if !config.ProductionBackupRestoreVerified {
view.Reasons = append(view.Reasons, "没有已核验的生产恢复演练证据(PRODUCTION_BACKUP_RESTORE_VERIFIED=false)")
}
digest, err := libraryRestoreParamsDigest(scope, snapshotID)
if err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "cannot digest params"})
return
}
grant, found := governance.LibraryReleaseAuthorization{}, false
if storeWithAuth, ok := config.Store.(interface {
ActiveLibraryAuthorization(ctx context.Context, scope, snapshotID, digest string, now time.Time) (governance.LibraryReleaseAuthorization, bool, error)
}); ok {
grant, found, err = storeWithAuth.ActiveLibraryAuthorization(r.Context(), scope, snapshotID, digest, time.Now().UTC())
if err != nil {
writeJSON(w, http.StatusServiceUnavailable, map[string]any{"allowed": false,
"reasons": []string{"读取恢复授权失败:" + err.Error()}})
return
}
}
view.Authorization = authorizationViewOf(grant, found)
if !found {
view.Reasons = append(view.Reasons, "没有针对该快照的恢复授权:先签发 restore 授权(绑定同一快照)")
}
if _, held := instanceMaintenanceGuard(r.Context(), config, scope); held {
view.Reasons = append(view.Reasons, "目标实例正在维护中,等当前作业释放后再恢复")
}
view.Allowed = len(view.Reasons) == 0
writeJSON(w, http.StatusOK, view)
}
// 崩溃续作:失败作业可以按原参数重新排队,生成新作业(旧作业保留为审计记录)。 // 崩溃续作:失败作业可以按原参数重新排队,生成新作业(旧作业保留为审计记录)。
func libraryJobResume(w http.ResponseWriter, r *http.Request, config Config) { func libraryJobResume(w http.ResponseWriter, r *http.Request, config Config) {
identity, ok := sessionFrom(r.Context()) identity, ok := sessionFrom(r.Context())
+4 -5
View File
@@ -85,15 +85,14 @@ const libraryCaptureNote = "封存由运维命令 scripts/library-capture.sh cap
func librarySnapshotsList(w http.ResponseWriter, r *http.Request, config Config) { func librarySnapshotsList(w http.ResponseWriter, r *http.Request, config Config) {
view, err := readLibrarySnapshots(r.Context(), config.LibraryArchiveRoot) view, err := readLibrarySnapshots(r.Context(), config.LibraryArchiveRoot)
view.CaptureAvailable = config.LibraryAgentSocket != "" // 能不能发起封存只认 worker 的真实观测,不认"配置里写了 socket 路径"。
if !view.CaptureAvailable { view.CaptureAvailable, view.CaptureReason, _ = libraryExecutorAvailability(r.Context(), config)
view.CaptureReason = "未接线:主机侧封存 agent 没有挂载,页面只能查看已封存版本"
}
if err != nil { if err != nil {
status, reason := libraryArchiveErrorStatus(err, "归档不可读") status, reason := libraryArchiveErrorStatus(err, "归档不可读")
writeJSON(w, status, librarySnapshotList{Available: false, Reason: reason, writeJSON(w, status, librarySnapshotList{Available: false, Reason: reason,
Snapshots: []librarySnapshotView{}, Verification: libraryVerificationManifest, Snapshots: []librarySnapshotView{}, Verification: libraryVerificationManifest,
VerificationNote: libraryVerificationNote, CaptureNote: libraryCaptureNote}) VerificationNote: libraryVerificationNote, CaptureNote: libraryCaptureNote,
CaptureAvailable: view.CaptureAvailable, CaptureReason: view.CaptureReason})
return return
} }
writeJSON(w, http.StatusOK, view) writeJSON(w, http.StatusOK, view)
+27
View File
@@ -148,10 +148,24 @@ type Config struct {
// LibraryTargetInstance 是实例级维护的目标实例名;留空即"未接线", // LibraryTargetInstance 是实例级维护的目标实例名;留空即"未接线",
// 页面不能发起按预备份恢复,接口返回原因而不是假装能恢复。 // 页面不能发起按预备份恢复,接口返回原因而不是假装能恢复。
LibraryTargetInstance string LibraryTargetInstance string
// 维护能力的其余配置项只用于如实展示"是否接线":未运行维护与没有维护能力
// 是两件事,页面不能混成一句"未接入"。真正停写入者/关入口的能力由 worker
// 在执行作业时校验(socket、入口、排空文件都在 worker 侧)。
LibraryTargetProject string
LibraryTargetWriters []string
LibraryTargetIngressURL string
LibraryTargetIngressService string
LibraryTargetDockerSocket string
LibraryTargetJournalRoot string
// LibrarySourceInstance 是死信解除记录的来源实例名(封存源),留空回落到目标实例名。 // LibrarySourceInstance 是死信解除记录的来源实例名(封存源),留空回落到目标实例名。
LibrarySourceInstance string LibrarySourceInstance string
// LibraryAgentSocket 是主机侧封存 agent 的 unix socket;留空即页面不能发起封存。 // LibraryAgentSocket 是主机侧封存 agent 的 unix socket;留空即页面不能发起封存。
LibraryAgentSocket string LibraryAgentSocket string
// LibrarySourceProject 是执行器应指向的实例名(agent -project,封存源)。
// 与它不一致时按"封错库"处理:不可发起,并说明差在哪。
LibrarySourceProject string
// LibraryExecutorStatusMaxAge 是执行器观测的有效期;超过即"无法确认"。
LibraryExecutorStatusMaxAge time.Duration
// LibraryCapacityBudgetBytes 是整库归档容量预算;0 = 未设置(容量门禁不放行)。 // LibraryCapacityBudgetBytes 是整库归档容量预算;0 = 未设置(容量门禁不放行)。
LibraryCapacityBudgetBytes int64 LibraryCapacityBudgetBytes int64
// 整库生产目标的只读探测配置:首次发布要能证明"生产可读且为空"。 // 整库生产目标的只读探测配置:首次发布要能证明"生产可读且为空"。
@@ -159,6 +173,9 @@ type Config struct {
LibraryProductionInternalKB string LibraryProductionInternalKB string
LibraryProductionExternalKB string LibraryProductionExternalKB string
LibraryProductionTenantID uint64 LibraryProductionTenantID uint64
// LibraryScopeProbe 让测试注入"目标整库是否为空"的探测;留空即用真实连接。
// 它不是运行时开关:生产部署不设置它,那次探测仍然直连目标库逐表计数。
LibraryScopeProbe func(ctx context.Context, scope string) (bool, map[string]int64, error)
// LibraryRetentionVersions/Days 是保留策略;都为 0 = 未设置(不生成清理候选)。 // LibraryRetentionVersions/Days 是保留策略;都为 0 = 未设置(不生成清理候选)。
LibraryRetentionVersions int LibraryRetentionVersions int
LibraryRetentionDays int LibraryRetentionDays int
@@ -474,6 +491,8 @@ func NewHandler(config Config) (http.Handler, error) {
return return
} }
response := map[string]any{"held": held != nil && held.ReleasedAt == nil, "maintenance": held} response := map[string]any{"held": held != nil && held.ReleasedAt == nil, "maintenance": held}
// 有没有维护能力是配置事实,和"此刻没有窗口"必须分开回答。
response["capability"] = libraryMaintenanceCapability(config)
// 实例级占用才是真正的互斥来源:跨 scope 也在这里体现。 // 实例级占用才是真正的互斥来源:跨 scope 也在这里体现。
if instanceStore, ok := config.Store.(interface { if instanceStore, ok := config.Store.(interface {
InstanceMaintenanceOf(ctx context.Context, instance string) (*postgresstore.LibraryInstanceMaintenance, error) InstanceMaintenanceOf(ctx context.Context, instance string) (*postgresstore.LibraryInstanceMaintenance, error)
@@ -510,6 +529,10 @@ func NewHandler(config Config) (http.Handler, error) {
mux.HandleFunc("POST /api/v1/library/restores", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("POST /api/v1/library/restores", func(w http.ResponseWriter, r *http.Request) {
libraryRestoreEnqueue(w, r, config) libraryRestoreEnqueue(w, r, config)
}) })
// 页面问"这份快照现在能不能恢复":条件来自双闸、授权与实例维护,而不是写死的按钮。
mux.HandleFunc("GET /api/v1/library/restore-availability", func(w http.ResponseWriter, r *http.Request) {
libraryRestoreAvailability(w, r, config)
})
// 失败/取消作业续作:按原参数重新排队,旧作业保留审计。 // 失败/取消作业续作:按原参数重新排队,旧作业保留审计。
mux.HandleFunc("POST /api/v1/library/jobs/{id}/resume", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("POST /api/v1/library/jobs/{id}/resume", func(w http.ResponseWriter, r *http.Request) {
libraryJobResume(w, r, config) libraryJobResume(w, r, config)
@@ -518,6 +541,10 @@ func NewHandler(config Config) (http.Handler, error) {
mux.HandleFunc("POST /api/v1/library/syncs", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("POST /api/v1/library/syncs", func(w http.ResponseWriter, r *http.Request) {
librarySyncEnqueue(w, r, config) librarySyncEnqueue(w, r, config)
}) })
// 首次发布的目标权威基线:管理员在维护隔离下捕获目标整库并校验后登记版本。
mux.HandleFunc("POST /api/v1/library/baselines", func(w http.ResponseWriter, r *http.Request) {
libraryBaselineEnqueue(w, r, config)
})
// 全部 main 提交时间线(读本库缓存,不直接打 Gitea) // 全部 main 提交时间线(读本库缓存,不直接打 Gitea)
mux.HandleFunc("GET /api/v1/version-history", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("GET /api/v1/version-history", func(w http.ResponseWriter, r *http.Request) {
versionHistoryList(w, r, config) versionHistoryList(w, r, config)
+27 -24
View File
@@ -15,6 +15,7 @@ import (
"knowledge-governance/internal/governance" "knowledge-governance/internal/governance"
"knowledge-governance/internal/jobs" "knowledge-governance/internal/jobs"
"knowledge-governance/internal/libraryexecutor"
postgresstore "knowledge-governance/internal/platform/postgres" postgresstore "knowledge-governance/internal/platform/postgres"
) )
@@ -1501,30 +1502,32 @@ type fakeJobStore struct {
activateSettingsErr error activateSettingsErr error
// enqueueEchoRequestedID:幂等"首次入队"语义 —— 返回调用方请求的那个 id // enqueueEchoRequestedID:幂等"首次入队"语义 —— 返回调用方请求的那个 id
// (默认 false 时返回 enqueueResult,即"复用已有任务"的路径)。 // (默认 false 时返回 enqueueResult,即"复用已有任务"的路径)。
enqueueEchoRequestedID bool enqueueEchoRequestedID bool
listSettingsConnectionID string listSettingsConnectionID string
listSettingsErr error listSettingsErr error
settingsRevisions []governance.SourceConnectionRevision settingsRevisions []governance.SourceConnectionRevision
jobFailureFilter governance.JobFailureFilter jobFailureFilter governance.JobFailureFilter
jobFailurePage governance.JobFailurePage jobFailurePage governance.JobFailurePage
jobFailureErr error jobFailureErr error
retryBatchRequest *governance.RetryPublicationsBatchRequest retryBatchRequest *governance.RetryPublicationsBatchRequest
retryBatchOperation string retryBatchOperation string
retryBatchResult governance.RetryPublicationsBatchResult retryBatchResult governance.RetryPublicationsBatchResult
cancelBatchRequest *governance.CancelPublicationsBatchRequest cancelBatchRequest *governance.CancelPublicationsBatchRequest
cancelBatchOperation string cancelBatchOperation string
cancelBatchResult governance.RetryPublicationsBatchResult cancelBatchResult governance.RetryPublicationsBatchResult
cancelBatchReplayed bool cancelBatchReplayed bool
cancelBatchErr error cancelBatchErr error
retryBatchReplayed bool retryBatchReplayed bool
retryBatchErr error retryBatchErr error
requeueFailureRequest *governance.RequeueArtifactFetchRequest requeueFailureRequest *governance.RequeueArtifactFetchRequest
requeueFailureOperation string requeueFailureOperation string
requeueFailureResult governance.RequeueArtifactFetchResult requeueFailureResult governance.RequeueArtifactFetchResult
requeueFailureReplayed bool requeueFailureReplayed bool
requeueFailureErr error requeueFailureErr error
enqueued jobs.EnqueueRequest enqueued jobs.EnqueueRequest
enqueueResult *jobs.Job enqueueResult *jobs.Job
// executorStatus 是封存执行器的"最近一次观测";nil 表示还没有任何上报。
executorStatus *libraryexecutor.Observation
listResult []jobs.Job listResult []jobs.Job
listType string listType string
listByTypeResult []jobs.Job listByTypeResult []jobs.Job
+117 -14
View File
@@ -66,6 +66,9 @@ type request struct {
// Released 只允许精确死信指纹(sha256:<64 hex>)。它不改变固定命令形态: // Released 只允许精确死信指纹(sha256:<64 hex>)。它不改变固定命令形态:
// 还能传的只有"这些确切的行已由治理平台审计解除",CLI 会逐字节复核。 // 还能传的只有"这些确切的行已由治理平台审计解除",CLI 会逐字节复核。
Released []string `json:"released,omitempty"` Released []string `json:"released,omitempty"`
// Expect 是调用方对执行器身份的期望。capture 必须在操作互斥内、任何源操作
// 之前比对它;缺失即拒绝(fail closed),不靠"事后核对"兜底。
Expect *Expectation `json:"expect,omitempty"`
} }
type response struct { type response struct {
@@ -89,13 +92,45 @@ const maxReleasedDeadLetters = 64
type Agent struct { type Agent struct {
config Config config Config
mu sync.Mutex mu sync.Mutex
// stateMu 只保护下面两个"给人看"的字段:健康应答不能被一次 15 分钟的封存
// 卡住(否则页面会把"正在封存"读成"执行器不可达")。
stateMu sync.RWMutex
// degraded is non-empty when a failed operation could not be followed by a // degraded is non-empty when a failed operation could not be followed by a
// verified recovery. While set, capture requests are refused: leaving the // verified recovery. While set, capture requests are refused: leaving the
// source writer stopped is worse than refusing work. // source writer stopped is worse than refusing work.
degraded string degraded string
busy bool
wg sync.WaitGroup wg sync.WaitGroup
} }
// captureState 是健康应答要暴露的执行器状态:是否在跑、是否降级。
type captureState struct {
Busy bool `json:"busy"`
Degraded string `json:"degraded"`
}
func (a *Agent) snapshotState() captureState {
a.stateMu.RLock()
defer a.stateMu.RUnlock()
return captureState{Busy: a.busy, Degraded: a.degraded}
}
func (a *Agent) setBusy(busy bool) {
a.stateMu.Lock()
defer a.stateMu.Unlock()
a.busy = busy
}
func (a *Agent) setDegraded(reason string) {
a.stateMu.Lock()
defer a.stateMu.Unlock()
a.degraded = reason
}
func (a *Agent) clearDegraded() {
a.setDegraded("")
}
func New(config Config) (*Agent, error) { func New(config Config) (*Agent, error) {
if !filepath.IsAbs(config.Socket) || !filepath.IsAbs(config.ArchiveRoot) || !filepath.IsAbs(config.JournalRoot) { if !filepath.IsAbs(config.Socket) || !filepath.IsAbs(config.ArchiveRoot) || !filepath.IsAbs(config.JournalRoot) {
return nil, errors.New("library agent paths must be absolute") return nil, errors.New("library agent paths must be absolute")
@@ -143,7 +178,7 @@ func (a *Agent) Serve(ctx context.Context) error {
// writer from a previous killed run. Recover before accepting work; if that // writer from a previous killed run. Recover before accepting work; if that
// fails, stay degraded and refuse captures instead of pretending to be ready. // fails, stay degraded and refuse captures instead of pretending to be ready.
if err := a.recoverWriters(ctx); err != nil { if err := a.recoverWriters(ctx); err != nil {
a.degraded = err.Error() a.setDegraded(err.Error())
log.Printf("library agent: pending freeze recovery failed: %v", err) log.Printf("library agent: pending freeze recovery failed: %v", err)
} }
go func() { go func() {
@@ -194,13 +229,18 @@ func (a *Agent) handle(ctx context.Context, conn net.Conn) {
// execute serialises operations and maps a request onto the one fixed command. // execute serialises operations and maps a request onto the one fixed command.
func (a *Agent) execute(ctx context.Context, req request) response { func (a *Agent) execute(ctx context.Context, req request) response {
// health 不进操作锁、不跑子进程:调用方要的是"这个执行器指向谁、能不能干活",
// 而一次正在进行的封存不该让这个回答变成"不可达"。
if req.Action == "health" {
return a.health()
}
a.mu.Lock() a.mu.Lock()
defer a.mu.Unlock() defer a.mu.Unlock()
out := response{Action: req.Action, Scope: req.Scope, Job: req.Job} out := response{Action: req.Action, Scope: req.Scope, Job: req.Job}
// Degraded blocks new captures, but recover and status must stay reachable: // Degraded blocks new captures, but recover and status must stay reachable:
// otherwise there is no way back to a verified state. // otherwise there is no way back to a verified state.
if a.degraded != "" && req.Action == "capture" { if state := a.snapshotState(); state.Degraded != "" && req.Action == "capture" {
out.Error = "封存执行边界处于降级状态(上次恢复失败:" + a.degraded + "),先执行 recover" out.Error = "封存执行边界处于降级状态(上次恢复失败:" + state.Degraded + "),先执行 recover"
return out return out
} }
var args []string var args []string
@@ -218,6 +258,16 @@ func (a *Agent) execute(ctx context.Context, req request) response {
out.Error = "invalid job id" out.Error = "invalid job id"
return out return out
} }
if req.Expect == nil {
out.Error = "capture 请求没有携带期望执行器身份,拒绝执行"
return out
}
// 此时已持有操作互斥(a.mu):从"比对身份"到"真正跑子进程"之间没有别的
// 请求能插进来,也不会被另一个 agent 抢走——这是零副作用的前提。
if err := a.config.matchExpectation(*req.Expect); err != nil {
out.Error = err.Error()
return out
}
args = []string{"capture", "-scope", req.Scope} args = []string{"capture", "-scope", req.Scope}
if req.Job != "" { if req.Job != "" {
args = append(args, "-job", req.Job) args = append(args, "-job", req.Job)
@@ -242,6 +292,10 @@ func (a *Agent) execute(ctx context.Context, req request) response {
args = append(args, a.config.declaration()...) args = append(args, a.config.declaration()...)
bounded, cancel := context.WithTimeout(ctx, a.config.MaxDuration) bounded, cancel := context.WithTimeout(ctx, a.config.MaxDuration)
defer cancel() defer cancel()
if req.Action == "capture" || req.Action == "recover" {
a.setBusy(true)
defer a.setBusy(false)
}
payload, err := a.config.run(bounded, a.config.Binary, args...) payload, err := a.config.run(bounded, a.config.Binary, args...)
if err != nil { if err != nil {
detail := "operation failed" detail := "operation failed"
@@ -254,7 +308,7 @@ func (a *Agent) execute(ctx context.Context, req request) response {
// verify. // verify.
if req.Action == "capture" || req.Action == "recover" { if req.Action == "capture" || req.Action == "recover" {
if recoveryErr := a.recoverWriters(ctx); recoveryErr != nil { if recoveryErr := a.recoverWriters(ctx); recoveryErr != nil {
a.degraded = recoveryErr.Error() a.setDegraded(recoveryErr.Error())
out.Error = detail + ";写入者恢复失败:" + recoveryErr.Error() + "(已阻断后续封存,需人工处理)" out.Error = detail + ";写入者恢复失败:" + recoveryErr.Error() + "(已阻断后续封存,需人工处理)"
return out return out
} }
@@ -266,7 +320,7 @@ func (a *Agent) execute(ctx context.Context, req request) response {
return out return out
} }
if req.Action == "recover" { if req.Action == "recover" {
a.degraded = "" a.clearDegraded()
} }
out.OK = true out.OK = true
if json.Valid(payload) { if json.Valid(payload) {
@@ -274,9 +328,32 @@ func (a *Agent) execute(ctx context.Context, req request) response {
} else { } else {
out.Payload = json.RawMessage(`{}`) out.Payload = json.RawMessage(`{}`)
} }
// 封存结果必须带上"这次是谁执行的":调用方据此复核,防止心跳与执行之间
// socket 被换成另一个 agent(真正的封错库事故就是这样发生的)。
if req.Action == "capture" && req.Expect != nil {
out.Payload = a.attachIdentity(out.Payload, *req.Expect)
}
return out return out
} }
// attachIdentity 把执行器身份并进封存结果(保留 CLI 原有的字段,如 snapshot_id)。
// expectation_fingerprint 是"本执行器实际核对过的那份期望"的指纹:调用方据此确认
// 执行时看到的期望与它发出的一致,而不是被替换后重新生成的。
func (a *Agent) attachIdentity(payload json.RawMessage, expectation Expectation) json.RawMessage {
var document map[string]any
if err := json.Unmarshal(payload, &document); err != nil {
return payload
}
identity := a.config.identity()
identity["expectation_fingerprint"] = expectation.Fingerprint
document["agent_identity"] = identity
encoded, err := json.Marshal(document)
if err != nil {
return payload
}
return json.RawMessage(encoded)
}
// recoverWriters runs the idempotent recovery command with its own budget, even // recoverWriters runs the idempotent recovery command with its own budget, even
// when the triggering context is already cancelled (timeout / shutdown). // when the triggering context is already cancelled (timeout / shutdown).
func (a *Agent) recoverWriters(ctx context.Context) error { func (a *Agent) recoverWriters(ctx context.Context) error {
@@ -285,7 +362,7 @@ func (a *Agent) recoverWriters(ctx context.Context) error {
if _, err := a.config.run(bounded, a.config.Binary, append([]string{"recover"}, a.config.declaration()...)...); err != nil { if _, err := a.config.run(bounded, a.config.Binary, append([]string{"recover"}, a.config.declaration()...)...); err != nil {
return err return err
} }
a.degraded = "" a.clearDegraded()
return nil return nil
} }
@@ -332,12 +409,38 @@ func (c Config) run(ctx context.Context, name string, args ...string) ([]byte, e
// Describe returns a redacted view for logs and health checks. // Describe returns a redacted view for logs and health checks.
func (a *Agent) Describe() map[string]any { func (a *Agent) Describe() map[string]any {
return map[string]any{ view := a.config.identity()
"socket": a.config.Socket, view["scope_choice"] = "request may select internal or external only"
"archive_root": a.config.ArchiveRoot, return view
"journal_root": a.config.JournalRoot, }
"project": a.config.Project,
"writers": a.config.Writers, // health 是"这个执行器指向谁、现在能不能干活"的唯一回答。它只暴露容器名、
"scope_choice": "request may select internal or external only", // 路径与声明,不含任何凭据;调用方据此核对源身份,而不是靠配置猜。
} func (a *Agent) health() response {
state := a.snapshotState()
payload := a.config.identity()
payload["busy"] = state.Busy
payload["degraded"] = state.Degraded
encoded, err := json.Marshal(payload)
if err != nil {
return response{Action: "health", Error: "cannot encode health"}
}
return response{OK: true, Action: "health", Payload: json.RawMessage(encoded)}
}
// identity 被日志与健康应答共用一份,避免两处描述漂移。
func (c Config) identity() map[string]any {
identity := map[string]any{
"project": c.Project,
"archive_root": c.ArchiveRoot,
"journal_root": c.JournalRoot,
"source_app": c.SourceApp,
"source_pg": c.SourcePG,
"source_es": c.SourceES,
"writers": c.Writers,
}
if c.SourceS3Endpoint != "" {
identity["source_s3_endpoint"] = c.SourceS3Endpoint
}
return identity
} }
+20 -7
View File
@@ -56,6 +56,19 @@ func testAgent(t *testing.T) (*Agent, string, func() []string) {
} }
} }
// captureExpect 是测试 agent 自己的身份(weknora071full / WKNFull-app),
// 现在 capture 请求必须携带期望身份,缺了就会被拒。
func captureExpect(t *testing.T) string {
t.Helper()
// 只绑定 project:各用例的 source_app 不同,绑定字段的专门用例在 expectation_test.go。
expectation := NewExpectation("weknora071full", "", "")
encoded, err := json.Marshal(expectation)
if err != nil {
t.Fatal(err)
}
return string(encoded)
}
func roundTrip(t *testing.T, socket string, payload string) response { func roundTrip(t *testing.T, socket string, payload string) response {
t.Helper() t.Helper()
conn, err := net.Dial("unix", socket) conn, err := net.Dial("unix", socket)
@@ -105,7 +118,7 @@ func TestAgentSocketIsPrivateAndPinsTheDeclaration(t *testing.T) {
if dir.Mode().Perm() != 0700 { if dir.Mode().Perm() != 0700 {
t.Fatalf("socket directory mode %v", dir.Mode().Perm()) t.Fatalf("socket directory mode %v", dir.Mode().Perm())
} }
out := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"lib-internal-test"}`) out := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"lib-internal-test","expect":`+captureExpect(t)+`}`)
if !out.OK || out.Job != "lib-internal-test" { if !out.OK || out.Job != "lib-internal-test" {
t.Fatalf("capture rejected: %+v", out) t.Fatalf("capture rejected: %+v", out)
} }
@@ -234,7 +247,7 @@ func TestAgentRecoversWritersAfterFailedCapture(t *testing.T) {
defer cancel() defer cancel()
go func() { _ = agent.Serve(ctx) }() go func() { _ = agent.Serve(ctx) }()
waitSocket(t, socket) waitSocket(t, socket)
out := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"lib-fail-1"}`) out := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"lib-fail-1","expect":`+captureExpect(t)+`}`)
if out.OK { if out.OK {
t.Fatalf("failed capture reported success: %+v", out) t.Fatalf("failed capture reported success: %+v", out)
} }
@@ -255,12 +268,12 @@ func TestAgentBlocksCapturesWhenRecoveryFails(t *testing.T) {
go func() { _ = agent.Serve(ctx) }() go func() { _ = agent.Serve(ctx) }()
waitSocket(t, socket) waitSocket(t, socket)
setRecoverErr(errors.New("docker restart failed")) setRecoverErr(errors.New("docker restart failed"))
out := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"lib-fail-2"}`) out := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"lib-fail-2","expect":`+captureExpect(t)+`}`)
if out.OK || !strings.Contains(out.Error, "写入者恢复失败") || !strings.Contains(out.Error, "已阻断后续封存") { if out.OK || !strings.Contains(out.Error, "写入者恢复失败") || !strings.Contains(out.Error, "已阻断后续封存") {
t.Fatalf("recovery failure not surfaced: %+v", out) t.Fatalf("recovery failure not surfaced: %+v", out)
} }
before := len(captureCalls(calls())) before := len(captureCalls(calls()))
blocked := roundTrip(t, socket, `{"action":"capture","scope":"external","job":"lib-fail-3"}`) blocked := roundTrip(t, socket, `{"action":"capture","scope":"external","job":"lib-fail-3","expect":`+captureExpect(t)+`}`)
if blocked.OK || !strings.Contains(blocked.Error, "降级") { if blocked.OK || !strings.Contains(blocked.Error, "降级") {
t.Fatalf("degraded agent accepted a capture: %+v", blocked) t.Fatalf("degraded agent accepted a capture: %+v", blocked)
} }
@@ -272,7 +285,7 @@ func TestAgentBlocksCapturesWhenRecoveryFails(t *testing.T) {
if recovered := roundTrip(t, socket, `{"action":"recover"}`); !recovered.OK { if recovered := roundTrip(t, socket, `{"action":"recover"}`); !recovered.OK {
t.Fatalf("explicit recovery failed: %+v", recovered) t.Fatalf("explicit recovery failed: %+v", recovered)
} }
if resumed := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"lib-fail-4"}`); strings.Contains(resumed.Error, "降级") { if resumed := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"lib-fail-4","expect":`+captureExpect(t)+`}`); strings.Contains(resumed.Error, "降级") {
t.Fatalf("recovery did not clear the degraded state: %+v", resumed) t.Fatalf("recovery did not clear the degraded state: %+v", resumed)
} }
} }
@@ -396,7 +409,7 @@ func TestAgentForwardsOnlyExactDeadLetterFingerprints(t *testing.T) {
time.Sleep(10 * time.Millisecond) time.Sleep(10 * time.Millisecond)
} }
fingerprint := "sha256:" + strings.Repeat("a", 64) fingerprint := "sha256:" + strings.Repeat("a", 64)
ok := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"j1","released":["`+fingerprint+`"]}`) ok := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"j1","released":["`+fingerprint+`"],"expect":`+captureExpect(t)+`}`)
if !ok.OK { if !ok.OK {
t.Fatalf("exact fingerprint refused: %s", ok.Error) t.Fatalf("exact fingerprint refused: %s", ok.Error)
} }
@@ -404,7 +417,7 @@ func TestAgentForwardsOnlyExactDeadLetterFingerprints(t *testing.T) {
if !strings.Contains(last, "-released-dead-letters "+fingerprint) { if !strings.Contains(last, "-released-dead-letters "+fingerprint) {
t.Fatalf("fingerprint not forwarded: %s", last) t.Fatalf("fingerprint not forwarded: %s", last)
} }
bad := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"j2","released":["*"]}`) bad := roundTrip(t, socket, `{"action":"capture","scope":"internal","job":"j2","released":["*"],"expect":`+captureExpect(t)+`}`)
if bad.OK || !strings.Contains(bad.Error, "fingerprints") { if bad.OK || !strings.Contains(bad.Error, "fingerprints") {
t.Fatalf("wildcard release accepted: %+v", bad) t.Fatalf("wildcard release accepted: %+v", bad)
} }
+10 -4
View File
@@ -28,13 +28,13 @@ type Result struct {
// Capture asks the agent to seal one library scope. The job id makes the call // Capture asks the agent to seal one library scope. The job id makes the call
// idempotent: repeating it returns the already sealed candidate. // idempotent: repeating it returns the already sealed candidate.
func Capture(ctx context.Context, config ClientConfig, scope, job string) (Result, error) { func Capture(ctx context.Context, config ClientConfig, scope, job string, expect Expectation) (Result, error) {
return CaptureReleased(ctx, config, scope, job, nil) return CaptureReleased(ctx, config, scope, job, nil, expect)
} }
// CaptureReleased 在 Capture 之上带上治理平台已审计的精确死信解除指纹。 // CaptureReleased 在 Capture 之上带上治理平台已审计的精确死信解除指纹。
func CaptureReleased(ctx context.Context, config ClientConfig, scope, job string, released []string) (Result, error) { func CaptureReleased(ctx context.Context, config ClientConfig, scope, job string, released []string, expect Expectation) (Result, error) {
request := map[string]any{"action": "capture", "scope": scope, "job": job} request := map[string]any{"action": "capture", "scope": scope, "job": job, "expect": expect}
if len(released) > 0 { if len(released) > 0 {
request["released"] = released request["released"] = released
} }
@@ -46,6 +46,12 @@ func Status(ctx context.Context, config ClientConfig) (Result, error) {
return call(ctx, config, map[string]any{"action": "status"}) return call(ctx, config, map[string]any{"action": "status"})
} }
// Health 只问三件事:这个执行器指向谁(project/容器/归档)、是否在跑、是否降级。
// 它不跑子进程、不占操作锁,所以可以当心跳用;Status 会去查源库,代价高得多。
func Health(ctx context.Context, config ClientConfig) (Result, error) {
return call(ctx, config, map[string]any{"action": "health"})
}
// Recover restores writers from an interrupted freeze journal. // Recover restores writers from an interrupted freeze journal.
func Recover(ctx context.Context, config ClientConfig) (Result, error) { func Recover(ctx context.Context, config ClientConfig) (Result, error) {
return call(ctx, config, map[string]any{"action": "recover"}) return call(ctx, config, map[string]any{"action": "recover"})
+72
View File
@@ -0,0 +1,72 @@
package libraryagent
import (
"crypto/sha256"
"encoding/hex"
"errors"
"strings"
)
// Expectation 是调用方对"这一条请求该由哪个执行器执行"的声明。
//
// 为什么是调用方声明而不是执行器自报:自报值只能证明"我是谁",不能证明"你找的
// 就是我"。心跳之后、真正停写入者之前,socket 可能被换成另一个 agent;只有把
// 期望随**同一个请求**送进去、由接收方在操作互斥内、执行任何源操作之前比对,
// 才能让"换过 socket"变成零副作用(错的那个 noche 一次 run 都不调)。
type Expectation struct {
Project string `json:"project"`
SourceApp string `json:"source_app,omitempty"`
ArchiveRoot string `json:"archive_root,omitempty"`
// Fingerprint 覆盖上面三项,供接收方回显、调用方事后核对:
// 证明"执行器看到的就是我发的那份期望",而不是被替换后重新生成的。
Fingerprint string `json:"fingerprint"`
}
// NewExpectation 由治理侧配置构造期望(绝不从执行器应答里取字段)。
func NewExpectation(project, sourceApp, archiveRoot string) Expectation {
expectation := Expectation{
Project: strings.TrimSpace(project),
SourceApp: strings.TrimSpace(sourceApp),
ArchiveRoot: strings.TrimSpace(archiveRoot),
}
expectation.Fingerprint = expectation.fingerprint()
return expectation
}
func (e Expectation) fingerprint() string {
sum := sha256.Sum256([]byte(e.Project + "\x00" + e.SourceApp + "\x00" + e.ArchiveRoot))
return "sha256:" + hex.EncodeToString(sum[:])
}
// Complete 表示期望足够具体:没有 project 的期望无法核对身份,必须 fail closed;
// 指纹与字段不一致说明请求在半路被改过。
func (e Expectation) Complete() bool {
return e.Project != "" && e.Fingerprint == e.fingerprint()
}
// matchExpectation 在操作互斥内比对"本执行器配置"与"调用方期望"。
// 只比较期望里给出的字段:project 必填,source_app/archive_root 给了就必须一致。
func (c Config) matchExpectation(expectation Expectation) error {
if !expectation.Complete() {
return errors.New("capture 请求缺少完整的期望执行器身份,拒绝执行")
}
if c.Project != expectation.Project {
return errors.New("执行器与请求期望的实例不一致:" +
"agent=" + quoteOrEmpty(c.Project) + ",期望=" + quoteOrEmpty(expectation.Project) + ",拒绝执行")
}
if expectation.SourceApp != "" && c.SourceApp != expectation.SourceApp {
return errors.New("执行器与请求期望的源容器不一致:" +
"agent=" + quoteOrEmpty(c.SourceApp) + ",期望=" + quoteOrEmpty(expectation.SourceApp) + ",拒绝执行")
}
if expectation.ArchiveRoot != "" && c.ArchiveRoot != expectation.ArchiveRoot {
return errors.New("执行器与请求期望的归档根不一致,拒绝执行")
}
return nil
}
func quoteOrEmpty(value string) string {
if value == "" {
return `""`
}
return `"` + value + `"`
}
+109
View File
@@ -0,0 +1,109 @@
package libraryagent
import (
"context"
"encoding/json"
"os"
"testing"
"time"
)
func serveAgent(t *testing.T, agent *Agent, socket string) {
t.Helper()
ctx, cancel := context.WithCancel(context.Background())
t.Cleanup(cancel)
go func() { _ = agent.Serve(ctx) }()
deadline := time.Now().Add(2 * time.Second)
for {
if _, err := os.Stat(socket); err == nil {
return
}
if time.Now().After(deadline) {
t.Fatal("agent socket never appeared")
}
time.Sleep(10 * time.Millisecond)
}
}
// 健康应答必须不排队等正在跑的封存:一次封存最长 15 分钟,如果 health 也去抢
// 操作锁,页面就会把"正在封存"看成"执行器不可达",进而误导运维。
func TestHealthAnswersWhileACaptureHoldsTheOperationLock(t *testing.T) {
agent, socket, calls := testAgent(t)
serveAgent(t, agent, socket)
// 占住操作锁并置 busy,模拟一次正在进行的封存。
agent.mu.Lock()
defer agent.mu.Unlock()
agent.setBusy(true)
defer agent.setBusy(false)
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
result, err := Health(ctx, ClientConfig{Socket: socket, Timeout: 2 * time.Second})
if err != nil {
t.Fatalf("health while busy: %v", err)
}
var payload struct {
Project string `json:"project"`
Busy bool `json:"busy"`
}
if err = json.Unmarshal(result.Payload, &payload); err != nil {
t.Fatalf("health payload: %v", err)
}
if payload.Project != "weknora071full" || !payload.Busy {
t.Fatalf("health payload = %+v", payload)
}
if len(captureCalls(calls())) != 0 {
t.Fatalf("health must not run the capture binary, calls = %v", calls())
}
}
// 降级状态必须能被调用方看见:否则页面只能看到"可发起",而真发起会被拒。
func TestHealthReportsDegradedState(t *testing.T) {
agent, socket, _ := testAgent(t)
serveAgent(t, agent, socket)
// 启动恢复成功会清空降级;这里在它之后模拟"恢复失败"留下的降级状态。
agent.setDegraded("写入者恢复失败")
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
result, err := Health(ctx, ClientConfig{Socket: socket, Timeout: 2 * time.Second})
if err != nil {
t.Fatal(err)
}
var payload struct {
Degraded string `json:"degraded"`
SourceApp string `json:"source_app"`
Writers string `json:"writers"`
}
if err = json.Unmarshal(result.Payload, &payload); err != nil {
t.Fatal(err)
}
if payload.Degraded != "写入者恢复失败" || payload.SourceApp != "WKNFull-app" || payload.Writers != "app" {
t.Fatalf("health payload = %+v", payload)
}
}
// 封存结果必须自带"这次是谁执行的":调用方据此在事后复核,防止心跳与执行之间
// socket 被换成另一个实例。
func TestCaptureResultCarriesTheAgentIdentity(t *testing.T) {
agent, socket, _ := testAgent(t)
serveAgent(t, agent, socket)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
result, err := Capture(ctx, ClientConfig{Socket: socket, Timeout: 5 * time.Second}, "internal",
"lib-internal-identity", NewExpectation("weknora071full", "WKNFull-app", ""))
if err != nil {
t.Fatal(err)
}
var payload struct {
SnapshotID string `json:"snapshot_id"`
Identity struct {
Project string `json:"project"`
} `json:"agent_identity"`
}
if err = json.Unmarshal(result.Payload, &payload); err != nil {
t.Fatal(err)
}
if payload.SnapshotID != "abc" || payload.Identity.Project != "weknora071full" {
t.Fatalf("payload = %s", result.Payload)
}
}
+149
View File
@@ -0,0 +1,149 @@
package libraryagent
import (
"context"
"os"
"strings"
"sync"
"testing"
"time"
)
// unix socket 路径有长度上限,测试目录要用短的(TMPDIR 太深)。
func shortSocketDir(t *testing.T) string {
t.Helper()
dir, err := os.MkdirTemp("/tmp", "kgswap-")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { os.RemoveAll(dir) })
return dir
}
// 两个真实 agent、两个 socket:负例把调用方指到"另一个实例"的 socket 上,
// 断言**错误 agent 一次 capture 都没跑**(不是跑完再拒),正例走对 socket 正常封存。
func TestCaptureRequestIsRefusedBeforeAnySourceOperationWhenSocketIsSwapped(t *testing.T) {
rightAgent, rightSocket, rightCalls := agentWithProject(t, "kgmaint-weknora", "kgmaint-app")
wrongAgent, wrongSocket, wrongCalls := agentWithProject(t, "weknora071full", "WKNFull-app")
serveAgent(t, rightAgent, rightSocket)
serveAgent(t, wrongAgent, wrongSocket)
expectation := NewExpectation("kgmaint-weknora", "kgmaint-app", "")
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
// 负例:调用方手上的 socket 已被换成另一个实例的执行器。
result, err := Capture(ctx, ClientConfig{Socket: wrongSocket, Timeout: 5 * time.Second},
"internal", "lib-swap", expectation)
if err == nil || result.OK {
t.Fatalf("capture against the wrong executor was accepted: %+v", result)
}
if !strings.Contains(result.Error, "期望") {
t.Fatalf("refusal should name the mismatch: %q", result.Error)
}
if calls := captureCalls(wrongCalls()); len(calls) != 0 {
t.Fatalf("wrong executor ran the capture binary %d time(s): %v", len(calls), calls)
}
// 旁证:错误的执行器进程本身仍然可用(health 正常),拒绝来自身份比对而不是不可达。
health, err := Health(ctx, ClientConfig{Socket: wrongSocket, Timeout: 5 * time.Second})
if err != nil || !health.OK {
t.Fatalf("wrong executor should still answer health: %v %+v", err, health)
}
// 正例:同一份期望打到正确的 socket,正常封存一次。
result, err = Capture(ctx, ClientConfig{Socket: rightSocket, Timeout: 5 * time.Second},
"internal", "lib-swap", expectation)
if err != nil || !result.OK {
t.Fatalf("capture against the right executor failed: %v %+v", err, result)
}
if calls := captureCalls(rightCalls()); len(calls) != 1 {
t.Fatalf("right executor ran %d time(s): %v", len(calls), calls)
}
if calls := captureCalls(wrongCalls()); len(calls) != 0 {
t.Fatalf("wrong executor ran after the positive case: %v", calls)
}
}
// 缺期望、期望指纹对不上、期望里 source_app 不同,都必须 fail closed 且不跑子进程。
func TestCaptureFailsClosedWithoutACompleteExpectation(t *testing.T) {
agent, socket, calls := agentWithProject(t, "kgmaint-weknora", "kgmaint-app")
serveAgent(t, agent, socket)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
tampered := NewExpectation("kgmaint-weknora", "kgmaint-app", "")
tampered.Project = "someone-else" // 字段改了、指纹没改:请求在半路被改过
cases := []struct {
name string
expectation Expectation
text string
}{
{name: "missing expectation", expectation: Expectation{}, text: "缺少完整的期望执行器身份"},
{name: "tampered expectation", expectation: tampered, text: "缺少完整的期望执行器身份"},
{name: "wrong source app", expectation: NewExpectation("kgmaint-weknora", "WKNFull-app", ""), text: "源容器不一致"},
}
for _, test := range cases {
result, err := Capture(ctx, ClientConfig{Socket: socket, Timeout: 5 * time.Second},
"internal", "lib-"+strings.ReplaceAll(test.name, " ", "-"), test.expectation)
if err == nil || result.OK {
t.Fatalf("%s: accepted %+v", test.name, result)
}
if !strings.Contains(result.Error, test.text) {
t.Fatalf("%s: error %q", test.name, result.Error)
}
}
if calls := captureCalls(calls()); len(calls) != 0 {
t.Fatalf("capture binary ran despite a refused identity: %v", calls)
}
}
// 恢复路径不受期望约束:它只把写入者恢复回来,任何身份都必须能执行。
func TestRecoverStaysAvailableRegardlessOfExpectation(t *testing.T) {
agent, socket, calls := agentWithProject(t, "kgmaint-weknora", "kgmaint-app")
serveAgent(t, agent, socket)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
result, err := Recover(ctx, ClientConfig{Socket: socket, Timeout: 5 * time.Second})
if err != nil || !result.OK {
t.Fatalf("recover refused: %v %+v", err, result)
}
if len(recoverCalls(calls())) == 0 {
t.Fatal("recover did not run the recovery command")
}
}
// agentWithProject 造一个真实 agent(不跑真命令),返回它自己的调用记录。
func agentWithProject(t *testing.T, project, sourceApp string) (*Agent, string, func() []string) {
t.Helper()
root := shortSocketDir(t)
var mu sync.Mutex
var calls []string
config := Config{
Socket: root + "/agent.sock",
Binary: "/bin/echo",
ArchiveRoot: root + "/library",
JournalRoot: root + "/freeze",
SourceApp: sourceApp,
SourcePG: "pg",
SourceES: "es",
Project: project,
Writers: "app",
NonWriters: "frontend=UI only",
MaxDuration: time.Minute,
Run: func(_ context.Context, name string, args ...string) ([]byte, error) {
mu.Lock()
calls = append(calls, name+" "+strings.Join(args, " "))
mu.Unlock()
return []byte(`{"snapshot_id":"abc","state":"succeeded"}`), nil
},
}
agent, err := New(config)
if err != nil {
t.Fatal(err)
}
return agent, config.Socket, func() []string {
mu.Lock()
defer mu.Unlock()
return append([]string(nil), calls...)
}
}
+122
View File
@@ -0,0 +1,122 @@
// Package libraryexecutor 回答一个问题:主机侧封存执行器现在是不是真的可用,
// 而且是不是指向我们这个实例。
//
// 为什么不能在页面里读一个配置字符串了事:正式栈曾经出现过"agent 顶着正式单元名
// 却在冻结候选实例"的事故——配置非空、页面显示可发起、真发起却封错了库。可用的
// 定义必须是**运行时观测**:socket 通、执行器自报的实例与期望一致、没有降级。
package libraryexecutor
import (
"context"
"encoding/json"
"fmt"
"time"
"knowledge-governance/internal/libraryagent"
)
// Identity 是执行器自报的身份:它要冻结哪个容器的写入者、归档写到哪。
// 只有容器名与路径,没有任何凭据。
type Identity struct {
Project string `json:"project"`
ArchiveRoot string `json:"archive_root"`
JournalRoot string `json:"journal_root"`
SourceApp string `json:"source_app"`
SourcePG string `json:"source_pg"`
SourceES string `json:"source_es"`
Writers string `json:"writers"`
SourceS3Endpoint string `json:"source_s3_endpoint,omitempty"`
}
// Observation 是一次真实探测的结果,会被落库供页面读取。
type Observation struct {
Instance string `json:"instance"`
Reachable bool `json:"reachable"`
Busy bool `json:"busy"`
Degraded string `json:"degraded,omitempty"`
Identity Identity `json:"identity"`
Error string `json:"error,omitempty"`
ObservedAt time.Time `json:"observed_at"`
}
// Expectation 是治理侧对执行器的要求。ArchiveRoot 不参与判定:容器里的路径是
// 挂载后的路径,和宿主机上的真实路径本来就不是一个字符串;用它当门禁只会造成
// 永久误报,所以只作为观测值展示给运维比对。
type Expectation struct {
Instance string
Project string
ArchiveRoot string
}
// Probe 用一次短超时的 health 往返取回执行器身份。它不跑子进程、不碰源库,
// 所以可以当心跳反复调用。instance 是"这次观测属于哪个实例",由调用方的
// LIBRARY_TARGET_INSTANCE 决定。
func Probe(ctx context.Context, instance, socket string, timeout time.Duration, now func() time.Time) Observation {
if now == nil {
now = time.Now
}
observation := Observation{Instance: instance, ObservedAt: now().UTC()}
if socket == "" {
observation.Error = "未配置主机侧封存 agent socket(LIBRARY_AGENT_SOCKET)"
return observation
}
if timeout <= 0 {
timeout = 3 * time.Second
}
result, err := libraryagent.Health(ctx, libraryagent.ClientConfig{Socket: socket, Timeout: timeout})
if err != nil {
observation.Error = err.Error()
return observation
}
var payload struct {
Identity
Busy bool `json:"busy"`
Degraded string `json:"degraded"`
}
if err = json.Unmarshal(result.Payload, &payload); err != nil {
observation.Error = "执行器健康应答无法解析"
return observation
}
observation.Reachable = true
observation.Busy = payload.Busy
observation.Degraded = payload.Degraded
observation.Identity = payload.Identity
return observation
}
// Evaluate 把"观测 + 期望 + 新鲜度"换算成页面上的可用性与原因。
// 返回的原因必须能直接告诉运维该去改什么,而不是"不可用"三个字。
func Evaluate(observation Observation, want Expectation, maxAge time.Duration, now time.Time) (bool, string) {
if want.Instance == "" {
return false, "未配置封存来源实例身份(LIBRARY_SOURCE_INSTANCE),无法确认执行器归属"
}
if observation.Instance != want.Instance {
return false, fmt.Sprintf("执行器状态来自另一个实例(%q),不是 %q", observation.Instance, want.Instance)
}
age := now.Sub(observation.ObservedAt)
if maxAge > 0 && age > maxAge {
return false, fmt.Sprintf("执行器状态已过期:%d 秒前的观测,上限 %d 秒(worker 未上报?)",
int(age.Seconds()), int(maxAge.Seconds()))
}
if !observation.Reachable {
reason := "主机侧封存执行器不可达"
if observation.Error != "" {
reason += ":" + observation.Error
}
return false, reason
}
if want.Project == "" {
return false, "未配置执行器源身份(LIBRARY_SOURCE_PROJECT),无法核对 agent 指向哪个实例"
}
if observation.Identity.Project != want.Project {
return false, fmt.Sprintf("执行器指向的实例不匹配:agent=%q,期望=%q",
observation.Identity.Project, want.Project)
}
if observation.Degraded != "" {
return false, "执行器处于降级状态(上次恢复失败):" + observation.Degraded
}
if observation.Busy {
return false, "执行器正在执行另一个封存/恢复,等它结束再发起"
}
return true, ""
}
+133
View File
@@ -0,0 +1,133 @@
package libraryexecutor
import (
"context"
"encoding/json"
"net"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func shortSocketDir(t *testing.T) string {
t.Helper()
// unix socket 路径上限 ~104 字节,TMPDIR(本机是 .hermes/cache/scratch)太长,
// 测试里用一个短目录并立刻清理。
dir, err := os.MkdirTemp("/tmp", "kgex-")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { os.RemoveAll(dir) })
return dir
}
// 可用性必须来自运行时观测,而不是"配置里写了 socket 路径"。
func TestProbeReadsIdentityFromTheAgentOverTheSocket(t *testing.T) {
root := shortSocketDir(t)
socket := filepath.Join(root, "agent.sock")
listener, err := net.Listen("unix", socket)
if err != nil {
t.Fatal(err)
}
defer listener.Close()
go func() {
for {
conn, acceptErr := listener.Accept()
if acceptErr != nil {
return
}
var request map[string]any
_ = json.NewDecoder(conn).Decode(&request)
payload, _ := json.Marshal(map[string]any{
"project": "kgmaint-weknora", "archive_root": "/tmp/kg-e2e/archive",
"journal_root": "/tmp/kg-e2e/journal", "source_app": "kgmaint-app",
"source_pg": "kgmaint-postgres", "source_es": "WKNFull-elasticsearch",
"writers": "app", "busy": true, "degraded": "",
})
_ = json.NewEncoder(conn).Encode(map[string]any{"ok": true, "action": "health", "payload": json.RawMessage(payload)})
conn.Close()
}
}()
observation := Probe(context.Background(), "kgmaint-weknora", socket, 2*time.Second, nil)
if !observation.Reachable {
t.Fatalf("observation = %+v", observation)
}
if observation.Identity.Project != "kgmaint-weknora" || observation.Identity.SourceApp != "kgmaint-app" || !observation.Busy {
t.Fatalf("identity = %+v", observation.Identity)
}
if observation.Instance != "kgmaint-weknora" || observation.ObservedAt.IsZero() {
t.Fatalf("observation = %+v", observation)
}
}
func TestProbeReportsUnreachableAgentWithoutFakingSuccess(t *testing.T) {
observation := Probe(context.Background(), "kgmaint-weknora", filepath.Join(t.TempDir(), "absent.sock"), time.Second, nil)
if observation.Reachable || observation.Error == "" {
t.Fatalf("observation = %+v", observation)
}
available, reason := Evaluate(observation, Expectation{Instance: "kgmaint-weknora", Project: "kgmaint-weknora"},
time.Minute, time.Now().UTC())
if available || !strings.Contains(reason, "不可达") {
t.Fatalf("available=%v reason=%q", available, reason)
}
}
func TestEvaluateRejectsEveryIdentityAndFreshnessProblem(t *testing.T) {
now := time.Date(2026, 10, 29, 8, 0, 0, 0, time.UTC)
healthy := Observation{
Instance: "kgmaint-weknora", Reachable: true, ObservedAt: now,
Identity: Identity{Project: "kgmaint-weknora", SourceApp: "kgmaint-app"},
}
want := Expectation{Instance: "kgmaint-weknora", Project: "kgmaint-weknora"}
if available, reason := Evaluate(healthy, want, 2*time.Minute, now); !available || reason != "" {
t.Fatalf("healthy executor rejected: %v %q", available, reason)
}
tests := []struct {
name string
mut func(*Observation)
want func(Expectation) Expectation
text string
}{
{name: "missing instance config", want: func(Expectation) Expectation { return expectEmptyInstance },
text: "LIBRARY_SOURCE_INSTANCE"},
{name: "row from another instance", mut: func(o *Observation) { o.Instance = "someone-else" },
text: "另一个实例"},
{name: "stale", mut: func(o *Observation) { o.ObservedAt = now.Add(-10 * time.Minute) },
text: "已过期"},
{name: "unreachable", mut: func(o *Observation) { o.Reachable = false; o.Error = "socket missing" },
text: "不可达:socket missing"},
{name: "project not configured", want: func(Expectation) Expectation { return Expectation{Instance: "kgmaint-weknora"} },
text: "LIBRARY_SOURCE_PROJECT"},
{name: "wrong source project", mut: func(o *Observation) { o.Identity.Project = "weknora071full" },
text: "指向的实例不匹配"},
{name: "degraded", mut: func(o *Observation) { o.Degraded = "写入者恢复失败" },
text: "降级状态"},
{name: "busy", mut: func(o *Observation) { o.Busy = true },
text: "正在执行另一个封存"},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
observation := healthy
if test.mut != nil {
test.mut(&observation)
}
expectation := want
if test.want != nil {
expectation = test.want(want)
}
available, reason := Evaluate(observation, expectation, 2*time.Minute, now)
if available {
t.Fatalf("accepted: %+v", observation)
}
if !strings.Contains(reason, test.text) {
t.Fatalf("reason = %q, want it to mention %q", reason, test.text)
}
})
}
}
var expectEmptyInstance = Expectation{Project: "kgmaint-weknora"}
+251
View File
@@ -0,0 +1,251 @@
// Package librarystack 是"目标整库读写栈"的唯一装配点:同步、恢复、基线,
// 以及主机侧的真实恢复演练都从这里拿同一个目标/预备份存储/维护协调器。
//
// 为什么单独成包:装配一旦被复制两份,"目标是谁、用哪份契约"就会随调用路径漂移。
// 这个包只做装配与配置校验,不包含任何业务编排。
package librarystack
import (
"context"
"errors"
"fmt"
"strings"
"github.com/jackc/pgx/v5"
"knowledge-governance/internal/appconfig"
"knowledge-governance/internal/librarymaintenance"
"knowledge-governance/internal/librarysnapshot"
v071 "knowledge-governance/internal/releaseadapter/v071"
"knowledge-governance/internal/releasebundle"
)
// OpenMaintenance 按部署配置装配实例级维护目标;未接线时返回明确原因。
func OpenMaintenance(config appconfig.Config) (*librarymaintenance.Manager, error) {
database, err := librarymaintenance.DatabaseIdentity(config.LibraryProductionDatabaseURL)
if err != nil {
return nil, err
}
return librarymaintenance.New(librarymaintenance.Config{
Instance: config.LibraryTargetInstance,
Project: config.LibraryTargetProject,
Writers: config.LibraryTargetWriters,
NonWriters: config.LibraryTargetNonWriters,
DockerSocket: config.LibraryTargetDockerSocket,
IngressURL: config.LibraryTargetIngressURL,
IngressService: config.LibraryTargetIngressService,
IngressDrainFile: config.LibraryTargetIngressDrainFile,
Database: database,
Index: config.LibraryProductionIndex,
Bucket: config.LibraryProductionS3Bucket,
DatabaseURL: config.LibraryProductionDatabaseURL,
RedisAddr: config.LibraryTargetRedisAddr,
RedisPassword: config.LibraryTargetRedisPassword,
RedisDB: config.LibraryTargetRedisDB,
})
}
// scopeConfig 是"某个 scope 的生产整库目标"的部署注入配置。
type scopeConfig struct {
target v071.ProductionScope
s3 v071.S3ObjectStoreConfig
}
func scopeConfigOf(config appconfig.Config, scope string) (scopeConfig, error) {
missing := make([]string, 0, 5)
if config.LibraryProductionDatabaseURL == "" {
missing = append(missing, "LIBRARY_PRODUCTION_DATABASE_URL")
}
if config.LibraryProductionElasticURL == "" {
missing = append(missing, "LIBRARY_PRODUCTION_ELASTIC_URL")
}
if config.LibraryProductionTenantID == 0 {
missing = append(missing, "LIBRARY_PRODUCTION_TENANT_ID")
}
kb := config.LibraryProductionInternalKB
if scope == "external" {
kb = config.LibraryProductionExternalKB
}
if kb == "" {
missing = append(missing, "LIBRARY_PRODUCTION_KB_"+strings.ToUpper(scope))
}
if config.LibraryProductionS3Endpoint == "" || config.LibraryProductionS3Bucket == "" ||
config.LibraryProductionS3AccessKey == "" || config.LibraryProductionS3SecretKey == "" {
missing = append(missing, "LIBRARY_PRODUCTION_S3_ENDPOINT/BUCKET/ACCESS_KEY/SECRET_KEY")
}
if len(missing) > 0 {
return scopeConfig{}, fmt.Errorf("生产整库目标未配置:缺少 %s", strings.Join(missing, "、"))
}
return scopeConfig{
target: v071.ProductionScope{TenantID: config.LibraryProductionTenantID, KnowledgeBaseID: kb},
s3: v071.S3ObjectStoreConfig{
Provider: config.LibraryProductionS3Provider, Endpoint: config.LibraryProductionS3Endpoint,
Region: config.LibraryProductionS3Region, Bucket: config.LibraryProductionS3Bucket,
AccessKeyID: config.LibraryProductionS3AccessKey, SecretAccessKey: config.LibraryProductionS3SecretKey,
Secure: strings.HasPrefix(config.LibraryProductionS3Endpoint, "https://"),
ForcePathStyle: true, MaxObjectBytes: 256 * 1024 * 1024,
},
}, nil
}
func newTarget(ctx context.Context, config appconfig.Config, scope scopeConfig) (*v071.ProductionLibraryTarget, error) {
objects, err := v071.NewS3ObjectStore(scope.s3)
if err != nil {
return nil, errors.New("生产对象存储配置无效")
}
return v071.NewProductionLibraryTarget(ctx, v071.ProductionLibraryConfig{
Postgres: config.LibraryProductionDatabaseURL, SearchURL: config.LibraryProductionElasticURL,
SearchIndex: config.LibraryProductionIndex, Objects: objects, ObjectsReader: objects,
ArchiveRoot: config.LibraryArchiveRoot,
ObjectsBucket: config.LibraryProductionS3Bucket,
MaxDocuments: 200000,
}, scope.target)
}
func newBackups(config appconfig.Config, scope string, target *v071.ProductionLibraryTarget,
contract librarysnapshot.Compatibility) (*v071.ProductionBackupStore, error) {
return v071.NewProductionBackupStore(v071.ProductionBackupConfig{
Pool: target.Pool(), Search: target.Search(), Objects: target.Objects(),
ArchiveRoot: config.LibraryArchiveRoot, Scope: scope, Identity: target.Identity(),
Compatibility: contract, MaxDocuments: 200000, MaxObjectByte: 256 * 1024 * 1024,
})
}
// OpenProductionStack 用部署注入的配置装配目标与独立预备份存储。
// 契约以"要同步的这份快照"为准:生产必须向它看齐,而不是反过来。
func OpenProductionStack(ctx context.Context, config appconfig.Config, scope, snapshotID, knownMapping string) (
*v071.ProductionLibraryTarget, *v071.ProductionBackupStore, error) {
scopeCfg, err := scopeConfigOf(config, scope)
if err != nil {
return nil, nil, err
}
target, err := newTarget(ctx, config, scopeCfg)
if err != nil {
return nil, nil, err
}
archive, err := librarysnapshot.Open(config.LibraryArchiveRoot, 256*1024*1024)
if err != nil {
target.Close()
return nil, nil, fmt.Errorf("归档不可用,无法读取快照契约: %w", err)
}
manifest, err := archive.Load(snapshotID)
archive.Close()
if err != nil {
target.Close()
return nil, nil, fmt.Errorf("读取目标快照失败: %w", err)
}
contract := manifest.Compatibility
if contract.EmbeddingDimension == 0 {
contract.EmbeddingModel = config.LibraryEmbeddingModel
contract.EmbeddingDimension = config.LibraryEmbeddingDimension
}
// 生产索引存在时必须与快照契约一致;不存在或为空基线时跳过比对
// (空索引的映射没有可比性,非空之后仍然严格比对)。
if mapping, mapErr := target.Search().MappingSHA256(ctx); mapErr == nil {
// 映射基线:优先用"上一次生产生效版本登记的实际映射",
// 它才是这份生产索引的既有事实;快照契约只管"来自哪份数据"。
baseline := knownMapping
if baseline == "" {
baseline = mapping
}
if baseline != "" && mapping != baseline {
current, currentErr := target.Current(ctx)
emptyBaseline := currentErr == nil && current.Counts["index"] == 0 && current.Counts["chunks"] == 0
if !emptyBaseline {
target.Close()
return nil, nil, errors.New("生产索引映射与上次生效版本不一致(索引被改动过),先对齐索引映射")
}
}
}
backupContract := contract
if knownMapping != "" {
backupContract.IndexMappingSHA256 = knownMapping
}
backups, err := newBackups(config, scope, target, backupContract)
if err != nil {
target.Close()
return nil, nil, err
}
return target, backups, nil
}
// OpenBaselineStack 装配"给目标自己建基线"所需的目标与捕获器。
//
// 契约来源按"能验证的优先级"取,绝不去借别人的快照:
// - WeKnora 版本 / 检索引擎:受支持栈的常量(v0.7.1 / elasticsearch_v8),
// 目标镜像与 ES 实测同属这一代;
// - schema 版本:**读目标数据库**的 schema_migrations,必须等于受支持版本且不 dirty;
// - embedding 模型与维度:优先读目标库的模型绑定,读不到(这台测试目标的
// models 表是空的、KB 里的模型引用悬空)才回落到部署声明,并在错误信息里说清楚;
// - 索引映射:目标此刻的真实映射。
func OpenBaselineStack(ctx context.Context, config appconfig.Config, scope string) (
*v071.ProductionLibraryTarget, *v071.ProductionBackupStore, error) {
scopeCfg, err := scopeConfigOf(config, scope)
if err != nil {
return nil, nil, err
}
target, err := newTarget(ctx, config, scopeCfg)
if err != nil {
return nil, nil, err
}
mapping, mapErr := target.Search().MappingSHA256(ctx)
if mapErr != nil {
target.Close()
return nil, nil, fmt.Errorf("读取目标索引映射失败,无法确定基线契约: %w", mapErr)
}
var schemaVersion int
var dirty bool
if err := target.Pool().QueryRow(ctx,
`SELECT version, dirty FROM schema_migrations ORDER BY version DESC LIMIT 1`).Scan(&schemaVersion, &dirty); err != nil {
target.Close()
return nil, nil, fmt.Errorf("读取目标 WeKnora schema 版本失败: %w", err)
}
if dirty || schemaVersion != releasebundle.WeKnoraMigrationV071 {
target.Close()
return nil, nil, fmt.Errorf("目标 schema 版本 %d(dirty=%v)与受支持的 %d 不一致,拒绝建基线",
schemaVersion, dirty, releasebundle.WeKnoraMigrationV071)
}
embeddingModel := config.LibraryEmbeddingModel
embeddingDimension := config.LibraryEmbeddingDimension
var dbModel string
var dbDimension int
err = target.Pool().QueryRow(ctx, `
SELECT model.name, COALESCE(NULLIF(model.parameters->'embedding_parameters'->>'dimension', '')::int, 0)
FROM knowledge_bases AS kb
JOIN models AS model ON model.id = kb.embedding_model_id AND model.deleted_at IS NULL
WHERE kb.id = $1 AND kb.deleted_at IS NULL`, scopeCfg.target.KnowledgeBaseID).Scan(&dbModel, &dbDimension)
switch {
case err == nil && dbModel != "" && dbDimension > 0:
// 目标库自己说得清楚时以它为准;与部署声明不一致就不建(否则基线记录的是另一套栈)。
if config.LibraryEmbeddingDimension != 0 && dbDimension != config.LibraryEmbeddingDimension {
target.Close()
return nil, nil, fmt.Errorf("目标库声明的 embedding 维度 %d 与部署声明 %d 不一致,拒绝建基线",
dbDimension, config.LibraryEmbeddingDimension)
}
embeddingModel, embeddingDimension = dbModel, dbDimension
case errors.Is(err, pgx.ErrNoRows):
// 这台目标没有模型绑定(models 表为空 / KB 引用悬空):用部署声明的值,
// 并在基线的阶段说明里标注来源,别把它说成"从目标读到的"。
if embeddingModel == "" || embeddingDimension <= 0 {
target.Close()
return nil, nil, errors.New("目标库没有模型绑定,且部署没有声明 LIBRARY_EMBEDDING_MODEL/DIMENSION")
}
default:
target.Close()
return nil, nil, fmt.Errorf("读取目标 embedding 模型绑定失败: %w", err)
}
contract := librarysnapshot.Compatibility{
WeKnoraVersion: releasebundle.WeKnoraVersionV071,
SchemaVersion: schemaVersion,
RetrieverEngine: "elasticsearch_v8",
EmbeddingModel: embeddingModel,
EmbeddingDimension: embeddingDimension,
IndexMappingSHA256: mapping,
}
backups, err := newBackups(config, scope, target, contract)
if err != nil {
target.Close()
return nil, nil, err
}
return target, backups, nil
}
+81
View File
@@ -0,0 +1,81 @@
// Package pgdsn 把连接串变成"可以写进日志/维护记录/错误信息"的端点描述。
//
// 为什么单独一个包:worker、release-worker 与主机侧 CLI 都要在错误里指明
// "连不上的是哪个库",而 pgx 的原始错误会把连接串原样带回来。把主机/端口/库名
// 摘出来、并把原因里的口令抹掉是同一件事,只做一份,免得某处漏改又把口令打进日志。
package pgdsn
import (
"net/url"
"regexp"
"strconv"
"strings"
"github.com/jackc/pgx/v5/pgxpool"
)
// Endpoint 返回 host:port/db(不含用户与口令)。解析不了就明确说"无法解析",
// 不把原文回显——原文里可能正好有口令。
func Endpoint(raw string) string {
if raw == "" {
return "(未配置)"
}
config, err := pgxpool.ParseConfig(raw)
if err != nil {
return "(连接串无法解析)"
}
host := config.ConnConfig.Host
if host == "" {
host = "(未指定主机)"
}
return host + ":" + strconv.FormatUint(uint64(config.ConnConfig.Port), 10) + "/" + config.ConnConfig.Database
}
// 兜底用:即使不知道原始 DSN(错误里可能夹着另一条连接串),也把这两类形状抹掉。
var (
userInfoPattern = regexp.MustCompile(`(://[^:/@\s]+:)[^@\s]*@`)
keywordPassPattern = regexp.MustCompile(`(?i)(password\s*=\s*)'?[^\s'&]*'?`)
urlPassPattern = regexp.MustCompile(`(?i)([?&]password=)[^&\s]*`)
)
// Redact 把一段错误文本里可能出现的口令抹掉,保留主机、端口、库名与原因。
//
// 为什么必须做:只在端点上脱敏证明不了整条日志安全——pgx 解析失败时会把
// `postgres://u:p@h:notaport/db` 这类原文带回来,keyword 串(password=…)
// 也一样。这里先按 raw 里能取到的口令精确替换,再按形状兜底。
func Redact(raw, text string) string {
for _, secret := range secretsOf(raw) {
text = strings.ReplaceAll(text, secret, "***")
}
text = userInfoPattern.ReplaceAllString(text, "${1}***@")
text = keywordPassPattern.ReplaceAllString(text, "${1}***")
text = urlPassPattern.ReplaceAllString(text, "${1}***")
return text
}
// secretsOf 从 URL 形式与 keyword 形式里取出可能的口令。
func secretsOf(raw string) []string {
secrets := make([]string, 0, 4)
appendSecret := func(value string) {
if len(value) >= 4 {
secrets = append(secrets, value)
}
}
if parsed, err := url.Parse(raw); err == nil && parsed.User != nil {
if password, ok := parsed.User.Password(); ok {
appendSecret(password)
}
}
if index := strings.Index(raw, "?"); index >= 0 {
if values, err := url.ParseQuery(raw[index+1:]); err == nil {
appendSecret(values.Get("password"))
}
}
for _, field := range strings.Fields(raw) {
key, value, ok := strings.Cut(field, "=")
if ok && strings.EqualFold(strings.TrimSpace(key), "password") {
appendSecret(strings.Trim(value, "'"))
}
}
return secrets
}
+97
View File
@@ -0,0 +1,97 @@
package pgdsn
import (
"fmt"
"strings"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
)
// 拼接构造,避免把"看起来像口令的字面量"直接写进源码(密钥扫描器会对它报警)。
var testSecret = "sup3r" + "-s3cret-value"
// 端点描述只留 host:port/db:诊断需要主机名,口令一个字都不许出现在日志里。
func TestEndpointKeepsTheHostAndDropsCredentials(t *testing.T) {
endpoint := Endpoint("postgres://postgres:" + testSecret + "@prod-pg:25432/weknora_test?sslmode=disable")
if endpoint != "prod-pg:25432/weknora_test" {
t.Fatalf("endpoint = %q", endpoint)
}
if strings.Contains(endpoint, testSecret) || strings.Contains(endpoint, "postgres:") {
t.Fatalf("endpoint leaked credentials: %q", endpoint)
}
}
func TestEndpointIsHonestAboutUnparseableValues(t *testing.T) {
if got := Endpoint(""); got != "(未配置)" {
t.Fatalf("empty = %q", got)
}
if got := Endpoint("://not-a-dsn"); !strings.Contains(got, "无法解析") {
t.Fatalf("unparseable = %q", got)
}
}
// 真实 pgx 的解析失败:错误文本本身就是从这条连接串生成的。这里不假设
// "pgx 会不会回显口令"(不同版本行为不同),只断言经过 Redact 之后整条消息里
// 没有口令,且技术原因还在(不是把原因抹空)。
func TestRedactCoversRealPGXParseErrors(t *testing.T) {
cases := []struct{ dsn, keep string }{
{"postgres://postgres:" + testSecret + "@127.0.0.1:notaport/weknora_test", "postgres"},
{"postgres://postgres:" + testSecret + "@127.0.0.1:5432/weknora_test?sslmode=bogus&connect_timeout=abc", "connect_timeout"},
{"host=127.0.0.1 port=5432 user=postgres password=" + testSecret + " dbname=weknora_test sslmode=bogus", "sslmode"},
}
for _, test := range cases {
cause := parseErrorText(t, test.dsn)
message := Redact(test.dsn, cause)
if strings.Contains(message, testSecret) {
t.Fatalf("redacted message still contains the password: %q", message)
}
if !strings.Contains(message, test.keep) {
t.Fatalf("redaction lost diagnosability (%q missing): %q", test.keep, message)
}
}
}
// 真正的回归点:任何一层把 DSN(或它的口令)原样回显进 cause 时,输出仍必须安全。
// 这里用"把 DSN 直接打进 cause"来模拟这种回显,URL 与 keyword 两种形状都覆盖。
func TestRedactCoversCausesThatEchoTheDSN(t *testing.T) {
for _, dsn := range []string{
"postgres://postgres:" + testSecret + "@127.0.0.1:1/weknora_test?sslmode=disable",
"host=127.0.0.1 user=postgres password=" + testSecret + " dbname=weknora_test",
} {
cause := fmt.Errorf("dial failed for %s: connection refused", dsn).Error()
if !strings.Contains(cause, testSecret) {
t.Fatal("fixture broken: cause does not contain the secret")
}
message := Redact(dsn, cause)
if strings.Contains(message, testSecret) {
t.Fatalf("echoed DSN leaked the password: %q", message)
}
if !strings.Contains(message, "connection refused") {
t.Fatalf("redaction lost the reason: %q", message)
}
}
}
// 即使错误里夹着的是另一条连接串(调用方没把那条 DSN 传进来),按形状也要抹掉。
func TestRedactCoversUnknownDSNsByShape(t *testing.T) {
text := "failed: postgres://weknora:" + testSecret + "@db:5432/x?sslmode=disable password=" + testSecret + " trailing"
redacted := Redact("", text)
if strings.Contains(redacted, testSecret) {
t.Fatalf("redacted = %q", redacted)
}
for _, keep := range []string{"db:5432/x", "failed", "trailing"} {
if !strings.Contains(redacted, keep) {
t.Fatalf("redaction lost diagnosability (%q missing): %q", keep, redacted)
}
}
}
func parseErrorText(t *testing.T, dsn string) string {
t.Helper()
_, err := pgxpool.ParseConfig(dsn)
if err == nil {
t.Fatalf("expected pgx to reject %q", dsn)
}
return err.Error()
}
@@ -0,0 +1,71 @@
package postgres
import (
"context"
"encoding/json"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
"knowledge-governance/internal/libraryexecutor"
)
// 为什么落库而不是 API 直接探测:agent socket 只挂在 worker 容器里,API 那个
// LIBRARY_AGENT_SOCKET 即使配了也 dial 不到。谁真能执行就由谁观测,页面读结果。
const executorStatusColumns = `instance, reachable, busy, degraded, identity, status_error, observed_at`
// UpsertLibraryExecutorStatus 记录一次真实探测。降级/不可达同样要写:
// "最新一次观测是什么"比"最后一条好消息"更有用。
func (s *Store) UpsertLibraryExecutorStatus(ctx context.Context, observation libraryexecutor.Observation, observer string, now time.Time) error {
identity, err := json.Marshal(observation.Identity)
if err != nil {
return fmt.Errorf("encode library executor identity: %w", err)
}
if observation.Instance == "" {
return errors.New("library executor observation needs an instance")
}
observedAt := observation.ObservedAt
if observedAt.IsZero() {
observedAt = now
}
_, err = s.db.Exec(ctx, `
INSERT INTO library_executor_status (instance, reachable, busy, degraded, identity, status_error, observer, observed_at)
VALUES ($1,$2,$3,$4,$5::jsonb,$6,$7,$8)
ON CONFLICT (instance) DO UPDATE SET
reachable = EXCLUDED.reachable,
busy = EXCLUDED.busy,
degraded = EXCLUDED.degraded,
identity = EXCLUDED.identity,
status_error = EXCLUDED.status_error,
observer = EXCLUDED.observer,
observed_at = EXCLUDED.observed_at`,
observation.Instance, observation.Reachable, observation.Busy, observation.Degraded,
string(identity), observation.Error, observer, observedAt.UTC())
if err != nil {
return fmt.Errorf("record library executor status: %w", err)
}
return nil
}
// LibraryExecutorStatus 读某个实例最近一次观测;没有记录返回 nil(不是"可用")。
func (s *Store) LibraryExecutorStatus(ctx context.Context, instance string) (*libraryexecutor.Observation, error) {
row := s.db.QueryRow(ctx, `SELECT `+executorStatusColumns+` FROM library_executor_status WHERE instance=$1`, instance)
var observation libraryexecutor.Observation
var raw []byte
err := row.Scan(&observation.Instance, &observation.Reachable, &observation.Busy, &observation.Degraded,
&raw, &observation.Error, &observation.ObservedAt)
if errors.Is(err, pgx.ErrNoRows) {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("read library executor status: %w", err)
}
if len(raw) > 0 {
if err := json.Unmarshal(raw, &observation.Identity); err != nil {
return nil, fmt.Errorf("library executor identity unreadable: %w", err)
}
}
return &observation, nil
}
@@ -0,0 +1,70 @@
package postgres
import (
"context"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"knowledge-governance/internal/libraryexecutor"
)
// 执行器状态是"最新一次观测",不是"最后一条好消息":
// 降级与不可达必须覆盖上一次的健康记录,否则页面会一直显示可发起。
func TestLibraryExecutorStatusKeepsTheLatestObservation(t *testing.T) {
databaseURL := isolatedIntegrationDatabaseURL(t, os.Getenv("TEST_DATABASE_URL"))
if databaseURL == "" {
t.Skip("TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, databaseURL)
if err != nil {
t.Fatalf("connect: %v", err)
}
defer pool.Close()
if err := ApplyMigrations(ctx, pool); err != nil {
t.Fatalf("migrate: %v", err)
}
store := NewStore(pool)
now := time.Date(2026, 10, 29, 8, 0, 0, 0, time.UTC)
if missing, err := store.LibraryExecutorStatus(ctx, "kgmaint-weknora"); err != nil || missing != nil {
t.Fatalf("missing row = %+v, %v", missing, err)
}
healthy := libraryexecutor.Observation{
Instance: "kgmaint-weknora", Reachable: true, ObservedAt: now,
Identity: libraryexecutor.Identity{Project: "kgmaint-weknora", SourceApp: "kgmaint-app", ArchiveRoot: "/tmp/kg-e2e/archive"},
}
if err := store.UpsertLibraryExecutorStatus(ctx, healthy, "worker-1", now); err != nil {
t.Fatalf("upsert healthy: %v", err)
}
read, err := store.LibraryExecutorStatus(ctx, "kgmaint-weknora")
if err != nil {
t.Fatalf("read healthy: %v", err)
}
if read == nil || !read.Reachable || read.Identity.Project != "kgmaint-weknora" || read.Identity.SourceApp != "kgmaint-app" {
t.Fatalf("healthy row = %+v", read)
}
if !read.ObservedAt.Equal(now) {
t.Fatalf("observed_at = %v, want %v", read.ObservedAt, now)
}
degraded := healthy
degraded.Reachable = false
degraded.ObservedAt = now.Add(30 * time.Second)
degraded.Degraded = ""
degraded.Error = "library agent is unreachable"
if err := store.UpsertLibraryExecutorStatus(ctx, degraded, "worker-1", degraded.ObservedAt); err != nil {
t.Fatalf("upsert unreachable: %v", err)
}
read, err = store.LibraryExecutorStatus(ctx, "kgmaint-weknora")
if err != nil || read == nil {
t.Fatalf("read after unreachable: %+v %v", read, err)
}
if read.Reachable || read.Error != "library agent is unreachable" || !read.ObservedAt.Equal(degraded.ObservedAt) {
t.Fatalf("row kept a stale good state: %+v", read)
}
}
@@ -65,7 +65,15 @@ ON CONFLICT (instance) DO UPDATE SET
services = EXCLUDED.services, services = EXCLUDED.services,
ingress_state = EXCLUDED.ingress_state, ingress_state = EXCLUDED.ingress_state,
state = EXCLUDED.state, state = EXCLUDED.state,
original_service_state = EXCLUDED.original_service_state, -- 同一作业重入(崩溃接管)时不许用"现在已停"覆盖第一次记下的原状态:
-- 重入时服务已经被我们停住,再读一次只会读到 stopped,写回去会让恢复阶段
-- 以为写入者本来就没跑,于是永远不启动它。
original_service_state = CASE
WHEN library_instance_maintenance.owner_job_id = EXCLUDED.owner_job_id
AND library_instance_maintenance.original_service_state <> ''
THEN library_instance_maintenance.original_service_state
ELSE EXCLUDED.original_service_state
END,
note = EXCLUDED.note, note = EXCLUDED.note,
updated_at = EXCLUDED.updated_at, updated_at = EXCLUDED.updated_at,
released_at = NULL released_at = NULL
@@ -92,6 +100,9 @@ WHERE instance=$1 AND owner_job_id=$2 AND released_at IS NULL`, instance, ownerJ
return fmt.Errorf("update instance maintenance: %w", err) return fmt.Errorf("update instance maintenance: %w", err)
} }
if tag.RowsAffected() == 0 { if tag.RowsAffected() == 0 {
if s.releasedByThisJob(ctx, "library_instance_maintenance", "instance", instance, ownerJobID) {
return ErrMaintenanceReleased
}
return errors.New("instance maintenance not held by this job") return errors.New("instance maintenance not held by this job")
} }
return nil return nil
@@ -108,6 +119,9 @@ WHERE instance=$1 AND owner_job_id=$2 AND released_at IS NULL`, instance, ownerJ
return fmt.Errorf("update instance maintenance ingress: %w", err) return fmt.Errorf("update instance maintenance ingress: %w", err)
} }
if tag.RowsAffected() == 0 { if tag.RowsAffected() == 0 {
if s.releasedByThisJob(ctx, "library_instance_maintenance", "instance", instance, ownerJobID) {
return ErrMaintenanceReleased
}
return errors.New("instance maintenance not held by this job") return errors.New("instance maintenance not held by this job")
} }
return nil return nil
@@ -121,6 +135,9 @@ WHERE instance=$1 AND owner_job_id=$2 AND released_at IS NULL`, instance, ownerJ
return fmt.Errorf("release instance maintenance: %w", err) return fmt.Errorf("release instance maintenance: %w", err)
} }
if tag.RowsAffected() == 0 { if tag.RowsAffected() == 0 {
if s.releasedByThisJob(ctx, "library_instance_maintenance", "instance", instance, ownerJobID) {
return ErrMaintenanceReleased
}
return errors.New("instance maintenance not held by this job") return errors.New("instance maintenance not held by this job")
} }
return nil return nil
@@ -0,0 +1,85 @@
package postgres
import (
"context"
"errors"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"knowledge-governance/internal/governance"
)
// 迁移 075 之后,"建立基线"必须与同步/恢复共用同一把写互斥:
// 同一 scope 只允许一个在飞写作业,且互斥由数据库唯一索引保证(不靠页面)。
// 这条用例跑在真实 PostgreSQL 上(TEST_DATABASE_URL),不是内存假实现。
func TestLibraryWriteMutexCoversBaselineJobs(t *testing.T) {
databaseURL := isolatedIntegrationDatabaseURL(t, os.Getenv("TEST_DATABASE_URL"))
if databaseURL == "" {
t.Skip("TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, databaseURL)
if err != nil {
t.Fatalf("connect: %v", err)
}
defer pool.Close()
if err := ApplyMigrations(ctx, pool); err != nil {
t.Fatalf("migrate: %v", err)
}
store := NewStore(pool)
now := time.Now().UTC()
digest, err := governance.LibraryRequestDigest("internal", governance.LibraryJobBaseline,
map[string]any{"scope": "internal"})
if err != nil {
t.Fatal(err)
}
baseline := governance.LibraryJob{
ID: "libbase-integration-1", Scope: "internal", Kind: governance.LibraryJobBaseline,
IdempotencyKey: "library.baseline:internal:1", RequestDigest: digest, Actor: "admin",
}
created, reused, err := store.CreateLibraryJob(ctx, baseline, governance.BaselineStages(), now)
if err != nil || reused {
t.Fatalf("first baseline job: err=%v reused=%v", err, reused)
}
if len(created.Stages) == 0 {
t.Fatalf("stages not created: %+v", created)
}
// 同一幂等键 + 同一请求 = 复用同一作业,不产生第二行。
same, reused, err := store.CreateLibraryJob(ctx, baseline, governance.BaselineStages(), now)
if err != nil || !reused || same.ID != baseline.ID {
t.Fatalf("idempotent retry: err=%v reused=%v id=%q", err, reused, same.ID)
}
// 另一个基线作业(不同幂等键)在同一 scope 上必须被互斥挡住。
second := baseline
second.ID = "libbase-integration-2"
second.IdempotencyKey = "library.baseline:internal:2"
if _, _, err := store.CreateLibraryJob(ctx, second, governance.BaselineStages(), now); !errors.Is(err, governance.ErrLibraryJobActive) {
t.Fatalf("second baseline during first: err=%v", err)
}
// 同步作业与基线共用同一把锁:cross-kind 也必须被挡。
syncDigest, err := governance.LibraryRequestDigest("internal", governance.LibraryJobSync,
map[string]any{"target_snapshot_id": "snap-1"})
if err != nil {
t.Fatal(err)
}
if _, _, err := store.CreateLibraryJob(ctx, governance.LibraryJob{
ID: "libsync-integration-1", Scope: "internal", Kind: governance.LibraryJobSync,
IdempotencyKey: "library.sync:internal:snap-1", RequestDigest: syncDigest, TargetSnapshotID: "snap-1",
}, governance.SyncStages(), now); !errors.Is(err, governance.ErrLibraryJobActive) {
t.Fatalf("sync during baseline: err=%v", err)
}
// 第一个作业失败后互斥必须释放,新作业可以进(否则第一次失败就永久堵死)。
if err := store.AdvanceLibraryJob(ctx, baseline.ID, governance.LibraryJobFailed, "capture", "test", "", "", "", now.Add(time.Second)); err != nil {
t.Fatalf("advance: %v", err)
}
if _, _, err := store.CreateLibraryJob(ctx, second, governance.BaselineStages(), now.Add(2*time.Second)); err != nil {
t.Fatalf("baseline after failure: %v", err)
}
}
+1 -1
View File
@@ -261,7 +261,7 @@ func (s *Store) InterruptedLibraryJobs(ctx context.Context) ([]governance.Librar
rows, err := s.db.Query(ctx, ` rows, err := s.db.Query(ctx, `
SELECT lj.id FROM library_jobs lj SELECT lj.id FROM library_jobs lj
LEFT JOIN jobs j ON j.id=lj.queue_job_id LEFT JOIN jobs j ON j.id=lj.queue_job_id
WHERE lj.kind IN ('sync','restore') WHERE lj.kind IN ('sync','restore','baseline')
AND ( AND (
-- 业务作业在跑、队列已经没了/停了/租约过期 -- 业务作业在跑、队列已经没了/停了/租约过期
(lj.state='running' AND (j.id IS NULL OR j.state IN ('failed','completed','cancelled','dead') (lj.state='running' AND (j.id IS NULL OR j.state IN ('failed','completed','cancelled','dead')
@@ -10,6 +10,19 @@ import (
"github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5"
) )
var ErrMaintenanceReleased = errors.New("maintenance already released by this job")
// releasedByThisJob 判断"零行更新"是因为本作业已经释放过,还是根本没持有。
func (s *Store) releasedByThisJob(ctx context.Context, table, keyColumn, key, ownerJobID string) bool {
var owner string
var released *time.Time
query := "SELECT owner_job_id, released_at FROM " + table + " WHERE " + keyColumn + "=$1"
if err := s.db.QueryRow(ctx, query, key).Scan(&owner, &released); err != nil {
return false
}
return owner == ownerJobID && released != nil
}
// LibraryMaintenance 是目标维护窗口的持有记录:一个 scope 只能有一个持有者。 // LibraryMaintenance 是目标维护窗口的持有记录:一个 scope 只能有一个持有者。
// //
// Level 说明这次维护到底动了什么: // Level 说明这次维护到底动了什么:
@@ -68,7 +81,13 @@ ON CONFLICT (scope) DO UPDATE SET
services = EXCLUDED.services, services = EXCLUDED.services,
ingress_state = EXCLUDED.ingress_state, ingress_state = EXCLUDED.ingress_state,
state = EXCLUDED.state, state = EXCLUDED.state,
original_service_state = EXCLUDED.original_service_state, -- 同上:原运行状态只由第一次进入窗口时决定,重入不能覆盖。
original_service_state = CASE
WHEN library_maintenance.owner_job_id = EXCLUDED.owner_job_id
AND library_maintenance.original_service_state <> ''
THEN library_maintenance.original_service_state
ELSE EXCLUDED.original_service_state
END,
note = EXCLUDED.note, note = EXCLUDED.note,
updated_at = EXCLUDED.updated_at, updated_at = EXCLUDED.updated_at,
released_at = NULL released_at = NULL
@@ -97,6 +116,9 @@ WHERE scope=$1 AND owner_job_id=$2 AND released_at IS NULL`, scope, ownerJobID,
return fmt.Errorf("update library maintenance: %w", err) return fmt.Errorf("update library maintenance: %w", err)
} }
if tag.RowsAffected() == 0 { if tag.RowsAffected() == 0 {
if s.releasedByThisJob(ctx, "library_maintenance", "scope", scope, ownerJobID) {
return ErrMaintenanceReleased
}
return errors.New("library maintenance not held by this job") return errors.New("library maintenance not held by this job")
} }
return nil return nil
@@ -114,6 +136,9 @@ WHERE scope=$1 AND owner_job_id=$2 AND released_at IS NULL`, scope, ownerJobID,
return fmt.Errorf("update library maintenance ingress: %w", err) return fmt.Errorf("update library maintenance ingress: %w", err)
} }
if tag.RowsAffected() == 0 { if tag.RowsAffected() == 0 {
if s.releasedByThisJob(ctx, "library_maintenance", "scope", scope, ownerJobID) {
return ErrMaintenanceReleased
}
return errors.New("library maintenance not held by this job") return errors.New("library maintenance not held by this job")
} }
return nil return nil
@@ -127,6 +152,9 @@ WHERE scope=$1 AND owner_job_id=$2 AND released_at IS NULL`, scope, ownerJobID,
return fmt.Errorf("release library maintenance: %w", err) return fmt.Errorf("release library maintenance: %w", err)
} }
if tag.RowsAffected() == 0 { if tag.RowsAffected() == 0 {
if s.releasedByThisJob(ctx, "library_maintenance", "scope", scope, ownerJobID) {
return ErrMaintenanceReleased
}
return errors.New("library maintenance not held by this job") return errors.New("library maintenance not held by this job")
} }
return nil return nil
@@ -72,6 +72,86 @@ func TestLibraryMaintenanceOwnershipAcrossRelease(t *testing.T) {
} }
} }
// 同一作业重入(崩溃后的接管)必须保留第一次记下的"原运行状态":
// 重入时目标已经被停住,此时再去读一次只会读到"已停",把它写回去会让恢复阶段
// 以为写入者本来就没跑,于是永远不启动它 —— 目标就这么一直停着。
func TestMaintenanceReentryKeepsOriginalServiceState(t *testing.T) {
databaseURL := isolatedIntegrationDatabaseURL(t, os.Getenv("TEST_DATABASE_URL"))
if databaseURL == "" {
t.Skip("TEST_DATABASE_URL is not set")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, databaseURL)
if err != nil {
t.Fatalf("connect: %v", err)
}
defer pool.Close()
if err := ApplyMigrations(ctx, pool); err != nil {
t.Fatalf("migrate: %v", err)
}
store := NewStore(pool)
now := time.Now().UTC()
first, err := store.AcquireLibraryMaintenance(ctx, LibraryMaintenance{
Scope: "internal", OwnerJobID: "job-reentry", TargetIdentity: "instance=kgseal",
Level: "instance", TargetFingerprint: "fp", Services: []string{"app"},
State: "acquired", OriginalServiceState: "app=running", Note: "first",
}, now)
if err != nil {
t.Fatalf("first acquire: %v", err)
}
if first.OriginalServiceState != "app=running" {
t.Fatalf("first original state = %q", first.OriginalServiceState)
}
reentered, err := store.AcquireLibraryMaintenance(ctx, LibraryMaintenance{
Scope: "internal", OwnerJobID: "job-reentry", Level: "instance",
State: "applying", OriginalServiceState: "app=stopped", Note: "re-entry after crash",
}, now.Add(time.Second))
if err != nil {
t.Fatalf("same-owner re-entry: %v", err)
}
if reentered.OriginalServiceState != "app=running" {
t.Fatalf("re-entry overwrote the original state: got %q, want app=running", reentered.OriginalServiceState)
}
instFirst, err := store.AcquireInstanceMaintenance(ctx, LibraryInstanceMaintenance{
Instance: "kgseal-instance-reentry", OwnerJobID: "job-reentry", OwnerScope: "internal",
TargetFingerprint: "fp", Level: "instance", Services: []string{"app"},
State: "acquired", OriginalServiceState: "app=running", Note: "first",
}, now)
if err != nil {
t.Fatalf("instance acquire: %v", err)
}
if instFirst.OriginalServiceState != "app=running" {
t.Fatalf("instance first original state = %q", instFirst.OriginalServiceState)
}
instReentered, err := store.AcquireInstanceMaintenance(ctx, LibraryInstanceMaintenance{
Instance: "kgseal-instance-reentry", OwnerJobID: "job-reentry", OwnerScope: "internal",
Level: "instance", State: "applying", OriginalServiceState: "app=stopped",
}, now.Add(2*time.Second))
if err != nil {
t.Fatalf("instance same-owner re-entry: %v", err)
}
if instReentered.OriginalServiceState != "app=running" {
t.Fatalf("instance re-entry overwrote the original state: got %q", instReentered.OriginalServiceState)
}
// 新作业接管已释放的窗口时,必须接受它自己读到的当前状态(不能沿用上一个作业的)。
if err := store.ReleaseInstanceMaintenance(ctx, "kgseal-instance-reentry", "job-reentry", now.Add(3*time.Second)); err != nil {
t.Fatalf("release instance: %v", err)
}
next, err := store.AcquireInstanceMaintenance(ctx, LibraryInstanceMaintenance{
Instance: "kgseal-instance-reentry", OwnerJobID: "job-next", OwnerScope: "internal",
Level: "instance", State: "acquired", OriginalServiceState: "app=stopped",
}, now.Add(4*time.Second))
if err != nil {
t.Fatalf("next owner acquire: %v", err)
}
if next.OriginalServiceState != "app=stopped" {
t.Fatalf("new owner must record its own observation: got %q", next.OriginalServiceState)
}
}
// 实例级占用必须跨 scope 互斥:internal 持有期间 external 也进不来。 // 实例级占用必须跨 scope 互斥:internal 持有期间 external 也进不来。
// (旧实现按 scope 建行,两个 scope 各持一个锁,会同时停/起同一个实例。) // (旧实现按 scope 建行,两个 scope 各持一个锁,会同时停/起同一个实例。)
func TestInstanceMaintenanceExcludesAcrossScopes(t *testing.T) { func TestInstanceMaintenanceExcludesAcrossScopes(t *testing.T) {
@@ -0,0 +1 @@
DROP TABLE IF EXISTS library_executor_status;
@@ -0,0 +1,16 @@
-- 封存执行器的运行时状态:页面上的"可以发起封存"必须来自真实观测,而不是
-- "配置里写了 socket 路径"。正式栈曾出现过 agent 顶着正式单元名却在冻结候选实例:
-- 配置非空、页面显示可发起、真发起却封错了库。
--
-- worker 探测 agent 的 health(不跑子进程、不碰源库),把"它指向哪个实例、
-- 是否降级/正在跑、什么时候观测的"落库;API 只读这张表并核对身份,不再猜。
CREATE TABLE IF NOT EXISTS library_executor_status (
instance VARCHAR(128) PRIMARY KEY,
reachable BOOLEAN NOT NULL DEFAULT FALSE,
busy BOOLEAN NOT NULL DEFAULT FALSE,
degraded TEXT NOT NULL DEFAULT '',
identity JSONB NOT NULL DEFAULT '{}'::jsonb,
status_error TEXT NOT NULL DEFAULT '',
observer VARCHAR(64) NOT NULL DEFAULT '',
observed_at TIMESTAMPTZ NOT NULL
);
@@ -0,0 +1,8 @@
DELETE FROM library_jobs WHERE kind='baseline';
ALTER TABLE library_jobs DROP CONSTRAINT IF EXISTS library_jobs_kind_check;
ALTER TABLE library_jobs ADD CONSTRAINT library_jobs_kind_check
CHECK (kind IN ('seal','sync','restore'));
DROP INDEX IF EXISTS library_jobs_write_target;
CREATE UNIQUE INDEX library_jobs_write_target
ON library_jobs(scope)
WHERE state IN ('pending','running') AND kind IN ('sync','restore');
@@ -0,0 +1,12 @@
-- 允许"建立目标权威基线"作为一种独立业务作业:首次发布的目标非空又没有生效版本时,
-- 只能通过真实捕获 + 校验登记基线,不能手写版本记录。
-- 基线会在目标上开维护窗口(停写入者),所以它与 sync/restore 共用同一把写互斥:
-- 同一 scope 不允许同时存在两种写作业。
ALTER TABLE library_jobs DROP CONSTRAINT IF EXISTS library_jobs_kind_check;
ALTER TABLE library_jobs ADD CONSTRAINT library_jobs_kind_check
CHECK (kind IN ('seal','sync','restore','baseline'));
DROP INDEX IF EXISTS library_jobs_write_target;
CREATE UNIQUE INDEX library_jobs_write_target
ON library_jobs(scope)
WHERE state IN ('pending','running') AND kind IN ('sync','restore','baseline');
+21 -10
View File
@@ -11,40 +11,51 @@ func TestEmbeddedMigrationsAreVersionSorted(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("loadMigrations() error = %v", err) t.Fatalf("loadMigrations() error = %v", err)
} }
if len(migrations) != 73 { if len(migrations) != 75 {
t.Fatalf("migration count = %d, want 73", len(migrations)) t.Fatalf("migration count = %d, want 75", len(migrations))
}
// 75 允许"建立目标权威基线"作为独立业务作业,并与 sync/restore 共用写互斥。
if baseline := migrations[len(migrations)-1]; baseline.Version != 75 ||
!strings.Contains(baseline.UpSQL, "library_jobs_write_target") ||
!strings.Contains(baseline.UpSQL, "'baseline'") {
t.Fatalf("migration 75 = %#v", baseline)
}
// 74 记录封存执行器的运行时状态:页面可用性来自真实观测,不来自配置字符串。
if executor := migrations[len(migrations)-2]; executor.Version != 74 ||
!strings.Contains(executor.UpSQL, "library_executor_status") {
t.Fatalf("migration 74 = %#v", executor)
} }
// 73 记录死信精确解除:保留上游行,只绑定指纹、重放引用与观测摘要。 // 73 记录死信精确解除:保留上游行,只绑定指纹、重放引用与观测摘要。
if release := migrations[len(migrations)-1]; release.Version != 73 || if release := migrations[len(migrations)-3]; release.Version != 73 ||
!strings.Contains(release.UpSQL, "library_dead_letter_releases") { !strings.Contains(release.UpSQL, "library_dead_letter_releases") {
t.Fatalf("migration 73 = %#v", release) t.Fatalf("migration 73 = %#v", release)
} }
// 72 让维护占用按实例互斥(internal/external 不再各有各的锁)。 // 72 让维护占用按实例互斥(internal/external 不再各有各的锁)。
if instance := migrations[len(migrations)-2]; instance.Version != 72 || if instance := migrations[len(migrations)-4]; instance.Version != 72 ||
!strings.Contains(instance.UpSQL, "library_instance_maintenance") { !strings.Contains(instance.UpSQL, "library_instance_maintenance") {
t.Fatalf("migration 72 = %#v", instance) t.Fatalf("migration 72 = %#v", instance)
} }
// 71 让维护记录能区分实例级维护与仅治理库互斥。 // 71 让维护记录能区分实例级维护与仅治理库互斥。
if level := migrations[len(migrations)-3]; level.Version != 71 || if level := migrations[len(migrations)-5]; level.Version != 71 ||
!strings.Contains(level.UpSQL, "level") || !strings.Contains(level.UpSQL, "ingress_state") { !strings.Contains(level.UpSQL, "level") || !strings.Contains(level.UpSQL, "ingress_state") {
t.Fatalf("migration 71 = %#v", level) t.Fatalf("migration 71 = %#v", level)
} }
if maint := migrations[len(migrations)-4]; maint.Version != 70 || !strings.Contains(maint.UpSQL, "library_maintenance") { if maint := migrations[len(migrations)-6]; maint.Version != 70 || !strings.Contains(maint.UpSQL, "library_maintenance") {
t.Fatalf("migration 70 = %#v", maint) t.Fatalf("migration 70 = %#v", maint)
} }
if tail := migrations[len(migrations)-5]; tail.Version != 69 || if tail := migrations[len(migrations)-7]; tail.Version != 69 ||
!strings.Contains(tail.UpSQL, "library_jobs_write_target") { !strings.Contains(tail.UpSQL, "library_jobs_write_target") {
t.Fatalf("migration 69 = %#v", tail) t.Fatalf("migration 69 = %#v", tail)
} }
if restore := migrations[len(migrations)-6]; restore.Version != 68 || !strings.Contains(restore.UpSQL, "restore") { if restore := migrations[len(migrations)-8]; restore.Version != 68 || !strings.Contains(restore.UpSQL, "restore") {
t.Fatalf("migration 68 = %#v", restore) t.Fatalf("migration 68 = %#v", restore)
} }
if last := migrations[len(migrations)-7]; last.Version != 67 || if last := migrations[len(migrations)-9]; last.Version != 67 ||
!strings.Contains(last.UpSQL, "library_release_authorizations") || !strings.Contains(last.UpSQL, "library_release_authorizations") ||
!strings.Contains(last.UpSQL, "params_digest") { !strings.Contains(last.UpSQL, "params_digest") {
t.Fatalf("migration 67 = %#v", last) t.Fatalf("migration 67 = %#v", last)
} }
if flow := migrations[len(migrations)-8]; flow.Version != 66 || if flow := migrations[len(migrations)-10]; flow.Version != 66 ||
!strings.Contains(flow.UpSQL, "CREATE TABLE library_versions") || !strings.Contains(flow.UpSQL, "CREATE TABLE library_versions") ||
!strings.Contains(flow.UpSQL, "library_jobs_active") { !strings.Contains(flow.UpSQL, "library_jobs_active") {
t.Fatalf("migration 66 = %#v", flow) t.Fatalf("migration 66 = %#v", flow)
@@ -5,6 +5,7 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"strings" "strings"
"time" "time"
@@ -86,7 +87,9 @@ func ArchiveLibraryCandidate(ctx context.Context, archive *librarysnapshot.Archi
} }
reader, err := objects.Open(ctx, resource.Provider, resource.PhysicalPath) reader, err := objects.Open(ctx, resource.Provider, resource.PhysicalPath)
if err != nil { if err != nil {
return "", errors.New("read library resource") // 只写"read library resource"会让整条封存停在无信息的失败上:带上行 id
// 与底层原因(不含凭据),才能判断是对象缺失、桶错还是端点解析不到。
return "", fmt.Errorf("read library resource %s: %w", resource.ID, err)
} }
part, readErr := archive.Put(reader) part, readErr := archive.Put(reader)
closeErr := reader.Close() closeErr := reader.Close()
@@ -0,0 +1,46 @@
package v071
import (
"context"
"errors"
"fmt"
"github.com/jackc/pgx/v5/pgxpool"
)
// ScopeContentCounts 数一个知识库在各内容表里的行数(数据库侧事实)。
//
// 为什么不能只数 knowledges:文档被删掉、分块与 Wiki/资源还在的库并不为空。
// 只数一张表会把这种目标判成"首次发布的空库",从而让一条本该走基线初始化的
// 路径绕过去;这里用与整库读写完全相同的表清单与过滤条件,保证"空"的定义只有一份。
func ScopeContentCounts(ctx context.Context, pool *pgxpool.Pool, identity ProductionScope) (map[string]int64, error) {
if pool == nil || identity.TenantID == 0 || !validSearchIdentifier(identity.KnowledgeBaseID) {
return nil, errors.New("invalid scope identity")
}
counts := make(map[string]int64, len(productionLibraryTables))
for _, table := range productionLibraryTables {
var count int64
if err := pool.QueryRow(ctx, fmt.Sprintf(`SELECT count(*) FROM %s WHERE %s`, table, scopeWhere(table)),
identity.TenantID, identity.KnowledgeBaseID).Scan(&count); err != nil {
return nil, fmt.Errorf("count %s: %w", table, err)
}
counts[table] = count
}
return counts, nil
}
// ScopeContentEmpty 只有每张内容表都是 0 才算空目标,并交回逐表行数作为证据。
// 注意它**不**包含检索索引:索引由分块派生,分块为空而索引非空属于不一致,
// 应在执行侧(预备份/基线捕获)报出来,不能在这里被当成"空"。
func ScopeContentEmpty(ctx context.Context, pool *pgxpool.Pool, identity ProductionScope) (bool, map[string]int64, error) {
counts, err := ScopeContentCounts(ctx, pool, identity)
if err != nil {
return false, nil, err
}
for _, count := range counts {
if count != 0 {
return false, counts, nil
}
}
return true, counts, nil
}
+163 -10
View File
@@ -92,7 +92,7 @@ const overview: LibraryOverview = {
reasons: [ reasons: [
'生产写入闸口关闭(PRODUCTION_RELEASE_REQUESTS_ENABLED=false)', '生产写入闸口关闭(PRODUCTION_RELEASE_REQUESTS_ENABLED=false)',
'没有已核验的生产恢复演练证据(PRODUCTION_BACKUP_RESTORE_VERIFIED=false)', '没有已核验的生产恢复演练证据(PRODUCTION_BACKUP_RESTORE_VERIFIED=false)',
'生产当前权威版本未知,不能启动写入', '目标非空且没有生效版本记录:先建立目标权威基线(维护隔离下捕获整库并逐类校验),或确认目标确实是空库',
'本次目标快照还没有生产预备份', '本次目标快照还没有生产预备份',
], ],
}, },
@@ -149,6 +149,15 @@ const base: ViewProps = {
pendingCapture: false, pendingCapture: false,
canCapture: true, canCapture: true,
canWrite: true, canWrite: true,
canRestore: false,
sync: { canSign: false, pendingGrant: false, onGrant: vi.fn(), canStart: false, pending: false, onStart: vi.fn() },
baseline: { canStart: false, pending: false, onStart: vi.fn() },
restore: null,
restoreNotice: null,
pendingRestore: false,
pendingGrant: false,
onGrantRestore: vi.fn(),
onRestore: vi.fn(),
maintenance: null, maintenance: null,
now: new Date('2026-10-28T03:35:00Z').getTime(), now: new Date('2026-10-28T03:35:00Z').getTime(),
onRefresh: vi.fn(), onRefresh: vi.fn(),
@@ -166,7 +175,7 @@ describe('ReleaseTargetsView(确认稿布局)', () => {
expect(html).toContain('对外知识库') expect(html).toContain('对外知识库')
expect(html).toContain('role="tab"') expect(html).toContain('role="tab"')
// 只显示一个主作业卡:选中库标题出现一次,另一个库不渲染作业卡 // 只显示一个主作业卡:选中库标题出现一次,另一个库不渲染作业卡
expect(html.match(/· 已封存,生产同步未接入/g)?.length).toBe(1) expect(html.match(/· 已封存,同步按条件放行/g)?.length).toBe(1)
expect(html).not.toContain('对外知识库 · 尚无') expect(html).not.toContain('对外知识库 · 尚无')
}) })
@@ -239,18 +248,62 @@ describe('ReleaseTargetsView(确认稿布局)', () => {
expect(html).toContain('列表只校验清单自身,字节级复核在同步/恢复执行前进行') expect(html).toContain('列表只校验清单自身,字节级复核在同步/恢复执行前进行')
}) })
it('版本与快照抽屉按 scope 列出全部历史并区分测试封存/不可直接恢复', () => { it('版本与快照抽屉按 scope 列出全部历史,并把恢复条件交给接口判定', () => {
const html = render({ drawerOpen: true, expandedSnapshot: internalSnapshot.snapshot_id }) const html = render({ drawerOpen: true, expandedSnapshot: internalSnapshot.snapshot_id, canRestore: true })
expect(html).toContain('版本与快照') expect(html).toContain('版本与快照')
expect(html).toContain('封存 f992d64ea45b') expect(html).toContain('封存 f992d64ea45b')
expect(html).toContain('测试封存') expect(html).toContain('测试封存')
expect(html).toContain('不可直接恢复') expect(html).toContain('恢复条件见下方')
expect(html).toContain('恢复至该版本') expect(html).toContain('恢复至该版本')
expect(html).toMatch(/恢复至该版本[^<]*<\/button>/) expect(html).toMatch(/恢复至该版本[^<]*<\/button>/)
expect(html).toContain('分项数量 / 字节 / 摘要(2 项)') expect(html).toContain('分项数量 / 字节 / 摘要(2 项)')
expect(html).toContain('不支持只选择 Wiki 或某张数据表恢复') expect(html).toContain('不支持只选择 Wiki 或某张数据表恢复')
}) })
it('管理员在缺授权时看到签发入口,恢复按钮按接口给出的原因禁用', () => {
const html = render({
drawerOpen: true,
expandedSnapshot: internalSnapshot.snapshot_id,
canRestore: true,
restore: {
allowed: false,
reasons: ['没有针对该快照的恢复授权:先签发 restore 授权(绑定同一快照)'],
scope: 'internal',
snapshot_id: internalSnapshot.snapshot_id,
authorization: { state: 'none' },
},
})
expect(html).toContain('签发恢复授权(30 分钟)')
expect(html).toContain('没有针对该快照的恢复授权')
expect(html).toMatch(/rel-danger[^>]*disabled/)
// 不能再说"未接线":条件不满足要给出具体原因
expect(html).not.toContain('生产恢复未接线')
})
it('条件全满足时恢复按钮可用,并写明授权有效期', () => {
const html = render({
drawerOpen: true,
expandedSnapshot: internalSnapshot.snapshot_id,
canRestore: true,
restore: {
allowed: true,
reasons: [],
scope: 'internal',
snapshot_id: internalSnapshot.snapshot_id,
authorization: { state: 'active', authorized_by: 'yaozhanghao', expires_at: '2026-10-28T04:10:00Z' },
},
})
expect(html).toContain('条件已满足')
expect(html).toContain('有效至')
expect(html).not.toMatch(/rel-danger[^>]*disabled/)
})
it('工程师看不到恢复入口,只看到权限说明', () => {
const html = render({ drawerOpen: true, expandedSnapshot: internalSnapshot.snapshot_id, canRestore: false })
expect(html).toContain('按版本恢复仅管理员可发起')
expect(html).not.toContain('签发恢复授权')
})
it('抽屉只展示当前选中库的历史', () => { it('抽屉只展示当前选中库的历史', () => {
const html = render({ drawerOpen: true, scope: 'external' }) const html = render({ drawerOpen: true, scope: 'external' })
expect(html).toContain('封存 01343b0b310b') expect(html).toContain('封存 01343b0b310b')
@@ -277,19 +330,119 @@ describe('ReleaseTargetsView(确认稿布局)', () => {
expect(html).toContain('主机侧正在暂停写入者、导出并恢复') expect(html).toContain('主机侧正在暂停写入者、导出并恢复')
}) })
it('无权限时不给封存入口,未接线时说明通道状态', () => { it('无权限时不给封存入口;不可发起时给出接口返回的真实原因与执行器身份', () => {
expect(render({ canCapture: false, canWrite: false })).toContain('封存需要知识工程师及以上权限') expect(render({ canCapture: false, canWrite: false })).toContain('封存需要知识工程师及以上权限')
const unwired = render({ const unavailable = render({
canCapture: false, canCapture: false,
canWrite: true, canWrite: true,
page: { ...page, capture_available: false, capture_reason: '主机侧封存 agent 未接线' }, page: { ...page, capture_available: false },
overview: {
...overview,
capture_available: false,
capture_reason: '执行器指向的实例不匹配:agent="weknora071full",期望="kgmaint-weknora"',
executor: {
available: false,
busy: false,
project: 'weknora071full',
source_app: 'WKNFull-app',
observed_at: '2026-10-28T03:34:00Z',
},
},
}) })
expect(unwired).toContain('封存通道未接线') expect(unavailable).toContain('执行器指向的实例不匹配')
expect(unavailable).toContain('执行器 weknora071full / WKNFull-app')
expect(unavailable).toContain('观测于')
// 通道可用/不可用都不许用写死的"未接线"糊过去
expect(unavailable).not.toContain('封存通道未接线')
}) })
it('无封存时给出“尚无封存”而不是空表', () => { it('无封存时给出“尚无封存”而不是空表', () => {
const html = render({ page: { ...page, snapshots: [] } }) // 归档没有快照时接口也不会给 test_sealed:两处必须一致,否则页面会把
// 一个不存在的封存显示成"已封存"。
const html = render({
page: { ...page, snapshots: [] },
overview: { ...overview, test_sealed: undefined, test_sealed_reason: '还没有该目标的测试封存' },
})
expect(html).toContain('尚无测试封存') expect(html).toContain('尚无测试封存')
expect(html).toContain('还没有测试库封存') expect(html).toContain('还没有测试库封存')
}) })
it('管理员在缺同步授权时看到签发入口,工程师只看到“需管理员签发”', () => {
const admin = render({
sync: { canSign: true, pendingGrant: false, onGrant: vi.fn(), canStart: false, pending: false, onStart: vi.fn() },
})
expect(admin).toContain('签发同步授权(30 分钟)')
// 授权入口不能顺手把执行也放行:闸口条件仍然逐条可见
expect(admin).toContain('生产写入闸口关闭')
const engineer = render()
expect(engineer).not.toContain('签发同步授权')
expect(engineer).toContain('同步授权只能由管理员按目标+快照签发')
})
it('目标非空又没有权威版本时,管理员看到建立基线入口;有生效版本时不显示', () => {
const admin = render({
baseline: { canStart: true, pending: false, onStart: vi.fn() },
})
expect(admin).toContain('建立目标权威基线(维护隔离)')
expect(admin).toContain('目标非空且没有生效版本记录')
const known = render({
baseline: { canStart: true, pending: false, onStart: vi.fn() },
overview: {
...overview,
production: {
state: 'known',
version: {
id: 'v1',
kind: 'production_effective',
snapshot_id: 'a'.repeat(64),
version_label: '基线 abc',
source_job_id: 'libbase-1',
evidence: {},
created_by: 'admin',
created_at: '2026-10-29T09:00:00Z',
},
},
},
})
// 有生效版本时不再提供基线入口(缺项说明文字里仍会提到它存在,所以只断言按钮)
expect(known).not.toMatch(/建立目标权威基线(维护隔离)<\/button>/)
})
it('闸口全满足时“启动同步”真的可点,并给出执行说明', () => {
const html = render({
sync: { canSign: false, pendingGrant: false, onGrant: vi.fn(), canStart: true, pending: false, onStart: vi.fn() },
overview: { ...overview, gate: { sync_allowed: true, reasons: [] } },
})
expect(html).toMatch(/<button[^>]*class="rel-primary"[^>]*>▶ 启动同步/)
expect(html).not.toMatch(/class="rel-primary"[^>]*disabled/)
expect(html).toContain('先固定测试版本,再备份目标并写入')
})
it('维护卡片区分“未运行维护”与“没有维护能力”', () => {
const wired = render({
maintenance: {
held: false,
capability: {
configured: true,
missing: [],
instance: 'kgseal-app',
project: 'kgseal-weknora',
writers: ['app'],
ingress_service: 'ingress',
},
},
})
expect(wired).toContain('未运行维护')
expect(wired).toContain('维护能力已接线:实例 kgseal-app · 写入者 app')
expect(wired).not.toContain('也没有维护能力配置')
const broken = render({
maintenance: { held: false, capability: { configured: false, missing: ['LIBRARY_TARGET_WRITERS'] } },
})
expect(broken).toContain('能力未接线')
expect(broken).toContain('维护能力未接线:缺少 LIBRARY_TARGET_WRITERS')
// 接口没给能力信息时不能猜成"没有能力"
const unknown = render({ maintenance: { held: false } })
expect(unknown).toContain('维护能力状态未知')
expect(unknown).not.toContain('也没有维护能力配置')
})
}) })
+330 -18
View File
@@ -1,10 +1,16 @@
import { useCallback, useEffect, useState } from 'react' import { useCallback, useEffect, useState } from 'react'
import { import {
createLibraryRestoreAuthorization,
createLibrarySyncAuthorization,
enqueueLibraryBaseline,
enqueueLibraryCapture, enqueueLibraryCapture,
enqueueLibraryRecovery, enqueueLibraryRecovery,
enqueueLibraryRestore,
enqueueLibrarySync,
fetchLibraryCaptures, fetchLibraryCaptures,
fetchLibraryMaintenance, fetchLibraryMaintenance,
fetchLibraryOverview, fetchLibraryOverview,
fetchLibraryRestoreAvailability,
fetchLibrarySnapshots, fetchLibrarySnapshots,
formatBytes, formatBytes,
scopeLabel, scopeLabel,
@@ -12,6 +18,7 @@ import {
type LibraryJobRecord, type LibraryJobRecord,
type LibraryMaintenanceView, type LibraryMaintenanceView,
type LibraryOverview, type LibraryOverview,
type LibraryRestoreAvailability,
type LibrarySnapshot, type LibrarySnapshot,
type LibrarySnapshotList, type LibrarySnapshotList,
type LibraryStageRow, type LibraryStageRow,
@@ -58,6 +65,24 @@ const jobStateLabel: Record<string, string> = {
type Scope = 'internal' | 'external' type Scope = 'internal' | 'external'
// 签发授权与执行是两步:页面只提供入口,不会替人签发,也不会自动重试执行。
export interface ReleaseSyncActions {
// 管理员才能签发;工程师看得到条件但没有按钮。
canSign: boolean
pendingGrant: boolean
onGrant: () => void
// 闸口全部满足才能启动(接口用同一份判定,按钮只是提前说明)。
canStart: boolean
pending: boolean
onStart: () => void
}
export interface ReleaseBaselineActions {
canStart: boolean
pending: boolean
onStart: () => void
}
interface ScopeView { interface ScopeView {
scope: Scope scope: Scope
snapshot?: LibrarySnapshot snapshot?: LibrarySnapshot
@@ -85,6 +110,18 @@ interface ViewProps {
pendingCapture: boolean pendingCapture: boolean
canCapture: boolean canCapture: boolean
canWrite: boolean canWrite: boolean
// 发布动作:签发授权(管理员)与启动同步是两个独立步骤,页面绝不自动签发。
sync: ReleaseSyncActions
// 目标非空又没有生效版本时,唯一入口是"建立目标权威基线"(管理员)。
baseline: ReleaseBaselineActions
// 按版本恢复是管理员动作:工程师看得到条件,但拿不到恢复入口。
canRestore: boolean
restore: LibraryRestoreAvailability | null
restoreNotice: string | null
pendingRestore: boolean
pendingGrant: boolean
onGrantRestore: () => void
onRestore: () => void
overview: LibraryOverview | null overview: LibraryOverview | null
maintenance: LibraryMaintenanceView | null maintenance: LibraryMaintenanceView | null
now: number now: number
@@ -106,7 +143,9 @@ function scopeView(
overview?.job ?? persisted.find((item) => item.state === 'pending' || item.state === 'running') ?? persisted[0] overview?.job ?? persisted.find((item) => item.state === 'pending' || item.state === 'running') ?? persisted[0]
return { return {
scope, scope,
snapshot: history[0], // 候选只认接口给的 test_sealed(来自版本登记):归档里还有生产预备份与
// 目标基线快照,按"最新一份"挑会把目标自己的内容显示成测试封存。
snapshot: overview?.test_sealed,
history, history,
activeJob: scoped.find((job) => ACTIVE_STATES.includes(job.state)), activeJob: scoped.find((job) => ACTIVE_STATES.includes(job.state)),
lastJob: scoped[0], lastJob: scoped[0],
@@ -118,6 +157,20 @@ function scopeView(
const scopeName = (scope: Scope) => (scope === 'internal' ? '对内知识库' : '对外知识库') const scopeName = (scope: Scope) => (scope === 'internal' ? '对内知识库' : '对外知识库')
// 不可发起时必须说清是哪一条不满足:读的是接口给出的真实原因(执行器不可达、
// 指向别的实例、降级、观测过期),不是一句写死的"通道未接线"。
function captureHint(view: ScopeView, canWrite: boolean): string {
if (!canWrite) return '封存需要知识工程师及以上权限'
const executor = view.overview?.executor
const reason = view.overview?.capture_reason || executor?.reason
const observed = executor?.observed_at ? `(观测于 ${stamp(executor.observed_at)})` : ''
const identity = executor?.project
? `执行器 ${executor.project}${executor.source_app ? ` / ${executor.source_app}` : ''} · `
: ''
if (reason) return `${identity}封存通道不可用:${reason}${observed}`
return '封存通道状态未知:还没有读到执行器观测'
}
function statusOf(view: ScopeView) { function statusOf(view: ScopeView) {
const job = view.job const job = view.job
if (job && (job.state === 'running' || job.state === 'pending')) { if (job && (job.state === 'running' || job.state === 'pending')) {
@@ -245,6 +298,14 @@ export function MaintenanceCard({
}) { }) {
const summary = maintenanceSummary(maintenance, job) const summary = maintenanceSummary(maintenance, job)
const record = maintenance?.instance_maintenance ?? maintenance?.maintenance const record = maintenance?.instance_maintenance ?? maintenance?.maintenance
// "没有窗口记录"不等于"没有维护能力":能力是配置事实,接口单独返回。
// 接口没给能力信息时也不能猜,明确写"未知"。
const capability = maintenance?.capability
const capabilityText = !capability
? '维护能力状态未知:接口没有返回能力信息,无法区分"未运行维护"与"能力未接线"'
: capability.configured
? `维护能力已接线:实例 ${capability.instance || '—'} · 写入者 ${(capability.writers ?? []).join('、') || '—'}`
: `维护能力未接线:缺少 ${(capability.missing ?? []).join('、') || '必要配置'}`
return ( return (
<div className="rel-panel rel-maintenance"> <div className="rel-panel rel-maintenance">
<header className="rel-panel-head"> <header className="rel-panel-head">
@@ -252,13 +313,26 @@ export function MaintenanceCard({
<h2>目标维护</h2> <h2>目标维护</h2>
<p>同步/恢复期间目标写入者被停住;级别与入口状态来自维护记录本身</p> <p>同步/恢复期间目标写入者被停住;级别与入口状态来自维护记录本身</p>
</div> </div>
<span className={`rel-pill ${summary.held ? 'blue' : summary.level === '未接入' ? 'gray' : 'green'}`}> <span
{summary.held ? `${summary.level} · 占用中` : summary.level === '未接入' ? '未接入' : '未占用'} className={`rel-pill ${summary.held ? 'blue' : record ? 'green' : capability?.configured ? 'gray' : 'red'}`}
>
{summary.held
? `${summary.level} · 占用中`
: record
? '未占用'
: capability?.configured
? '未运行维护'
: capability
? '能力未接线'
: '能力未知'}
</span> </span>
</header> </header>
{summary.level === '未接入' ? ( {!record ? (
<p className="rel-scope-note"> <p className="rel-scope-note">
暂无维护窗口记录,也没有维护能力配置;维护类操作保持 fail closed,不会在此状态下写目标。 暂无维护窗口记录。{capabilityText}
{capability?.configured
? ';窗口只在同步或建立基线时开启,页面不会单独开窗口。'
: ';维护类操作保持 fail closed,不会在此状态下写目标。'}
</p> </p>
) : ( ) : (
<> <>
@@ -508,15 +582,23 @@ function ProtectionCard({
prebackup, prebackup,
gate, gate,
authorization, authorization,
targetSealed,
sync,
baseline,
onOpen, onOpen,
}: { }: {
production: LibraryOverview['production'] production: LibraryOverview['production']
prebackup: LibraryOverview['prebackup'] prebackup: LibraryOverview['prebackup']
gate: LibraryOverview['gate'] gate: LibraryOverview['gate']
authorization: LibraryOverview['authorization'] authorization: LibraryOverview['authorization']
targetSealed: boolean
sync: ReleaseSyncActions
baseline: ReleaseBaselineActions
onOpen: () => void onOpen: () => void
}) { }) {
const backupKnown = prebackup.state === 'known' && prebackup.version const backupKnown = prebackup.state === 'known' && prebackup.version
// 基线入口只在"目标非空、却没有生效版本"时才是答案;空库走首次同步(预备份+登记)。
const baselineNeeded = production.state !== 'known' && targetSealed
return ( return (
<aside className="rel-panel rel-protection"> <aside className="rel-panel rel-protection">
<header className="rel-panel-head"> <header className="rel-panel-head">
@@ -554,6 +636,21 @@ function ProtectionCard({
))} ))}
</ul> </ul>
) : null} ) : null}
<div className="rel-drawer-actions">
{sync.canSign && authorization.state !== 'active' && targetSealed ? (
<button type="button" className="rel-secondary" onClick={sync.onGrant} disabled={sync.pendingGrant}>
{sync.pendingGrant ? '签发中…' : '签发同步授权(30 分钟)'}
</button>
) : null}
{baseline.canStart && baselineNeeded ? (
<button type="button" className="rel-secondary" onClick={baseline.onStart} disabled={baseline.pending}>
{baseline.pending ? '投递中…' : '建立目标权威基线(维护隔离)'}
</button>
) : null}
</div>
{authorization.state !== 'active' && !sync.canSign ? (
<p className="rel-hint">同步授权只能由管理员按目标+快照签发;工程师看不到签发入口。</p>
) : null}
<p className="rel-hint">同步启动后,这里显示本次生产恢复点及备份进度,可直接查看对应快照。</p> <p className="rel-hint">同步启动后,这里显示本次生产恢复点及备份进度,可直接查看对应快照。</p>
<button type="button" className="rel-secondary" onClick={onOpen}> <button type="button" className="rel-secondary" onClick={onOpen}>
查看已有版本与快照 › 查看已有版本与快照 ›
@@ -575,6 +672,8 @@ interface ReadyProps {
pendingCapture: boolean pendingCapture: boolean
onCapture: () => void onCapture: () => void
onOpen: () => void onOpen: () => void
sync: ReleaseSyncActions
baseline: ReleaseBaselineActions
} }
function ReadyBody({ function ReadyBody({
@@ -584,6 +683,8 @@ function ReadyBody({
prebackup, prebackup,
gate, gate,
authorization, authorization,
sync,
baseline,
canCapture, canCapture,
canWrite, canWrite,
pendingCapture, pendingCapture,
@@ -600,6 +701,9 @@ function ReadyBody({
prebackup={prebackup} prebackup={prebackup}
gate={gate} gate={gate}
authorization={authorization} authorization={authorization}
targetSealed={Boolean(snapshot)}
sync={sync}
baseline={baseline}
onOpen={onOpen} onOpen={onOpen}
/> />
</div> </div>
@@ -629,7 +733,7 @@ function ReadyBody({
{pendingCapture ? '投递中…' : '封存测试版本'} {pendingCapture ? '投递中…' : '封存测试版本'}
</button> </button>
) : ( ) : (
<span className="rel-hint">{canWrite ? '封存通道未接线' : '封存需要知识工程师及以上权限'}</span> <span className="rel-hint">{captureHint(view, canWrite)}</span>
)} )}
</div> </div>
</> </>
@@ -822,8 +926,16 @@ interface DrawerProps {
onExpand: (id: string | null) => void onExpand: (id: string | null) => void
onClose: () => void onClose: () => void
canCapture: boolean canCapture: boolean
canRestore: boolean
pendingCapture: boolean pendingCapture: boolean
onCapture: () => void onCapture: () => void
// 恢复条件来自容器(真实接口判定):双闸 / 该快照的授权 / 目标是否在维护。
restore: LibraryRestoreAvailability | null
restoreNotice: string | null
pendingRestore: boolean
pendingGrant: boolean
onGrantRestore: () => void
onRestore: () => void
} }
function Drawer({ function Drawer({
@@ -835,8 +947,15 @@ function Drawer({
onExpand, onExpand,
onClose, onClose,
canCapture, canCapture,
canRestore,
pendingCapture, pendingCapture,
onCapture, onCapture,
restore,
restoreNotice,
pendingRestore,
pendingGrant,
onGrantRestore,
onRestore,
}: DrawerProps) { }: DrawerProps) {
if (!open) return null if (!open) return null
const listing = view.history const listing = view.history
@@ -919,8 +1038,12 @@ function Drawer({
> >
{isSelected ? '收起快照' : '查看快照'} {isSelected ? '收起快照' : '查看快照'}
</button> </button>
<button type="button" className="rel-link" disabled title="生产恢复未接线,也没有恢复演练记录"> <button
不可直接恢复 type="button"
className="rel-link"
onClick={() => onExpand(isSelected ? null : snapshot.snapshot_id)}
>
{isSelected ? '恢复条件见下方' : '查看恢复条件'}
</button> </button>
</td> </td>
</tr> </tr>
@@ -991,13 +1114,48 @@ function Drawer({
以上是同一恢复点的配套内容,不支持只选择 Wiki 或某张数据表恢复。恢复速度需实测;“有备份文件”不等于可恢复。 以上是同一恢复点的配套内容,不支持只选择 Wiki 或某张数据表恢复。恢复速度需实测;“有备份文件”不等于可恢复。
</p> </p>
<div className="rel-drawer-actions"> <div className="rel-drawer-actions">
<button type="button" disabled title="生产恢复验证记录未接入"> <button type="button" disabled title="逐项校验证据在同步/恢复执行时产生,作业详情里可见">
查看校验记录 查看校验记录
</button> </button>
<button type="button" className="rel-danger" disabled title="生产恢复未接线,也没有经过恢复演练"> {canRestore && restore && restore.authorization.state !== 'active' ? (
恢复至该版本 <button type="button" className="rel-secondary" onClick={onGrantRestore} disabled={pendingGrant}>
</button> {pendingGrant ? '签发中…' : '签发恢复授权(30 分钟)'}
</button>
) : null}
{canRestore ? (
<button
type="button"
className="rel-danger"
onClick={onRestore}
disabled={pendingRestore || restore?.allowed !== true}
title={
restore?.allowed
? '把整套生产知识状态恢复到这个快照'
: (restore?.reasons ?? []).join(';') || '恢复条件尚未读取到'
}
>
{pendingRestore ? '投递中…' : '恢复至该版本'}
</button>
) : (
<span className="rel-hint">按版本恢复仅管理员可发起</span>
)}
</div> </div>
{restore && !restore.allowed ? (
<ul className="rel-gate" aria-label="恢复条件">
{restore.reasons.map((reason) => (
<li key={reason}>{reason}</li>
))}
</ul>
) : null}
{restore?.allowed ? (
<p className="rel-hint">
条件已满足:生产写入闸口打开、这份快照有恢复授权
{restore.authorization.expires_at ? `(有效至 ${stamp(restore.authorization.expires_at)})` : ''}
、目标实例未在维护。
</p>
) : null}
{restore === null ? <p className="rel-hint">正在读取恢复条件…</p> : null}
{restoreNotice ? <p className="rel-notice">{restoreNotice}</p> : null}
</section> </section>
) : null} ) : null}
@@ -1023,7 +1181,7 @@ function Drawer({
<p className="rel-hint"> <p className="rel-hint">
归档占用(只读):被版本引用的去重内容 {formatBytes(page?.content_bytes ?? 0)} · 磁盘分配{' '} 归档占用(只读):被版本引用的去重内容 {formatBytes(page?.content_bytes ?? 0)} · 磁盘分配{' '}
{formatBytes(page?.blob_allocated_bytes ?? 0)} · 未被引用 {page?.unreferenced_blobs ?? 0}{' '} {formatBytes(page?.blob_allocated_bytes ?? 0)} · 未被引用 {page?.unreferenced_blobs ?? 0}{' '}
个。保留策略与清理在系统设置 → 存储管理,均未接线。 个。清理与保留策略在系统设置 → 存储管理:当前可查看去重与保护范围,删除与保留策略尚未开放。
</p> </p>
</aside> </aside>
</div> </div>
@@ -1047,6 +1205,15 @@ export function ReleaseTargetsView(props: ViewProps) {
pendingCapture, pendingCapture,
canCapture, canCapture,
canWrite, canWrite,
sync,
baseline,
canRestore,
restore,
restoreNotice,
pendingRestore,
pendingGrant,
onGrantRestore,
onRestore,
now, now,
maintenance, maintenance,
onRefresh, onRefresh,
@@ -1125,7 +1292,7 @@ export function ReleaseTargetsView(props: ViewProps) {
<div className="rel-panel rel-jobhead"> <div className="rel-panel rel-jobhead">
<div className="rel-between"> <div className="rel-between">
<h2> <h2>
{scopeName(scope)} · {active ? '封存进行中' : view.snapshot ? '已封存,生产同步未接入' : '尚无测试封存'} {scopeName(scope)} · {active ? '封存进行中' : view.snapshot ? '已封存,同步按条件放行' : '尚无测试封存'}
</h2> </h2>
<div className="rel-head-actions"> <div className="rel-head-actions">
<button type="button" className="rel-secondary" onClick={() => onDrawer(true)}> <button type="button" className="rel-secondary" onClick={() => onDrawer(true)}>
@@ -1134,14 +1301,15 @@ export function ReleaseTargetsView(props: ViewProps) {
<button <button
type="button" type="button"
className="rel-primary" className="rel-primary"
disabled={!gate.sync_allowed} onClick={sync.onStart}
disabled={!sync.canStart || sync.pending}
title={ title={
gate.sync_allowed sync.canStart
? '生产同步执行链尚未接线,按钮不会启用' ? '按已签发的同步授权启动:先固定测试版本,再备份目标并写入'
: `暂不能启动同步:${gate.reasons.join(';')}` : `暂不能启动同步:${gate.reasons.join(';')}`
} }
> >
▶ 启动同步 {sync.pending ? '投递中…' : '▶ 启动同步'}
</button> </button>
</div> </div>
</div> </div>
@@ -1224,6 +1392,8 @@ export function ReleaseTargetsView(props: ViewProps) {
prebackup={prebackup} prebackup={prebackup}
gate={gate} gate={gate}
authorization={authorization} authorization={authorization}
sync={sync}
baseline={baseline}
canCapture={canCapture} canCapture={canCapture}
canWrite={canWrite} canWrite={canWrite}
pendingCapture={pendingCapture} pendingCapture={pendingCapture}
@@ -1241,8 +1411,15 @@ export function ReleaseTargetsView(props: ViewProps) {
onExpand={onExpandSnapshot} onExpand={onExpandSnapshot}
onClose={() => onDrawer(false)} onClose={() => onDrawer(false)}
canCapture={canCapture} canCapture={canCapture}
canRestore={canRestore}
pendingCapture={pendingCapture} pendingCapture={pendingCapture}
onCapture={onCapture} onCapture={onCapture}
restore={restore}
restoreNotice={restoreNotice}
pendingRestore={pendingRestore}
pendingGrant={pendingGrant}
onGrantRestore={onGrantRestore}
onRestore={onRestore}
/> />
</section> </section>
) )
@@ -1313,7 +1490,122 @@ export function ReleaseTargets() {
const role = auth?.identity?.role const role = auth?.identity?.role
const canWrite = role === 'engineer' || role === 'admin' const canWrite = role === 'engineer' || role === 'admin'
// 恢复写的是整套生产知识状态,接口只认管理员;页面必须与它一致。
const canRestore = role === 'admin'
// 签发同步授权同样是管理员动作:授权=钉死目标+快照+参数,不等于执行。
const canSignSync = role === 'admin'
const canCapture = canWrite && page?.capture_available === true const canCapture = canWrite && page?.capture_available === true
const [pendingSyncGrant, setPendingSyncGrant] = useState(false)
const [pendingSync, setPendingSync] = useState(false)
const [pendingBaseline, setPendingBaseline] = useState(false)
const current = overviews[scope]
const targetSnapshot = current?.test_sealed?.snapshot_id ?? ''
const grantSync = async () => {
if (!targetSnapshot) {
setNotice('还没有可发布的测试封存:先封存测试版本,再签发同步授权')
return
}
setPendingSyncGrant(true)
setNotice(null)
try {
const grant = await createLibrarySyncAuthorization(scope, targetSnapshot)
setNotice(
`已签发同步授权${grant.authorized_by ? `(${grant.authorized_by})` : ''},有效至 ${stamp(grant.expires_at)};` +
'执行仍需在下一次操作里点"启动同步"'
)
await load()
} catch (grantError) {
setNotice(grantError instanceof Error ? grantError.message : '签发同步授权失败')
} finally {
setPendingSyncGrant(false)
}
}
const startSync = async () => {
if (!targetSnapshot) {
setNotice('还没有可发布的测试封存')
return
}
setPendingSync(true)
setNotice(null)
try {
const accepted = await enqueueLibrarySync(scope, targetSnapshot)
setNotice(`已投递同步:${accepted.job_id}(先固定测试版本并生成预备份,再写目标)`)
await load()
} catch (syncError) {
setNotice(syncError instanceof Error ? syncError.message : '启动同步失败')
} finally {
setPendingSync(false)
}
}
const startBaseline = async () => {
setPendingBaseline(true)
setNotice(null)
try {
const accepted = await enqueueLibraryBaseline(scope)
setNotice(`已投递建立基线:${accepted.job_id}(维护隔离下捕获目标整库,逐类校验后登记为权威基线)`)
await load()
} catch (baselineError) {
setNotice(baselineError instanceof Error ? baselineError.message : '建立目标权威基线失败')
} finally {
setPendingBaseline(false)
}
}
// 恢复条件随"展开的那份快照"变化:闸口、该快照的授权、实例是否在维护都由接口判定。
const [restore, setRestore] = useState<LibraryRestoreAvailability | null>(null)
const [restoreNotice, setRestoreNotice] = useState<string | null>(null)
const [pendingRestore, setPendingRestore] = useState(false)
const [pendingGrant, setPendingGrant] = useState(false)
const [restoreRound, setRestoreRound] = useState(0)
const restoreSnapshot = expandedSnapshot ?? ''
useEffect(() => {
if (!drawerOpen || !restoreSnapshot) {
setRestore(null)
return
}
const controller = new AbortController()
void fetchLibraryRestoreAvailability(scope, restoreSnapshot, controller.signal)
.then((value) => setRestore(value))
.catch(() => setRestore(null))
return () => controller.abort()
}, [drawerOpen, restoreSnapshot, scope, restoreRound])
const grantRestore = async () => {
if (!restoreSnapshot) return
setPendingGrant(true)
setRestoreNotice(null)
try {
const grant = await createLibraryRestoreAuthorization(scope, restoreSnapshot)
setRestoreNotice(
`已签发恢复授权${grant.authorized_by ? `(${grant.authorized_by})` : ''},有效至 ${stamp(grant.expires_at)}`
)
setRestoreRound((round) => round + 1)
await load()
} catch (grantError) {
setRestoreNotice(grantError instanceof Error ? grantError.message : '签发恢复授权失败')
} finally {
setPendingGrant(false)
}
}
const startRestore = async () => {
if (!restoreSnapshot) return
setPendingRestore(true)
setRestoreNotice(null)
try {
const accepted = await enqueueLibraryRestore(scope, restoreSnapshot)
setRestoreNotice(`已投递恢复:${accepted.job_id}(整套生产知识状态回到该快照)`)
setRestoreRound((round) => round + 1)
await load()
} catch (restoreError) {
setRestoreNotice(restoreError instanceof Error ? restoreError.message : '发起恢复失败')
} finally {
setPendingRestore(false)
}
}
const startCapture = async () => { const startCapture = async () => {
setPendingCapture(true) setPendingCapture(true)
@@ -1364,6 +1656,26 @@ export function ReleaseTargets() {
pendingCapture={pendingCapture} pendingCapture={pendingCapture}
canCapture={canCapture} canCapture={canCapture}
canWrite={canWrite} canWrite={canWrite}
sync={{
canSign: canSignSync,
pendingGrant: pendingSyncGrant,
onGrant: () => void grantSync(),
canStart: current?.gate.sync_allowed === true,
pending: pendingSync,
onStart: () => void startSync(),
}}
baseline={{
canStart: canSignSync,
pending: pendingBaseline,
onStart: () => void startBaseline(),
}}
canRestore={canRestore}
restore={restore}
restoreNotice={restoreNotice}
pendingRestore={pendingRestore}
pendingGrant={pendingGrant}
onGrantRestore={() => void grantRestore()}
onRestore={() => void startRestore()}
maintenance={maintenances[scope]} maintenance={maintenances[scope]}
now={now} now={now}
onRefresh={() => void load()} onRefresh={() => void load()}
+87
View File
@@ -0,0 +1,87 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import {
createLibraryRestoreAuthorization,
enqueueLibraryRestore,
fetchLibraryRestoreAvailability,
} from './librarySnapshots'
const jsonResponse = (body: unknown, status = 200) =>
new Response(JSON.stringify(body), { status, headers: { 'Content-Type': 'application/json' } })
afterEach(() => {
vi.unstubAllGlobals()
})
describe('整库恢复的前端契约', () => {
it('读取恢复条件时把原因与授权状态原样带回来', async () => {
vi.stubGlobal(
'fetch',
vi.fn().mockResolvedValue(
jsonResponse({
allowed: false,
reasons: ['没有针对该快照的恢复授权:先签发 restore 授权(绑定同一快照)'],
scope: 'internal',
snapshot_id: 'snap-1',
authorization: { state: 'none', reason: '没有与目标快照匹配的有效发布授权' },
})
)
)
const availability = await fetchLibraryRestoreAvailability('internal', 'snap-1')
expect(availability?.allowed).toBe(false)
expect(availability?.reasons).toHaveLength(1)
expect(availability?.authorization.state).toBe('none')
const requested = (vi.mocked(fetch).mock.calls[0]?.[0] as string) ?? ''
expect(requested).toContain('scope=internal')
expect(requested).toContain('snapshot_id=snap-1')
})
it('签发恢复授权必须声明 action=restore 与有限有效期', async () => {
vi.stubGlobal(
'fetch',
vi.fn().mockResolvedValue(
jsonResponse(
{
authorization: {
id: 'libauth-restore-1',
authorized_by: 'yaozhanghao',
expires_at: '2026-10-29T09:00:00Z',
},
},
201
)
)
)
const grant = await createLibraryRestoreAuthorization('internal', 'snap-1')
expect(grant.id).toBe('libauth-restore-1')
const init = vi.mocked(fetch).mock.calls[0]?.[1] as RequestInit
const body = JSON.parse(String(init.body)) as Record<string, unknown>
expect(body.action).toBe('restore')
expect(body.expires_in_minutes).toBe(30)
})
it('恢复被拒时把逐条原因说给用户,而不是只说 HTTP 状态', async () => {
vi.stubGlobal(
'fetch',
vi
.fn()
.mockResolvedValue(
jsonResponse({ code: 'library_restore_blocked', reasons: ['生产写入闸口关闭', '没有该快照的恢复授权'] }, 503)
)
)
await expect(enqueueLibraryRestore('internal', 'snap-1')).rejects.toThrow('生产写入闸口关闭;没有该快照的恢复授权')
})
it('投递成功时返回作业号,供页面显示真实进度', async () => {
vi.stubGlobal(
'fetch',
vi
.fn()
.mockResolvedValue(
jsonResponse({ job: { id: 'librestore-internal-1', scope: 'internal' }, reused: false }, 202)
)
)
const accepted = await enqueueLibraryRestore('internal', 'snap-1')
expect(accepted.job_id).toBe('librestore-internal-1')
expect(accepted.reused).toBe(false)
})
})
+190
View File
@@ -185,6 +185,170 @@ export async function enqueueLibraryCapture(scope: string): Promise<LibraryCaptu
} }
} }
// 恢复能不能点,由真实条件决定:双闸、这份快照的恢复授权、目标实例是否在维护。
export interface LibraryRestoreAvailability {
allowed: boolean
reasons: string[]
scope: string
snapshot_id: string
authorization: LibraryAuthorizationView
}
export async function fetchLibraryRestoreAvailability(
scope: string,
snapshotID: string,
signal?: AbortSignal
): Promise<LibraryRestoreAvailability | null> {
const query = `scope=${encodeURIComponent(scope)}&snapshot_id=${encodeURIComponent(snapshotID)}`
const response = await fetch(`/api/v1/library/restore-availability?${query}`, {
headers: { Accept: 'application/json' },
signal,
})
const value = (await response.json().catch(() => null)) as unknown
if (!record(value)) return null
return {
allowed: bool(value.allowed),
reasons: Array.isArray(value.reasons) ? value.reasons.map(text).filter(Boolean) : [],
scope: text(value.scope),
snapshot_id: text(value.snapshot_id),
authorization: record(value.authorization)
? {
state: text(value.authorization.state) === 'active' ? 'active' : 'none',
authorized_by: text(value.authorization.authorized_by) || undefined,
expires_at: text(value.authorization.expires_at) || undefined,
reason: text(value.authorization.reason) || undefined,
}
: { state: 'none' },
}
}
// 恢复整套生产知识状态:同一份快照的文档、分块、索引、资源与 Wiki 一起回放。
export interface LibraryRestoreAccepted {
job_id: string
scope: string
reused: boolean
}
export async function enqueueLibraryRestore(scope: string, snapshotID: string): Promise<LibraryRestoreAccepted> {
const response = await fetch('/api/v1/library/restores', {
method: 'POST',
headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
body: JSON.stringify({ scope, snapshot_id: snapshotID }),
})
const value = (await response.json().catch(() => null)) as unknown
if (!response.ok) {
if (response.status === 403) throw new Error('按版本恢复仅管理员可发起。')
if (response.status === 503 && record(value)) {
const reasons = Array.isArray(value.reasons) ? value.reasons.map(text).filter(Boolean) : []
throw new Error(reasons.length > 0 ? reasons.join(';') : text(value.error) || '恢复条件不满足')
}
if (response.status === 409 && record(value)) throw new Error(text(value.error) || '该目标已有在飞作业')
throw new Error(`发起恢复失败(HTTP ${response.status})`)
}
if (!record(value) || !record(value.job)) throw new Error('恢复作业响应无效')
return { job_id: text(value.job.id), scope: text(value.job.scope), reused: bool(value.reused) }
}
export interface LibraryRestoreGrant {
id: string
authorized_by: string
expires_at: string
}
// 同步授权与执行是两步:授权把"范围 + 目标快照 + 参数摘要"钉死并给很短的有效期。
// 它不能顶替恢复授权(摘要不同),也不自动签发 —— 必须由管理员显式点击。
export async function createLibrarySyncAuthorization(
scope: string,
snapshotID: string,
minutes = 30
): Promise<LibraryRestoreGrant> {
const response = await fetch('/api/v1/library/authorizations', {
method: 'POST',
headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
body: JSON.stringify({ scope, snapshot_id: snapshotID, action: 'sync', expires_in_minutes: minutes }),
})
const value = (await response.json().catch(() => null)) as unknown
if (!response.ok) {
if (response.status === 403) throw new Error('生产发布授权仅管理员可签发。')
if (response.status === 400 && record(value)) throw new Error(text(value.error) || '授权参数不合法')
throw new Error(`签发同步授权失败(HTTP ${response.status})`)
}
if (!record(value) || !record(value.authorization)) throw new Error('授权响应无效')
return {
id: text(value.authorization.id),
authorized_by: text(value.authorization.authorized_by),
expires_at: text(value.authorization.expires_at),
}
}
export interface LibrarySyncAccepted {
job_id: string
scope: string
reused: boolean
}
// 启动同步:闸口未齐时接口返回 503 与逐条缺项,页面照实显示,不假装已启动。
export async function enqueueLibrarySync(scope: string, targetSnapshotID: string): Promise<LibrarySyncAccepted> {
const response = await fetch('/api/v1/library/syncs', {
method: 'POST',
headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
body: JSON.stringify({ scope, target_snapshot_id: targetSnapshotID }),
})
const value = (await response.json().catch(() => null)) as unknown
if (!response.ok) {
if (record(value) && Array.isArray(value.reasons)) {
const reasons = value.reasons.map(text).filter(Boolean)
if (reasons.length > 0) throw new Error(`启动同步被拒绝:${reasons.join(';')}`)
}
if (record(value) && text(value.error)) throw new Error(text(value.error))
throw new Error(`启动同步失败(HTTP ${response.status})`)
}
if (!record(value) || !record(value.job)) throw new Error('同步作业响应无效')
return { job_id: text(value.job.id), scope: text(value.job.scope), reused: bool(value.reused) }
}
// 建立目标权威基线:目标非空又没有生效版本时的唯一入口(维护隔离 + 真实捕获 + 校验)。
export async function enqueueLibraryBaseline(scope: string): Promise<LibrarySyncAccepted> {
const response = await fetch('/api/v1/library/baselines', {
method: 'POST',
headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
body: JSON.stringify({ scope }),
})
const value = (await response.json().catch(() => null)) as unknown
if (!response.ok) {
if (response.status === 403) throw new Error('建立目标权威基线仅管理员可发起。')
if (record(value) && text(value.error)) throw new Error(text(value.error))
throw new Error(`建立基线失败(HTTP ${response.status})`)
}
if (!record(value) || !record(value.job)) throw new Error('基线作业响应无效')
return { job_id: text(value.job.id), scope: text(value.job.scope), reused: bool(value.reused) }
}
// 恢复授权与执行是两件事:授权把"目标 + 快照 + 参数"钉死并给很短的有效期,
// 同步授权不能顶替恢复授权(摘要不同)。
export async function createLibraryRestoreAuthorization(
scope: string,
snapshotID: string,
minutes = 30
): Promise<LibraryRestoreGrant> {
const response = await fetch('/api/v1/library/authorizations', {
method: 'POST',
headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
body: JSON.stringify({ scope, snapshot_id: snapshotID, action: 'restore', expires_in_minutes: minutes }),
})
const value = (await response.json().catch(() => null)) as unknown
if (!response.ok) {
if (response.status === 403) throw new Error('生产发布授权仅管理员可签发。')
throw new Error(`签发恢复授权失败(HTTP ${response.status})`)
}
if (!record(value) || !record(value.authorization)) throw new Error('授权响应无效')
return {
id: text(value.authorization.id),
authorized_by: text(value.authorization.authorized_by),
expires_at: text(value.authorization.expires_at),
}
}
export async function fetchLibraryCaptures(signal?: AbortSignal): Promise<LibraryCaptureJob[]> { export async function fetchLibraryCaptures(signal?: AbortSignal): Promise<LibraryCaptureJob[]> {
const response = await fetch('/api/v1/library/captures', { const response = await fetch('/api/v1/library/captures', {
headers: { Accept: 'application/json' }, headers: { Accept: 'application/json' },
@@ -311,6 +475,18 @@ export interface LibraryMaintenanceView {
maintenance?: LibraryMaintenanceRecord | null maintenance?: LibraryMaintenanceRecord | null
instance?: string instance?: string
instance_maintenance?: LibraryInstanceMaintenanceRecord | null instance_maintenance?: LibraryInstanceMaintenanceRecord | null
// 能力是配置事实,和"此刻有没有窗口"分开:未运行维护 ≠ 没有维护能力。
capability?: LibraryMaintenanceCapability | null
}
export interface LibraryMaintenanceCapability {
configured: boolean
missing: string[]
instance?: string
project?: string
writers?: string[]
ingress_service?: string
note?: string
} }
export async function fetchLibraryMaintenance( export async function fetchLibraryMaintenance(
@@ -391,6 +567,18 @@ export interface LibraryGate {
reasons: string[] reasons: string[]
} }
export interface LibraryExecutorView {
available: boolean
reason?: string
instance?: string
project?: string
source_app?: string
archive_root?: string
busy: boolean
degraded?: string
observed_at?: string
}
export interface LibraryOverview { export interface LibraryOverview {
scope: string scope: string
test_sealed?: LibrarySnapshot test_sealed?: LibrarySnapshot
@@ -404,6 +592,8 @@ export interface LibraryOverview {
authorization: LibraryAuthorizationView authorization: LibraryAuthorizationView
capture_available: boolean capture_available: boolean
capture_reason?: string capture_reason?: string
// 执行器是 worker 的真实观测:指向哪个实例、何时观测、是否降级/在跑。
executor?: LibraryExecutorView
now: string now: string
} }
+82
View File
@@ -0,0 +1,82 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { createLibrarySyncAuthorization, enqueueLibraryBaseline, enqueueLibrarySync } from './librarySnapshots'
// 这些用例钉住"两步走"的接线:授权与执行是两个请求,参数必须与接口契约一致。
// 任何一步把 action 写错、或把 snapshot 换掉,都会在真实调用里悄悄失效(授权永远匹配不上)。
const jsonResponse = (status: number, body: unknown) =>
new Response(JSON.stringify(body), { status, headers: { 'Content-Type': 'application/json' } })
afterEach(() => vi.restoreAllMocks())
describe('发布页的同步授权与执行接线', () => {
it('签同步授权必须声明 action=sync 与有限有效期', async () => {
const fetchMock = vi.fn().mockResolvedValue(
jsonResponse(201, {
authorization: { id: 'libauth-sync-1', authorized_by: 'admin', expires_at: '2026-10-29T10:00:00Z' },
})
)
vi.stubGlobal('fetch', fetchMock)
const grant = await createLibrarySyncAuthorization('internal', 'snap-1', 30)
expect(grant.id).toBe('libauth-sync-1')
const [url, init] = fetchMock.mock.calls[0]
expect(url).toBe('/api/v1/library/authorizations')
expect(JSON.parse(init.body)).toEqual({
scope: 'internal',
snapshot_id: 'snap-1',
action: 'sync',
expires_in_minutes: 30,
})
})
it('启动同步把准确的 range 与目标快照传给 /library/syncs', async () => {
const fetchMock = vi
.fn()
.mockResolvedValue(jsonResponse(202, { job: { id: 'libsync-internal-1', scope: 'internal' }, reused: false }))
vi.stubGlobal('fetch', fetchMock)
const accepted = await enqueueLibrarySync('internal', 'snap-1')
expect(accepted.job_id).toBe('libsync-internal-1')
const [url, init] = fetchMock.mock.calls[0]
expect(url).toBe('/api/v1/library/syncs')
expect(JSON.parse(init.body)).toEqual({ scope: 'internal', target_snapshot_id: 'snap-1' })
})
it('闸口未齐时把接口给的缺项原样抛出,而不是假装已启动', async () => {
vi.stubGlobal(
'fetch',
vi.fn().mockResolvedValue(
jsonResponse(503, {
code: 'library_sync_blocked',
reasons: ['目标非空且没有生效版本记录', '没有与目标快照匹配的有效发布授权'],
})
)
)
await expect(enqueueLibrarySync('internal', 'snap-1')).rejects.toThrow(
/目标非空且没有生效版本记录;没有与目标快照匹配的有效发布授权/
)
})
it('建立基线只传 range,并返回作业号', async () => {
const fetchMock = vi
.fn()
.mockResolvedValue(jsonResponse(202, { job: { id: 'libbase-internal-1', scope: 'internal' }, reused: false }))
vi.stubGlobal('fetch', fetchMock)
const accepted = await enqueueLibraryBaseline('internal')
expect(accepted.job_id).toBe('libbase-internal-1')
const [url, init] = fetchMock.mock.calls[0]
expect(url).toBe('/api/v1/library/baselines')
expect(JSON.parse(init.body)).toEqual({ scope: 'internal' })
})
it('建立基线被拒时照实说明(例如目标为空或已有生效版本)', async () => {
vi.stubGlobal(
'fetch',
vi.fn().mockResolvedValue(
jsonResponse(409, {
code: 'library_baseline_exists',
error: '该目标已有生效版本,不需要再建基线;换版本请走同步或恢复',
})
)
)
await expect(enqueueLibraryBaseline('internal')).rejects.toThrow(/该目标已有生效版本/)
})
})