fix(monitor): 需下架车道恒为 0(判据用了没人写的状态)+ 隔离会改线上设置的集成测试
界面验收第三遍(禁止导入 → 需下架)暴露两处真问题: 1) 需下架车道死掉:判据 `retrieval_state='enabled'`,而 retrieval_state 在这套状态机里 描述「是否发布到生产」(unpublished/disabling/disabled/failed),全仓没有任何代码写过 'enabled',线上 5/5 发布单元都是 unpublished → 车道恒 0,界面「仍在知识库里的文件会 出现在这里」的承诺不成立。改为与设计注释一致的口径:仍在测试库存活(completed + 有 knowledge_id + 未 disabling/disabled)+ 来源项 policy_state='blocked'。 线上对照:旧判据 0 → 新判据 1,命中被禁止导入的 DH-AG95夹爪适配-指令协议.md; 修后页面显示「需下架(1)· 对内知识库 · 1 份仍在检索中」。新增集成测试覆盖 存活/未禁止/移除中/已移除/未上传五种情形(真实 Postgres + 独立 schema)。 2) 集成测试改到线上设置(直接踩到并已恢复):ui_settings_integration_test.go 只判断 TEST_DATABASE_URL 是否存在、无 schema 隔离,而它「创建候选并激活」——跑在共享库上会把 真实 active 设置挤成 retired。已先备份(backups/ui-settings/…pre-restore.jsonl),再用 单事务(先 retire 再 activate,uq_ui_setting_active 只允许一条 active)恢复 uis-d40ea1fc…(15/30/1800/7200 + https://weknora-test.dobot.cn/),并复核线上值。 根因修复:给该用例加 per-test schema 隔离,复跑后线上设置不再被改动。 同时把两处与事实不符的文案改实话(WeKnoraMonitor.tsx 需下架说明、SourceCatalogBrowser.tsx 禁止导入说明):移除走 test.remove 工作流(删除测试库知识条目,可再入库),入口当前由 变更集删除路径触发,页面尚无独立按钮——条目级列表与下架按钮列为待排期。
This commit is contained in:
@@ -93,6 +93,16 @@
|
||||
- 期望:失败可重排;"需下架"列出仍留在库里的禁止项
|
||||
- 证据:死信与重排链路(`cddc4c9`)
|
||||
- 状态:✅ 链路|⚠️ "重排"按钮行为目前只有单测覆盖(页面当时无失败项可点)
|
||||
- **2026-10-10 界面实测(第三遍:禁止导入 → 需下架)**:把已入库的 `DH-AG95夹爪适配-指令协议.md` 改成「禁止导入」
|
||||
→ 回执「策略已保存 1 份、标记禁止 1 份、取件 0、待入库 0」✓,目录表格变为「禁止导入」✓
|
||||
- **发现真问题(已修)**:`需下架` 车道恒为 0。车道 SQL 判 `retrieval_state='enabled'`,但 `retrieval_state`
|
||||
在这套状态机里描述的是**是否发布到生产**(`unpublished/disabling/disabled/failed`),全仓没有任何代码写过 `'enabled'`
|
||||
—— 实测线上 5/5 发布单元都是 `unpublished`,界面上「仍在知识库里的文件会出现在这里」的承诺不成立。
|
||||
修为与设计注释一致的口径:**仍在测试库存活(`completed` + 有 `knowledge_id` + 未 `disabling/disabled`)+ 来源项 `policy_state='blocked'`**;
|
||||
线上库对照:旧判据 0 → 新判据 1,命中正是那支被禁止导入的 `.md` ✓;页面随即显示 `需下架(1)· 对内知识库 · 1 份仍在检索中` ✓
|
||||
- **仍未闭环(待排期)**:① 车道只按库计数,**没有条目级列表**;② 移除动作(`test.remove` 工作流,已实现且 worker 已注册)
|
||||
目前只由变更集删除路径触发,**页面没有入口**;③ 该工作流是 `DeleteKnowledge`(从测试库删知识条目,重新标注可再入库),
|
||||
与旧文案"停用检索(可恢复)"的措辞不同 —— 文案已按事实改写(`WeKnoraMonitor.tsx` / `SourceCatalogBrowser.tsx`)
|
||||
|
||||
## C. 发布与回滚
|
||||
|
||||
@@ -221,6 +231,15 @@
|
||||
3. **wiki 通道的真实跨环境发布已跑通**(2026-10-10,见 C2):发现并修复"写入保留外来源页、校验却要求集合相等"的口径冲突(`eabe55b`);降级保留(`degraded=true`)现在是发布常态路径。
|
||||
3b. **指标条的健康探测有误报**:`解析监控` 顶部把 Embedding 模型 / LLM 模型 / OSS / Redis-MQ / WeKnora API 显示为"阻断",但本期入库与发布全链路(上传/解析/索引/发布/回滚)都实际成功 → 探测口径待校正(不作阻塞项)。
|
||||
3c. **`系统设置 → 服务配置` 的"测试端"探测报 404**:`test_weknora_base_url` 为空时 api 用 `TEST_WEKNORA_BASE_URL=http://app:8080` 探测,而 `app` 在治理栈网络内不可解析 → 显示"可达·不可接单";worker 走的是可用路径(本清单 B1/C2 均成功),仅探测口径问题。
|
||||
3f. **集成测试曾改到线上设置(已修 + 已恢复数据)**:`internal/httpapi/ui_settings_integration_test.go` 只判断
|
||||
`TEST_DATABASE_URL` 是否存在、**没有 schema 隔离**,而它就做「创建候选并激活」——直接跑在共享/线上库上时,
|
||||
会把真实的 active 设置挤成 retired。2026-10-10 排查车道问题时踩到:线上 active 变成测试负载
|
||||
(30/60/3600/14400 + `https://weknora.prod.example:8443/weknora`),真实的 `uis-d40ea1fc…`
|
||||
(15/30/1800/7200 + 生产端点 `https://weknora-test.dobot.cn/`)被 retired。
|
||||
处理:先备份 `backups/ui-settings/ui_setting_revisions.pre-restore.*.jsonl`,再用单事务(先 retire 再 activate,
|
||||
唯一约束 `uq_ui_setting_active` 只允许一条 active)恢复,两条 update 各影响 1 行,并复核线上值 ✓;
|
||||
根因修复=给该用例加 per-test schema 隔离(照 `internal/monitoring`),复跑后线上设置不再被改动 ✓。
|
||||
**规矩**:任何"创建并激活"的集成测试必须建独立 schema;只判 `TEST_DATABASE_URL` 存在就写共享库的用例一律视为危险。
|
||||
3e. **发布/回滚结果页不自动刷新**:Release Worker 完成后,`发布管理` 的执行行仍显示"待执行"(首次与第二遍都遇到),
|
||||
需手动 `刷新当前页面`/reload 才看到 `已完成`/`已回滚`。后端状态是对的(DB 已是终态),属展示层轮询缺口。
|
||||
3d. **`release_executions.snapshot_verification_id` 为空**:发布前快照门禁确实生效(缺少/过期会 `release_target_snapshot_missing`),但执行行没有记下"用的是哪份快照",追踪性待补。
|
||||
|
||||
@@ -3,9 +3,12 @@ package httpapi
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -16,13 +19,49 @@ import (
|
||||
postgresstore "knowledge-governance/internal/platform/postgres"
|
||||
)
|
||||
|
||||
func TestUISettingsWriteReadBackIntegration(t *testing.T) {
|
||||
// isolatedUISettingsURL 给本用例一个独立 schema。
|
||||
//
|
||||
// 这个用例会「创建候选并激活」——激活会把当前的 active 设置挤成 retired。若直接跑在
|
||||
// 共享库(尤其是线上治理库)上,它会静默改掉真实的刷新策略与端点:2026-10-10 实测踩到,
|
||||
// 真实那条 uis-d40ea1fc… 被 retired、测试负载(30/60/3600/14400 + 示例端点)被激活,
|
||||
// 已用备份恢复。参照 internal/monitoring 的做法加 schema 隔离,杜绝再次发生。
|
||||
func isolatedUISettingsURL(t *testing.T) string {
|
||||
t.Helper()
|
||||
databaseURL := os.Getenv("TEST_DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
t.Skip("TEST_DATABASE_URL is not set")
|
||||
}
|
||||
parsed, err := url.Parse(databaseURL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
digest := sha256.Sum256([]byte(t.Name()))
|
||||
schema := "kgtest_" + hex.EncodeToString(digest[:8])
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
admin, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer admin.Close()
|
||||
if _, err := admin.Exec(ctx, "CREATE SCHEMA IF NOT EXISTS "+schema); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
cleanup, err := pgxpool.New(context.Background(), databaseURL)
|
||||
if err == nil {
|
||||
_, _ = cleanup.Exec(context.Background(), "DROP SCHEMA IF EXISTS "+schema+" CASCADE")
|
||||
cleanup.Close()
|
||||
}
|
||||
})
|
||||
query := parsed.Query()
|
||||
query.Set("search_path", schema)
|
||||
parsed.RawQuery = query.Encode()
|
||||
return parsed.String()
|
||||
}
|
||||
|
||||
func TestUISettingsWriteReadBackIntegration(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, isolatedUISettingsURL(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -134,7 +134,14 @@ func (provider *RealProvider) Overview(ctx context.Context) (governance.MonitorO
|
||||
COUNT(*) FILTER (WHERE p.superseded_at IS NULL AND p.target_scope='external')::integer,
|
||||
COUNT(*) FILTER (WHERE p.superseded_at IS NULL AND p.target_scope='external' AND p.processing_state='completed')::integer,
|
||||
COUNT(*) FILTER (WHERE p.superseded_at IS NULL AND p.processing_state='planned')::integer,
|
||||
COUNT(*) FILTER (WHERE p.superseded_at IS NULL AND p.retrieval_state='enabled'
|
||||
-- 需下架:在测试库里仍然存活(已完成、有知识 id、尚未标记移除)、但来源项已被判为禁止导入的条目。
|
||||
-- 曾经这里判 p.retrieval_state='enabled',而 retrieval_state 在这套状态机里描述的是
|
||||
-- “是否发布到生产”(unpublished/disabling/disabled/failed),全仓没有任何代码写过 'enabled',
|
||||
-- 所以这条车道恒为 0 —— 2026-10-10 界面验收实测:把已入库的文件改成“禁止导入”后
|
||||
-- 车道仍显示 0,界面承诺的“仍在知识库里的文件会出现在这里”不成立。
|
||||
COUNT(*) FILTER (WHERE p.superseded_at IS NULL AND p.knowledge_id<>''
|
||||
AND p.processing_state='completed'
|
||||
AND p.retrieval_state NOT IN ('disabling','disabled')
|
||||
AND item.policy_state='blocked')::integer,
|
||||
COUNT(*) FILTER (WHERE p.superseded_at IS NULL AND p.processing_state='planned'
|
||||
AND item.policy_state='blocked')::integer
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
package monitoring
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
"knowledge-governance/internal/platform/postgres"
|
||||
)
|
||||
|
||||
// TestRealOverviewRemovalDemandCountsLiveBlockedPublications freezes the
|
||||
// 需下架 lane definition after the 2026-10-10 UI acceptance run proved it dead:
|
||||
// the SQL asked for `retrieval_state='enabled'`, a value this state machine
|
||||
// never writes (retrieval_state describes production publication:
|
||||
// unpublished/disabling/disabled/failed), so "把已入库的文件改成禁止导入后
|
||||
// 车道仍显示 0" — the page promise in SourceCatalogBrowser ("仍在知识库里的文件
|
||||
// 会出现在解析监控 → 需下架") did not hold.
|
||||
//
|
||||
// The lane must count knowledge that is still alive in the test/authoring KB
|
||||
// (completed + real knowledge id + not already flagged/removed) whose source
|
||||
// item has been marked 禁止导入, and must drop each row as soon as removal is
|
||||
// under way ('disabling') or done ('disabled').
|
||||
func TestRealOverviewRemovalDemandCountsLiveBlockedPublications(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, isolatedMetricsURL(t))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
if err := postgres.ApplyMigrations(ctx, pool); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO source_connections(id,source_type,name,enabled) VALUES('conn-removal','dingtalk','需下架用例',TRUE)`); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
seedItem := func(itemID, stableID, policy string) {
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO source_items(id,source_connection_id,stable_external_id,canonical_uri_hash,title,item_type,policy_state)
|
||||
VALUES($1,'conn-removal',$2,repeat('a',64),$2,'file',$3)`, itemID, stableID, policy); err != nil {
|
||||
t.Fatalf("seed item %s: %v", itemID, err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO source_item_versions(id,source_item_id,revision,content_state) VALUES($1,$2,'r1','verified')`, "ver-"+itemID, itemID); err != nil {
|
||||
t.Fatalf("seed version %s: %v", itemID, err)
|
||||
}
|
||||
}
|
||||
seedPublication := func(publicationID, itemID, knowledgeID, processing, retrieval string) {
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO test_publications(id,source_item_version_id,test_kb_id,knowledge_id,ingest_state,processing_state,retrieval_state)
|
||||
VALUES($1,$2,'kb-removal',$3,'uploaded',$4,$5)`, publicationID, "ver-"+itemID, knowledgeID, processing, retrieval); err != nil {
|
||||
t.Fatalf("seed publication %s: %v", publicationID, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Live in the test KB and now marked 禁止导入 → must be exposed.
|
||||
seedItem("item-blocked", "blocked.md", "blocked")
|
||||
seedPublication("pub-blocked", "item-blocked", "knowledge-blocked", "completed", "unpublished")
|
||||
// Same shape but still routed internally → nothing to do.
|
||||
seedItem("item-internal", "internal.md", "internal")
|
||||
seedPublication("pub-internal", "item-internal", "knowledge-internal", "completed", "unpublished")
|
||||
// Removal already running / already done → drop out of the lane.
|
||||
seedItem("item-disabling", "disabling.md", "blocked")
|
||||
seedPublication("pub-disabling", "item-disabling", "knowledge-disabling", "completed", "disabling")
|
||||
seedItem("item-disabled", "disabled.md", "blocked")
|
||||
seedPublication("pub-disabled", "item-disabled", "knowledge-disabled", "completed", "disabled")
|
||||
// Blocked before it ever reached a KB → nothing live to remove.
|
||||
seedItem("item-planned", "planned.md", "blocked")
|
||||
seedPublication("pub-planned", "item-planned", "", "planned", "unpublished")
|
||||
|
||||
provider, err := NewRealProvider(RealProviderConfig{DB: metricsPool{pool: pool}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
overview, err := provider.Overview(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("overview: %v", err)
|
||||
}
|
||||
if len(overview.KnowledgeBases) != 1 {
|
||||
t.Fatalf("expected one knowledge base row, got %#v", overview.KnowledgeBases)
|
||||
}
|
||||
progress := overview.KnowledgeBases[0]
|
||||
if progress.NeedsRemoval != 1 {
|
||||
t.Fatalf("needs_removal=%d, want 1 (live + blocked only)", progress.NeedsRemoval)
|
||||
}
|
||||
if progress.StalePlanned != 1 {
|
||||
t.Fatalf("stale_planned=%d, want 1 (blocked before upload)", progress.StalePlanned)
|
||||
}
|
||||
if progress.Completed != 4 {
|
||||
t.Fatalf("completed=%d, want 4", progress.Completed)
|
||||
}
|
||||
}
|
||||
@@ -73,7 +73,7 @@ export function SourceCatalogBrowserView({sources,selectedSource,page,loading,er
|
||||
<label className="routing-check"><input type="checkbox" checked={selectableNodes.length>0&&selectableNodes.every(node=>selectedIDs.includes(node.node_id))} disabled={!selectableNodes.length} onChange={event=>setSelectedIDs(event.target.checked?selectableNodes.map(node=>node.node_id):[])} />全选当前页</label>
|
||||
<div className="catalog-policy-actions"><button className="internal-policy-button" disabled={!routeReady||!selectedNodeIDs.length||!policyEnabled('internal')} onClick={()=>void applyPolicy('internal')}>对内</button><button className="external-policy-button" disabled={!routeReady||!selectedNodeIDs.length||!policyEnabled('external')} onClick={()=>void applyPolicy('external')}>对外</button><div className="blocked-policy-stack"><button className="blocked-policy-button" disabled={!routeReady||!selectedNodeIDs.length} onClick={()=>void applyPolicy('blocked')}>禁止导入</button><small className="selected-count">已选 {selectedCount} 项</small></div></div>
|
||||
</div>
|
||||
<small className="blocked-policy-note">禁止导入即生效:待入库条目自动撤销;仍在知识库里的文件会出现在“解析监控 → 需下架”,在知识清单停用检索,不删除来源文件,也不操作生产知识库。</small>
|
||||
<small className="blocked-policy-note">禁止导入即生效:待入库条目自动撤销;仍在测试库里的文件会出现在“解析监控 → 需下架”暴露差异,移除走 test.remove 工作流;不删除来源文件,也不操作生产知识库。</small>
|
||||
{routingMessage?<p role="status">{routingMessage}</p>:null}
|
||||
</section>:null}
|
||||
{visibleItems.length?<div className="catalog-table-wrap"><table className="catalog-table"><thead><tr><th>名称</th><th>类型</th><th>处理策略</th><th>文件数量</th><th>导入与解析</th><th>大小</th><th>更新时间</th></tr></thead><tbody>{visibleItems.map(node=>{const visibility=appliedPolicies[node.node_id]||node.visibility;const size=node.node_type==='folder'?(node.subtree_bytes>0?`已知合计 ${bytes(node.subtree_bytes)}`:'大小未提供'):typeof node.size_bytes==='number'?bytes(node.size_bytes):'大小未提供';return <tr key={node.node_id}><td>{node.node_type==='folder'?<div className="catalog-node-control"><input type="checkbox" aria-label={`选择${node.name}`} checked={selectedIDs.includes(node.node_id)} onChange={()=>toggleNode(node.node_id)} /><button className="catalog-name" onClick={()=>onOpenFolder(node)}><span aria-hidden="true">📁</span>{node.name}</button></div>:<label className="catalog-file"><input type="checkbox" aria-label={`选择${node.name}`} checked={selectedIDs.includes(node.node_id)} onChange={()=>toggleNode(node.node_id)} /><span aria-hidden="true">📄</span>{node.name}</label>}</td><td>{node.node_type==='folder'?'文件夹':'文件'}</td><td><div className="catalog-policy-cell"><span className={`scope-badge ${visibility}`}>{scope(visibility)}</span></div></td><td>{node.node_type==='folder'?`${node.subtree_file_count} 份`:'—'}</td><td><span>{progress(node)}</span>{node.active_count?<small>处理中 {node.active_count}</small>:null}{node.failed_count?<small className="problem-text">失败 {node.failed_count}</small>:null}</td><td>{size}</td><td>{node.modified_at?new Date(node.modified_at).toLocaleString('zh-CN',{hour12:false}):'—'}</td></tr>})}</tbody></table></div>:<div className="empty-state">当前目录为空。</div>}
|
||||
|
||||
@@ -218,7 +218,7 @@ export function WeKnoraMonitorView({
|
||||
{lane === 'removal' ? <section className="inbox-section" aria-label="需下架">
|
||||
<div className="section-head"><h2>需下架({laneCounts.removal})</h2><span className="tiny">来源目录里被改为“禁止导入”或撤销标注、但仍在知识库中可检索的文件</span></div>
|
||||
{(overview?.knowledge_bases || []).some((kb) => kb.needs_removal > 0) ? <ul className="dlq-list">{(overview?.knowledge_bases || []).filter((kb) => kb.needs_removal > 0).map((kb) => <li key={kb.id}><span className="monitor-status failed">需下架</span><b>{kb.name}</b><small>{kb.needs_removal} 份仍在检索中</small></li>)}</ul> : emptyLane('没有标注为禁止但仍留在知识库里的文件。')}
|
||||
<p className="tiny">下架在“知识清单”里操作(停用检索,可恢复);这里只负责把差异暴露出来。</p>
|
||||
<p className="tiny">这里只负责把差异暴露出来:来源项被判“禁止导入”后,仍存活在测试库里的知识必须移除。移除动作走 <code>test.remove</code> 工作流(会删除测试库里的知识条目,重新标注后可再次入库);该入口当前由变更集删除路径触发,页面尚未提供独立按钮。</p>
|
||||
</section> : null}
|
||||
{lane === 'inflight' ? <section className="inbox-section" aria-label="处理中">
|
||||
<div className="section-head"><h2>处理中({inflightItems.length})</h2><span className="tiny">无需操作,仅供查看</span></div>
|
||||
|
||||
Reference in New Issue
Block a user