Files
knowledge-governance/web/src/sourceConnections.test.ts
姚章浩 2c7265efc4
build / build (push) Successful in 58s
feat(sources): 数据源配置收敛成 dws 导入 + PP 导入;「去掉来源」= 归档下架(不删数据)
主人要求:去掉入口,统一成 dws 导入与 PP 导入;其他来源连数据一起下架,之后用 dws 重新获取。

界面(web):
- 「数据源配置」只剩两块:**dws 导入(钉钉)**(按当前钉钉账号列出可访问的知识库与钉盘空间)
  与 **PP 导入**(PP 资料下载中心)。上一步加的「显示 AList/钉钉知识库来源」折叠开关删除。
- 来源卡片新增管理员可见的「去掉来源」(带确认弹窗,说明是归档下架、数据保留可恢复)。
- 侧栏文案改成"只留两个入口";发现面板标题改为「dws 导入(钉钉)」。

后端(归档语义,可逆):
- 迁移 000047:source_connections 增 archived_at / archived_by + 只覆盖未归档行的部分索引。
- store:ArchiveSourceConnection / RestoreSourceConnection(幂等;归档同时停用并清空
  next_refresh_at,避免被手工改回后立刻带着旧调度跑)。列表、目录、计划刷新(claimDue)、
  手动扫描(triggerBlock/inspect)四类查询一律排除归档行。
- 注册路径(dws 重新获取):RegisterDingtalkSpace 的 ON CONFLICT 清掉归档标记 ——
  重新登记同一个空间**原地复活同一行**,不产生重复,历史快照仍属于同一行。
- HTTP:POST /api/v1/source-connections/{id}/archive|restore(管理员专属,写意图头
  source-archive / source-restore + Idempotency-Key;404 区分"不存在"与"已是目标状态");
  openapi 同步两条契约。

验证:
- 集成测试(真 postgres):归档从列表消失但 source_items 一行不少、重复归档不漂移归档时间与人、
  未知 ID 报 ErrSourceConnectionNotFound、归档来源不能再被扫描、恢复后回到列表但保持停用;
  dws 重新登记同一空间复活同一行。
- httpapi:写意图/幂等键门禁、200/403/404/500、错误体不回显内部细节、classifyRole 管理员专属。
- 迁移护栏:000047 结构 + 四类 SQL 必须含 archived_at is null(漏一条就是入口没去掉)。
- make verify 15/15。
2026-10-14 12:36:32 +08:00

402 lines
13 KiB
TypeScript

import { afterEach, describe, expect, it, vi } from 'vitest'
import {
activateSourceSettingsRevision,
archiveSourceConnection,
createSourceSettingsRevision,
fetchSourceConnectionList,
fetchSourceConnections,
fetchSourceScanDetail,
fetchSourceSettingsRevisions,
fetchSourceScans,
parseSourceConnectionsResponse,
parseSourceScanDetailResponse,
parseSourceScansResponse,
restoreSourceConnection,
triggerSourceScan,
} from './sourceConnections'
afterEach(() => vi.unstubAllGlobals())
describe('parseSourceConnectionsResponse', () => {
it('normalizes only the safe source connection projection', () => {
const result = parseSourceConnectionsResponse({
data: [
{
id: 'conn-safe',
source_type: 'alist',
name: 'AList只读源',
enabled: true,
schedule_configured: true,
health_status: 'healthy',
item_count: 42,
updated_at: '2026-09-06T01:00:00Z',
base_url: 'https://secret-origin.example',
secret_ref: 'secret-name',
last_scan: {
id: 'scan-1',
state: 'completed',
observed_items: 42,
unchanged_items: 40,
added_items: 1,
updated_items: 1,
over_limit_items: 0,
missing_items: 0,
planned_uploads: 1,
error_code: '',
created_at: '2026-09-06T00:59:00Z',
completed_at: '2026-09-06T01:00:00Z',
snapshot_sha256: 'must-strip',
previous_revision: 'must-strip',
},
},
],
summary: { total: 1, enabled: 1, disabled: 0, healthy: 1, degraded: 0, unhealthy: 0, unknown: 0 },
})
expect(result.items).toEqual([
{
id: 'conn-safe',
source_type: 'alist',
name: 'AList只读源',
enabled: true,
schedule_configured: true,
health_status: 'healthy',
item_count: 42,
updated_at: '2026-09-06T01:00:00Z',
last_scan: {
id: 'scan-1',
state: 'completed',
observed_items: 42,
unchanged_items: 40,
added_items: 1,
updated_items: 1,
over_limit_items: 0,
missing_items: 0,
planned_uploads: 1,
error_code: '',
created_at: '2026-09-06T00:59:00Z',
completed_at: '2026-09-06T01:00:00Z',
},
},
])
expect(result.summary.total).toBe(1)
expect(result.items[0]).not.toHaveProperty('base_url')
expect(result.items[0]).not.toHaveProperty('secret_ref')
expect(result.items[0].last_scan).not.toHaveProperty('snapshot_sha256')
})
})
describe('fetchSourceConnections', () => {
it('uses the fixed read-only source connection endpoint', async () => {
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
json: async () => ({
data: [],
summary: { total: 0, enabled: 0, disabled: 0, healthy: 0, degraded: 0, unhealthy: 0, unknown: 0 },
}),
})
vi.stubGlobal('fetch', fetchMock)
await fetchSourceConnections()
expect(fetchMock).toHaveBeenCalledWith(
'/api/v1/source-connections',
expect.objectContaining({ headers: { Accept: 'application/json' } })
)
})
})
describe('triggerSourceScan', () => {
it('posts to the encoded connection scan endpoint and normalizes the response', async () => {
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
json: async () => ({
id: 'scan-safe',
source_connection_id: 'conn/a',
source_type: 'pp',
state: 'pending',
created_at: '2026-09-06T01:00:00Z',
snapshot_sha256: 'must-strip',
}),
})
vi.stubGlobal('fetch', fetchMock)
const scan = await triggerSourceScan('conn/a')
expect(fetchMock).toHaveBeenCalledWith(
'/api/v1/source-connections/conn%2Fa/scan',
expect.objectContaining({ method: 'POST' })
)
expect(scan).toEqual({
id: 'scan-safe',
source_connection_id: 'conn/a',
source_type: 'pp',
state: 'pending',
created_at: '2026-09-06T01:00:00Z',
})
})
})
describe('source scan history and details', () => {
const scan = {
id: 'scan-safe',
state: 'completed',
observed_items: 2,
unchanged_items: 0,
added_items: 1,
updated_items: 0,
over_limit_items: 0,
missing_items: 1,
planned_uploads: 1,
error_code: '',
created_at: '2026-09-06T06:00:00Z',
completed_at: '2026-09-06T06:01:00Z',
snapshot_sha256: 'must-be-dropped',
stable_external_id: 'must-be-dropped',
}
it('strictly parses safe history and detail projections', () => {
expect(parseSourceScansResponse({ data: [scan], total: 1, limit: 20, offset: 0 })).toEqual({
items: [
{
id: 'scan-safe',
state: 'completed',
observed_items: 2,
unchanged_items: 0,
added_items: 1,
updated_items: 0,
over_limit_items: 0,
missing_items: 1,
planned_uploads: 1,
error_code: '',
created_at: '2026-09-06T06:00:00Z',
completed_at: '2026-09-06T06:01:00Z',
},
],
total: 1,
limit: 20,
offset: 0,
})
expect(
parseSourceScanDetailResponse({
data: {
...scan,
changes: [
{
id: 'chg-safe',
source_item_id: 'si-safe',
change_type: 'updated',
observed_size_bytes: 42,
created_at: '2026-09-06T06:01:00Z',
previous_revision: 'must-be-dropped',
observed_revision: 'must-be-dropped',
},
],
},
})
).toEqual({
id: 'scan-safe',
state: 'completed',
observed_items: 2,
unchanged_items: 0,
added_items: 1,
updated_items: 0,
over_limit_items: 0,
missing_items: 1,
planned_uploads: 1,
error_code: '',
created_at: '2026-09-06T06:00:00Z',
completed_at: '2026-09-06T06:01:00Z',
changes: [
{
id: 'chg-safe',
source_item_id: 'si-safe',
change_type: 'updated',
observed_size_bytes: 42,
created_at: '2026-09-06T06:01:00Z',
},
],
})
})
it('requests connection-scoped history and details', async () => {
const fetchMock = vi
.fn()
.mockResolvedValueOnce(
new Response(JSON.stringify({ data: [scan], total: 1, limit: 20, offset: 0 }), { status: 200 })
)
.mockResolvedValueOnce(new Response(JSON.stringify({ data: { ...scan, changes: [] } }), { status: 200 }))
vi.stubGlobal('fetch', fetchMock)
await fetchSourceScans('conn / pp')
await fetchSourceScanDetail('conn / pp', 'scan / one')
expect(fetchMock.mock.calls[0][0]).toBe('/api/v1/source-connections/conn%20%2F%20pp/scans?limit=20&offset=0')
expect(fetchMock.mock.calls[1][0]).toBe('/api/v1/source-connections/conn%20%2F%20pp/scans/scan%20%2F%20one')
})
})
describe('V1 source settings client', () => {
it('projects the connection list without sensitive fields', async () => {
const fetchMock = vi.fn().mockResolvedValue(
new Response(
JSON.stringify({
data: [
{
id: 'conn-1',
source_type: 'alist',
name: 'A',
enabled: true,
schedule_configured: true,
health_status: 'healthy',
item_count: 3,
updated_at: '2026-09-12T01:00:00Z',
next_refresh_at: '2026-09-12T09:00:00Z',
base_url: 'https://strip.me',
secret_ref: 'strip-too',
},
],
}),
{ status: 200 }
)
)
vi.stubGlobal('fetch', fetchMock)
const items = await fetchSourceConnectionList()
expect(items).toEqual([
{
id: 'conn-1',
source_type: 'alist',
name: 'A',
enabled: true,
schedule_configured: true,
health_status: 'healthy',
item_count: 3,
updated_at: '2026-09-12T01:00:00Z',
next_refresh_at: '2026-09-12T09:00:00Z',
},
])
expect(JSON.stringify(items)).not.toContain('strip')
})
it('rejects malformed connection items', async () => {
vi.stubGlobal(
'fetch',
vi.fn().mockResolvedValue(
new Response(
JSON.stringify({
data: [{ id: 'x' }],
}),
{ status: 200 }
)
)
)
await expect(fetchSourceConnectionList()).rejects.toThrow('invalid source connection item')
})
it('creates and activates revisions with intent + idempotency headers and surfaces operation', async () => {
const fetchMock = vi
.fn()
.mockResolvedValueOnce(
new Response(JSON.stringify({ data: { id: 'scr-2' }, operation: 'created' }), { status: 201 })
)
.mockResolvedValueOnce(
new Response(JSON.stringify({ data: { connection_id: 'c', revision_id: 'scr-2' }, operation: 'replayed' }), {
status: 200,
})
)
vi.stubGlobal('fetch', fetchMock)
expect(
await createSourceSettingsRevision('conn/1', {
source_type: 'alist',
name: 'A',
endpoint: '',
credential_ref: '',
enabled: true,
schedule: { refresh_interval_seconds: 3600 },
})
).toEqual({ operation: 'created', id: 'scr-2' })
expect(await activateSourceSettingsRevision('conn/1', 'scr/2')).toEqual({ operation: 'replayed', id: null })
const [createCall, activateCall] = fetchMock.mock.calls
expect(createCall[0]).toBe('/api/v1/source-connections/conn%2F1/settings/revisions')
expect((createCall[1] as RequestInit).headers).toMatchObject({ 'X-Governance-Action': 'source-settings' })
expect(((createCall[1] as RequestInit).headers as Record<string, string>)['Idempotency-Key']).toMatch(
/[0-9a-f-]{36}/
)
expect(activateCall[0]).toBe('/api/v1/source-connections/conn%2F1/settings/revisions/scr%2F2/activate')
expect(createCall[1]).not.toBe(activateCall[1])
})
it('surfaces server error details on failure', async () => {
vi.stubGlobal(
'fetch',
vi
.fn()
.mockResolvedValue(
new Response(JSON.stringify({ error: 'manual sources cannot be scheduled' }), { status: 400 })
)
)
await expect(
createSourceSettingsRevision('c', {
source_type: 'manual',
name: 'A',
endpoint: '',
credential_ref: '',
enabled: true,
schedule: { refresh_interval_seconds: 3600 },
})
).rejects.toThrow('保存来源设置草稿失败:manual sources cannot be scheduled')
})
it('parses revision history strictly', async () => {
const revision = {
id: 'scr-1',
revision_number: 1,
state: 'active',
name: 'A',
endpoint: 'http://x',
credential_ref: 'ref',
enabled: true,
refresh_interval_seconds: '3600',
created_by: 'z',
created_at: '2026-09-12T01:00:00Z',
validated_at: null,
activated_at: '2026-09-12T01:00:00Z',
retired_at: null,
}
vi.stubGlobal(
'fetch',
vi.fn().mockResolvedValue(new Response(JSON.stringify({ data: [revision] }), { status: 200 }))
)
const history = await fetchSourceSettingsRevisions('c')
expect(history[0].state).toBe('active')
vi.stubGlobal(
'fetch',
vi
.fn()
.mockResolvedValue(new Response(JSON.stringify({ data: [{ ...revision, state: 'weird' }] }), { status: 200 }))
)
await expect(fetchSourceSettingsRevisions('c')).rejects.toThrow('invalid source settings revision')
})
})
describe('去掉来源(归档)客户端', () => {
it('归档/恢复都带写意图头与幂等键,并正确编码来源 ID', async () => {
const fetchMock = vi.fn().mockResolvedValue(new Response('{}', { status: 200 }))
vi.stubGlobal('fetch', fetchMock)
await archiveSourceConnection('conn/1')
const [archiveURL, archiveInit] = fetchMock.mock.calls[0]
expect(archiveURL).toBe('/api/v1/source-connections/conn%2F1/archive')
expect(archiveInit.method).toBe('POST')
expect(archiveInit.headers['X-Governance-Action']).toBe('source-archive')
expect(typeof archiveInit.headers['Idempotency-Key']).toBe('string')
expect(archiveInit.headers['Idempotency-Key'].length).toBeGreaterThan(0)
await restoreSourceConnection('conn/1')
const [restoreURL, restoreInit] = fetchMock.mock.calls[1]
expect(restoreURL).toBe('/api/v1/source-connections/conn%2F1/restore')
expect(restoreInit.headers['X-Governance-Action']).toBe('source-restore')
})
it('后端拒绝时把原因带出来(不吞错误)', async () => {
vi.stubGlobal(
'fetch',
vi.fn().mockResolvedValue(new Response(JSON.stringify({ error: 'source connection not found' }), { status: 404 }))
)
await expect(archiveSourceConnection('conn-x')).rejects.toThrow('去掉来源失败:source connection not found')
})
})