feat(settings): 打通 PP 认证配置并统一 DWS 来源授权
build / build (push) Successful in 2m20s

PP 增加受控凭据更新、真实登录与目录检查,待验证修改不会替换生效配置;扫描与取件共享生效修订。
钉钉来源向导、校验与登记共用 DWS 授权,支持中途恢复与换账号保护,取消旧扫码自动回落。
认证审计复用已有来源审计页,配置响应不回传密码;补齐运行镜像脚本与权限。

验证:make verify 退出 0;PP/DWS/httpapi 目标 race 回归通过;隔离 HTTP 服务验证 PP 协议及扫描/取件消费者。
尚未部署;真实 PP 凭据与目录可用性在测试站部署后验证。
This commit is contained in:
姚章浩
2026-11-17 19:08:46 +08:00
parent e0fa31c9d6
commit 3825859da6
54 changed files with 3219 additions and 639 deletions
+3
View File
@@ -2386,6 +2386,9 @@
### Added
- 数据源配置新增 PP API 地址、账号与密码更新、真实登录及目录连接检查;配置校验通过才生效,临时故障保留待验证修改,扫描与取件共用生效配置。
- 钉钉添加来源统一使用 DWS 官方授权,增加真实空间校验、中途失效恢复与账号变化保护,取消登记自动回落旧扫码侧车;认证操作接入既有审计。
- 数据源配置新增 DWS 登录与重新授权入口,使用钉钉官方设备授权;成功后刷新账号并独立检查目录访问权限。支持取消、过期和重试,授权仅限发起的管理员会话。[操作说明](docs/DWS_LOGIN.md)
- 新增解析监控统一工作台:顶部运行指标与依赖卡片、下方单一文件处理表;来源文件与目标测试库归并最新版本,动作、阶段、状态分列,后端统一筛选、计数及分页。支持单条/跨页勾选重试和版本、失败、重试详情;测试库移除重试独立核验治理规则,不误走上传流程。取消六分页和手动开始入口,生产授权仍在发布管理;本次不改变后台自动调度策略,自动处理闭环需另行端到端验收。
+2 -1
View File
@@ -51,6 +51,7 @@ RUN set -eux; \
WORKDIR /app
COPY --from=build /out/ /app/
COPY scripts/source_catalog_import.py scripts/live_catalog_refresh.py scripts/dws_dingtalk_scan.py \
scripts/artifact_fetch.py scripts/axls_to_xlsx.py scripts/dws_error_map.py /app/scripts/
scripts/pp_client.py scripts/artifact_fetch.py scripts/axls_to_xlsx.py scripts/dws_error_map.py /app/scripts/
RUN chmod 0644 /app/scripts/*.py
USER 65532:65532
CMD ["/app/api"]
+1 -1
View File
@@ -54,7 +54,7 @@ Knowledge Adapter / DobotAI 网关
| --- | --- | --- |
| 钉钉 | 知识库目录发现与取件;优先 dws,按错误类型决定是否回落兼容路径 | [登录授权](docs/DWS_LOGIN.md) · [钉钉取件](docs/DINGTALK_DWS_FETCHER.md) |
| AList | 来源目录与资料获取,依赖指定连接和凭据 | [用户操作流程](docs/OPERATIONS_RUNBOOK.md) |
| PP(资料门户) | 本地清单、历史账本接管与只读扫描 | [历史接管](docs/P2_PP_LEDGER_TAKEOVER_ACCEPTANCE.md)、[来源扫描](docs/P42_SOURCE_SCAN_ACCEPTANCE.md) |
| PP(资料门户) | 页面管理 API 与认证配置;真实校验、只读扫描及取件 | [认证配置](docs/SOURCE_AUTH_CONFIGURATION.md) · [历史接管](docs/P2_PP_LEDGER_TAKEOVER_ACCEPTANCE.md)、[来源扫描](docs/P42_SOURCE_SCAN_ACCEPTANCE.md) |
IT、行政等是待扩展的业务资料范围,不是已经完成的新连接器。接入敏感资料前,需要先确认 Owner、有效版本、可见范围与知识库路由。
+339
View File
@@ -2682,6 +2682,211 @@
}
}
},
"/api/v1/pp-connection": {
"get": {
"operationId": "getPPConnection",
"summary": "管理员读取 PP 配置元数据及最近检查结果",
"responses": {
"200": {
"description": "请求完成",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"data": {
"$ref": "#/components/schemas/PPConnectionView"
}
}
}
}
}
},
"400": {
"description": "输入格式错误"
},
"401": {
"description": "需要登录或重新授权"
},
"403": {
"description": "需要管理员权限或缺少目录权限"
},
"409": {
"description": "配置或账号已变化"
},
"422": {
"description": "明确拒绝本次配置,原配置保留"
},
"500": {
"description": "内部错误"
},
"503": {
"description": "依赖服务暂不可用"
}
}
},
"put": {
"operationId": "savePPConnection",
"summary": "管理员保存并真实校验 PP 配置",
"responses": {
"200": {
"description": "请求完成",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"data": {
"$ref": "#/components/schemas/PPConnectionView"
}
}
}
}
}
},
"400": {
"description": "输入格式错误"
},
"401": {
"description": "需要登录或重新授权"
},
"403": {
"description": "需要管理员权限或缺少目录权限"
},
"409": {
"description": "配置或账号已变化"
},
"422": {
"description": "明确拒绝本次配置,原配置保留"
},
"500": {
"description": "内部错误"
},
"503": {
"description": "依赖服务暂不可用"
}
},
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PPConnectionDraft"
}
}
}
}
}
},
"/api/v1/pp-connection/test": {
"post": {
"operationId": "testPPConnection",
"summary": "管理员使用草稿真实登录并读取最小目录,不登记来源",
"responses": {
"200": {
"description": "请求完成",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"data": {
"$ref": "#/components/schemas/PPObservation"
}
}
}
}
}
},
"400": {
"description": "输入格式错误"
},
"401": {
"description": "需要登录或重新授权"
},
"403": {
"description": "需要管理员权限或缺少目录权限"
},
"409": {
"description": "配置或账号已变化"
},
"422": {
"description": "明确拒绝本次配置,原配置保留"
},
"500": {
"description": "内部错误"
},
"503": {
"description": "依赖服务暂不可用"
}
},
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PPConnectionDraft"
}
}
}
}
}
},
"/api/v1/dingtalk-spaces/validate": {
"post": {
"operationId": "validateDingtalkSourceSpaces",
"summary": "管理员用共享 DWS 身份真实检查所选空间可见性",
"responses": {
"200": {
"description": "请求完成"
},
"400": {
"description": "输入格式错误"
},
"401": {
"description": "需要登录或重新授权"
},
"403": {
"description": "需要管理员权限或缺少目录权限"
},
"409": {
"description": "配置或账号已变化"
},
"422": {
"description": "明确拒绝本次配置,原配置保留"
},
"500": {
"description": "内部错误"
},
"503": {
"description": "依赖服务暂不可用"
}
},
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"required": [
"urls"
],
"properties": {
"urls": {
"type": "array",
"minItems": 1,
"maxItems": 100,
"items": {
"type": "string"
}
}
}
}
}
}
}
}
},
"/api/v1/dingtalk-spaces": {
"post": {
"tags": [
@@ -5879,6 +6084,140 @@
}
},
"schemas": {
"PPConnectionDraft": {
"type": "object",
"required": [
"base_url",
"username",
"revision"
],
"additionalProperties": false,
"properties": {
"base_url": {
"type": "string",
"format": "uri",
"description": "HTTPS API 根地址"
},
"username": {
"type": "string",
"maxLength": 255
},
"password": {
"type": "string",
"writeOnly": true,
"maxLength": 8192,
"description": "省略或空值保留已存密码"
},
"revision": {
"type": "integer",
"minimum": 0
},
"pending_id": {
"type": "string",
"description": "继续校验已保存的待验证修改"
}
}
},
"PPSafeMetadata": {
"type": "object",
"required": [
"id",
"base_url",
"username",
"password_configured"
],
"properties": {
"id": {
"type": "string"
},
"base_url": {
"type": "string",
"format": "uri"
},
"username": {
"type": "string"
},
"password_configured": {
"type": "boolean"
}
}
},
"PPObservation": {
"type": "object",
"required": [
"state",
"message",
"checked_at"
],
"properties": {
"state": {
"type": "string",
"enum": [
"valid",
"auth_failed",
"permission_denied",
"incomplete",
"unavailable",
"invalid_response"
]
},
"message": {
"type": "string"
},
"checked_at": {
"type": "string",
"format": "date-time"
}
}
},
"PPConnectionView": {
"type": "object",
"properties": {
"revision": {
"type": "integer"
},
"active": {
"anyOf": [
{
"$ref": "#/components/schemas/PPSafeMetadata"
},
{
"type": "null"
}
]
},
"pending": {
"anyOf": [
{
"$ref": "#/components/schemas/PPSafeMetadata"
},
{
"type": "null"
}
]
},
"observation": {
"anyOf": [
{
"$ref": "#/components/schemas/PPObservation"
},
{
"type": "null"
}
]
},
"pending_observation": {
"anyOf": [
{
"$ref": "#/components/schemas/PPObservation"
},
{
"type": "null"
}
]
}
}
},
"QAReportedHistoricalRate": {
"type": "object",
"additionalProperties": false,
+14
View File
@@ -21,6 +21,7 @@ import (
"knowledge-governance/internal/monitoring"
postgresstore "knowledge-governance/internal/platform/postgres"
"knowledge-governance/internal/platform/weknora"
"knowledge-governance/internal/ppauth"
"knowledge-governance/internal/qaopsai"
"knowledge-governance/internal/qaopsorg"
"knowledge-governance/internal/qaopsquality"
@@ -193,7 +194,20 @@ func run(ctx context.Context, getenv func(string) string) error {
return fmt.Errorf("快照搬运接线无效: %w", handoffErr)
}
log.Printf("library target evidence identity=%s", config.LibraryTargetIdentity())
ppStateFile := getenv("PP_AUTH_STATE_FILE")
if ppStateFile == "" {
ppStateFile = "/data/source-auth/pp/settings.json"
}
ppPython := getenv("PP_AUTH_PYTHON")
if ppPython == "" {
ppPython = "/usr/bin/python3"
}
ppScript := getenv("PP_AUTH_CHECK_SCRIPT")
if ppScript == "" {
ppScript = "/app/scripts/pp_client.py"
}
handler, err := httpapi.NewHandler(httpapi.Config{
PPConnection: &ppauth.Manager{Path: ppStateFile, LegacyPath: getenv("PP_LEGACY_CREDENTIALS_FILE"), Probe: ppauth.CommandProbe(ppPython, ppScript)},
LibraryHandoffEnabled: handoffConfigured,
QAAI: qaopsai.NewFromEnvironment(pool, getenv, config.DatabaseURL),
QAWeeklyMail: qawemail.NewStore(pool, qaSettings.TenantKey, getenv("QA_AI_MASTER_KEY_FILE"), nil),
+4
View File
@@ -34,6 +34,10 @@ API 与 worker 必须使用相同 UID,以及同一组可写挂载:
- 每次只运行一个授权,防止覆盖共享状态。服务重启会结束在途授权;页面提示重新发起。
- CLI 成功退出后必须再次读到有效本地凭据才显示登录成功;真实目录访问另行检查。
## 添加来源
添加来源向导与账号卡共用官方 DWS 授权;列举、校验、登记使用同一身份。来源登记不再自动回落旧扫码侧车。保存中授权失效时保留已成功项并恢复剩余项;换账号或组织后需重新确认范围。详细交互、认证审计与 PP 接线见 [数据源认证配置](SOURCE_AUTH_CONFIGURATION.md)。
## 验证边界
自动回归覆盖授权生成、成功后状态更新、目录访问失败与部分可用、取消与进程退出、过期、重试、跨管理员/会话隔离、同源限制、官方链接校验及元数据脱敏。
+57
View File
@@ -0,0 +1,57 @@
# 数据源认证配置
管理员入口:系统设置 → 数据源配置。PP 连接位于钉钉账号卡旁;添加来源向导使用同一组账号与配置。
## PP 连接
- API 地址填写 HTTPS 根地址;后台沿用 `/admin/login/token`、`/potal/file/back/page` 接口路径。`potal` 是上游真实路径,不能改写。
- 账号允许修改。密码只显示已配置/未配置;点击“更新密码”录入新值。省略或留空保留既有密码,不回显原值。
- “测试连接”使用当前表单,真实登录并读取一条目录记录;合法空目录也可通过。它不保存配置、不登记来源、不启动扫描。
- “保存并校验”在后端重新检查本次提交。通过后切换生效;明确的认证拒绝、权限拒绝、接口不兼容阻止生效。
- 超时、限流或服务临时故障可保存为待验证修改;缺少凭据可保存未完成配置。两者都不会替换当前生效配置;需要再次保存并校验,通过后才切换。
- 草稿测试结果与当前生效配置状态分开;每次修改字段会使原草稿测试结果作废。检查失败时保留上次观测及时间,不能把网络失败当成退出登录。
- 后续目录扫描和远程取件读取同一生效配置;验证过的本地文件仍按既有内容校验逻辑复用。配置保存不恢复暂停的计划、不触发入库或发布。
## PP 服务端接线
| 参数 | 默认与用途 |
|---|---|
| `PP_AUTH_STATE_FILE` | `/data/source-auth/pp/settings.json`;API 和 worker 必须读同一份原子配置文件 |
| `PP_AUTH_PYTHON` | `/usr/bin/python3`;API 连接检查使用的解释器 |
| `PP_AUTH_CHECK_SCRIPT` | `/app/scripts/pp_client.py`;与扫描、取件共享的只读协议实现 |
| `PP_LEGACY_CREDENTIALS_FILE` | 可选,只用于首次迁移;有效状态文件存在后不再回落旧文件 |
配置目录及状态文件分别使用 0700 和 0600。API/worker 使用相同 UID,将宿主凭据目录挂载为共享目录,不能绑定单个文件:单文件 bind mount 会在原子 rename 后继续指向旧 inode。迁移时从目标机既有凭据文件直接读取,禁止通过聊天、命令行参数或仓库传递密码。
状态文件持有生效配置、待验证修改、最近检查及认证审计,包含密码,因此必须按密钥文件保护,不能进普通配置备份、版本记录或日志。更新使用跨进程文件锁、临时文件、fsync 和原子 rename。配置修订采用乐观并发控制;旧请求不能覆盖新修订。worker 的迟到检查只能更新自身生效配置 ID 对应的观测。
PP 检查子进程只从 stdin 接收本次凭据,stdout 只返回白名单检查结论。GET 接口仅返回配置元数据和是否存在密码;密码输入字段只可写。
| 接口 | 用途 |
|---|---|
| `GET /api/v1/pp-connection` | 管理员读取安全元数据、待验证修改与检查状态 |
| `POST /api/v1/pp-connection/test` | 测试当前草稿,不激活 |
| `PUT /api/v1/pp-connection` | 校验并保存;携带当前 `revision` |
使用已保存的待验证密码继续校验时携带 `pending_id`,避免空密码误用旧生效密码。修改请求验证管理员、同源操作意图和输入大小;响应禁止缓存。
## 钉钉添加向导
保留 类型 → 授权 → 范围 → 校验 → 保存 五步;授权步骤复用钉钉账号卡的官方 DWS 设备授权流程。
- 可见范围、校验和登记都使用共享 DWS 身份;登记不再自动回落旧扫码侧车。DWS 无法定位的历史空间返回明确原因。
- `POST /api/v1/dingtalk-spaces/validate` 真实读取并检查所选空间。失败、权限不足、部分范围不完整与合法空列表分别展示。
- 已接入空间按稳定空间身份标记,历史连接也从其存储的空间标识识别,不靠名称去重。
- 保存中授权失效:保留已成功项及待处理项,在向导内重新授权;同账号恢复后继续剩余项,复用操作标识,首次扫描按连接身份去重。
- 取消或失败后保留选择并停止自动提交;再次恢复失败不循环弹授权。
- 授权后换账号/组织:重新加载范围并清除旧选择,要求重新确认。客户端提交身份指纹,后端在列举前后确认身份,变化时拒绝登记。
- 只有已点击保存的操作才会在重新授权后继续登记;账号登录、刷新和连接检查本身不创建来源或扫描任务。
- 登记成功和首次扫描入队分别反馈,不能用登记成功冒充扫描已运行。
其他历史入口若仍调用旧扫码侧车,保留其原行为;本功能不删除整个侧车服务。
## 审计与验证
PP 配置/密码更新、连接检查及 DWS 授权结果,复用现有数据源操作审计页面。凭据事件与配置在同一文件内原子提交,并与数据库原有事件按时间合并分页;不把密码写入 SQL。DWS 终态审计按授权 ID 与状态去重,轮询不刷记录。
回归覆盖真实协议请求、空目录、认证拒绝、权限与限流、密码保留/替换、待验证密码重校验、并发修订冲突、响应脱敏、同源及角色限制、无意外任务、审计分页、DWS 部分成功恢复与换账号保护。协议与消费者测试使用隔离的本机 HTTP 服务;运行环境可用性需另行通过真实页面检查,不能用这些用例替代。
+18 -8
View File
@@ -2,12 +2,15 @@ package dingtalkdiscovery
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
)
// CredentialStatus contains metadata only, never credential values.
type CredentialStatus struct {
Identity string `json:"identity,omitempty"`
Authenticated bool `json:"authenticated"`
RefreshTokenValid bool `json:"refresh_token_valid"`
ExpiresAt string `json:"expires_at,omitempty"`
@@ -23,16 +26,23 @@ func ReadCredentialStatus(ctx context.Context, runner Runner) (CredentialStatus,
return CredentialStatus{}, errors.New("could not read DWS credential status")
}
var decoded struct {
Authenticated *bool `json:"authenticated"`
RefreshTokenValid bool `json:"refresh_token_valid"`
ExpiresAt string `json:"expires_at"`
RefreshExpiresAt string `json:"refresh_expires_at"`
CorpName string `json:"corp_name"`
UserName string `json:"user_name"`
Reason string `json:"reason"`
CorpID json.RawMessage `json:"corp_id"`
UserID json.RawMessage `json:"user_id"`
Authenticated *bool `json:"authenticated"`
RefreshTokenValid bool `json:"refresh_token_valid"`
ExpiresAt string `json:"expires_at"`
RefreshExpiresAt string `json:"refresh_expires_at"`
CorpName string `json:"corp_name"`
UserName string `json:"user_name"`
Reason string `json:"reason"`
}
if json.Unmarshal(raw, &decoded) != nil || decoded.Authenticated == nil {
return CredentialStatus{}, errors.New("invalid DWS credential status")
}
return CredentialStatus{Authenticated: *decoded.Authenticated, RefreshTokenValid: decoded.RefreshTokenValid, ExpiresAt: decoded.ExpiresAt, RefreshExpiresAt: decoded.RefreshExpiresAt, CorpName: decoded.CorpName, UserName: decoded.UserName, StatusKnown: decoded.Reason == ""}, nil
identity := ""
if len(decoded.CorpID) > 0 && len(decoded.UserID) > 0 && string(decoded.CorpID) != "null" && string(decoded.UserID) != "null" {
sum := sha256.Sum256(append(append(decoded.CorpID, 0), decoded.UserID...))
identity = hex.EncodeToString(sum[:])
}
return CredentialStatus{Identity: identity, Authenticated: *decoded.Authenticated, RefreshTokenValid: decoded.RefreshTokenValid, ExpiresAt: decoded.ExpiresAt, RefreshExpiresAt: decoded.RefreshExpiresAt, CorpName: decoded.CorpName, UserName: decoded.UserName, StatusKnown: decoded.Reason == ""}, nil
}
+27 -9
View File
@@ -21,6 +21,7 @@ import (
// KnowledgeBase 是一个钉钉知识库(wiki 空间)。URL 是 alidocs 空间链接,直接可用于登记来源。
type KnowledgeBase struct {
Registered bool `json:"registered"`
Name string `json:"name"`
SpaceID string `json:"space_id"`
SpaceType string `json:"space_type"`
@@ -30,6 +31,7 @@ type KnowledgeBase struct {
// DriveSpace 是一个钉盘空间(我的文件 = mySpace;组织空间 = orgSpace)。
type DriveSpace struct {
Registered bool `json:"registered"`
Name string `json:"name"`
SpaceID string `json:"space_id"`
RootFolderID string `json:"root_folder_id,omitempty"`
@@ -38,6 +40,7 @@ type DriveSpace struct {
// Result 是一次发现的结果。
type Result struct {
Identity string `json:"identity,omitempty"`
KnowledgeBases []KnowledgeBase `json:"knowledge_bases"`
DriveSpaces []DriveSpace `json:"drive_spaces"`
// KnowledgeNotice / DriveNotice 说明「这一类为什么不全」:权限不足、未登录、或被上限截断。
@@ -70,21 +73,26 @@ func SpaceListArgs(spaceType string) []string {
// 知识库整体列不出来 → 返回 ErrUnavailable(页面必须提示「列不出来」,不能装成空列表)。
// 单独一类失败、钉盘失败、或结果被上限截断 → 只丢那一部分,把原因写进对应的 Notice。
func Discover(ctx context.Context, runner Runner) (Result, error) {
result := Result{FetchedAt: time.Now().UTC()}
result := Result{FetchedAt: time.Now().UTC(), KnowledgeBases: []KnowledgeBase{}, DriveSpaces: []DriveSpace{}}
successful := 0
causes := []error{}
failures := make([]string, 0, len(knowledgeSpaceTypes))
seenKnowledge := map[string]bool{}
for _, spaceType := range knowledgeSpaceTypes {
raw, err := runner.RunDWS(ctx, SpaceListArgs(spaceType)...)
if err != nil {
failures = append(failures, describe(spaceType, err))
failures = append(failures, describe(spaceType, failureCause(raw, err)))
causes = append(causes, failureCause(raw, err))
continue
}
bases, notice, err := parseKnowledgeBases(raw, spaceType)
if err != nil {
failures = append(failures, describe(spaceType, err))
failures = append(failures, describe(spaceType, failureCause(raw, err)))
causes = append(causes, failureCause(raw, err))
continue
}
successful++
if notice != "" {
result.KnowledgeNotice = notice
}
@@ -96,9 +104,6 @@ func Discover(ctx context.Context, runner Runner) (Result, error) {
result.KnowledgeBases = append(result.KnowledgeBases, base)
}
}
if len(result.KnowledgeBases) == 0 && len(failures) > 0 {
return result, fmt.Errorf("%w: %s", ErrUnavailable, strings.Join(failures, ";"))
}
if len(failures) > 0 {
result.KnowledgeNotice = strings.Join(append([]string{result.KnowledgeNotice}, failures...), ";")
}
@@ -107,14 +112,17 @@ func Discover(ctx context.Context, runner Runner) (Result, error) {
for _, spaceType := range []string{"mySpace", "orgSpace"} {
raw, err := runner.RunDWS(ctx, "wiki", "space", "list", "--type", spaceType)
if err != nil {
result.DriveNotice = err.Error()
result.DriveNotice = "钉盘目录暂时不可用,请检查该范围权限"
causes = append(causes, failureCause(raw, err))
continue
}
spaces, err := parseDriveSpaces(raw, spaceType)
if err != nil {
result.DriveNotice = err.Error()
result.DriveNotice = "钉盘目录暂时不可用,请检查该范围权限"
causes = append(causes, failureCause(raw, err))
continue
}
successful++
for _, space := range spaces {
if seenDrive[space.SpaceType+":"+space.SpaceID] {
continue
@@ -123,11 +131,21 @@ func Discover(ctx context.Context, runner Runner) (Result, error) {
result.DriveSpaces = append(result.DriveSpaces, space)
}
}
if successful == 0 && len(causes) > 0 {
return result, fmt.Errorf("%w: %w", ErrUnavailable, errors.Join(causes...))
}
return result, nil
}
func describe(scope string, err error) string {
return scope + ":" + err.Error()
reason := "读取失败,请重试"
if errors.Is(err, ErrCredentials) {
reason = "DWS 授权已失效"
}
if errors.Is(err, ErrPermission) {
reason = "当前范围权限不足"
}
return scope + ":" + reason
}
// parseKnowledgeBases 解析 `dws wiki +space-list`:
+1 -1
View File
@@ -127,7 +127,7 @@ func TestDiscoverKeepsKnowledgeBasesWhenDriveIsDenied(t *testing.T) {
if len(result.DriveSpaces) != 0 {
t.Fatalf("drive spaces = %d, want 0", len(result.DriveSpaces))
}
if !strings.Contains(result.DriveNotice, "permission denied") {
if !strings.Contains(result.DriveNotice, "权限") {
t.Fatalf("drive notice = %q", result.DriveNotice)
}
}
+40
View File
@@ -0,0 +1,40 @@
package dingtalkdiscovery
import (
"encoding/json"
"errors"
"fmt"
"strings"
)
var ErrCredentials = errors.New("DWS authorization required")
var ErrPermission = errors.New("DWS directory permission denied")
// failureCause inspects only structured CLI errors; no upstream text is returned.
func failureCause(raw []byte, cause error) error {
var v struct {
Error struct {
Type string `json:"type"`
Code any `json:"code"`
Message string `json:"message"`
} `json:"error"`
}
if json.Unmarshal(raw, &v) != nil {
return ErrUnavailable
}
text := strings.ToLower(v.Error.Type + " " + fmt.Sprint(v.Error.Code) + " " + v.Error.Message)
for _, key := range []string{"permission", "forbidden", "scope", "access denied", "权限", "403"} {
if strings.Contains(text, key) {
return ErrPermission
}
}
for _, key := range []string{"unauth", "not_logged_in", "not logged", "login_required", "login required", "authentication", "auth_required", "credential", "token_expired", "invalid_token", "token expired", "invalid_grant", "未登录", "授权已过期", "授权失效"} {
if strings.Contains(text, key) {
return ErrCredentials
}
}
if errors.Is(cause, ErrCredentials) || errors.Is(cause, ErrPermission) {
return cause
}
return ErrUnavailable
}
+1 -1
View File
@@ -107,7 +107,7 @@ func classifyRole(method, path string) string {
if strings.HasPrefix(path, "/api/v1/users") || strings.HasPrefix(path, "/api/v1/qa-ops/") {
return roleAdmin
}
if strings.HasPrefix(path, "/api/v1/dws-login") {
if strings.HasPrefix(path, "/api/v1/dws-login") || strings.HasPrefix(path, "/api/v1/pp-connection") || strings.HasPrefix(path, "/api/v1/dingtalk-spaces") || strings.HasPrefix(path, "/api/v1/dingtalk-session") {
return roleAdmin
}
if method == http.MethodGet {
+63
View File
@@ -0,0 +1,63 @@
package httpapi
import (
"context"
postgresstore "knowledge-governance/internal/platform/postgres"
"knowledge-governance/internal/ppauth"
"sort"
)
// Credential events are committed with their secret revision in one atomic file.
// Merge them into the existing audit view without copying secrets into SQL.
func combinedSourceAudit(ctx context.Context, reader sourceAuditReader, manager *ppauth.Manager, limit, offset int) (postgresstore.SourceAuditPage, error) {
if manager == nil {
return reader.ListSourceAudit(ctx, limit, offset)
}
events, err := manager.Events(ctx)
if err != nil {
return postgresstore.SourceAuditPage{}, err
}
if len(events) == 0 {
return reader.ListSourceAudit(ctx, limit, offset)
}
start := offset - len(events)
if start < 0 {
start = 0
}
rows := make([]postgresstore.SourceAuditItem, 0, limit+len(events))
total := 0
for next := start; len(rows) < limit+len(events); {
take := limit + len(events) - len(rows)
if take > 100 {
take = 100
}
page, e := reader.ListSourceAudit(ctx, take, next)
if e != nil {
return postgresstore.SourceAuditPage{}, e
}
total = page.Total
rows = append(rows, page.Items...)
next += len(page.Items)
if next >= total || len(page.Items) == 0 {
break
}
}
for _, e := range events {
rows = append(rows, postgresstore.SourceAuditItem{ID: e.ID, ConnectionID: "", SourceName: e.SourceName, Action: e.Action, Actor: e.Actor, CreatedAt: e.CreatedAt, Detail: e.Detail})
}
sort.SliceStable(rows, func(i, j int) bool {
if rows[i].CreatedAt.Equal(rows[j].CreatedAt) {
return rows[i].ID > rows[j].ID
}
return rows[i].CreatedAt.After(rows[j].CreatedAt)
})
skip := offset - start
if skip > len(rows) {
skip = len(rows)
}
end := skip + limit
if end > len(rows) {
end = len(rows)
}
return postgresstore.SourceAuditPage{Items: rows[skip:end], Total: total + len(events), Limit: limit, Offset: offset}, nil
}
+5 -15
View File
@@ -2,7 +2,6 @@ package httpapi
import (
"context"
"errors"
"net/http"
"time"
@@ -25,25 +24,16 @@ func listDingtalkSources(w http.ResponseWriter, r *http.Request, config Config)
if _, ok := requireSessionIdentity(w, r); !ok {
return
}
lister := config.DingtalkSources
if lister == nil {
writeJSON(w, http.StatusNotImplemented, map[string]string{"error": "dingtalk source discovery is not wired"})
identity, _ := requireSessionIdentity(w, r)
if identity.Role != roleAdmin {
writeJSON(w, 403, map[string]string{"error": "需要管理员身份列举数据源"})
return
}
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
defer cancel()
result, err := lister.ListAvailableDingtalkSources(ctx)
result, err := availableDWS(ctx, config)
if err != nil {
if errors.Is(err, dingtalkdiscovery.ErrUnavailable) {
// 环境问题(dws 未登录 / 权限不足 / 二进制缺失):页面显示"暂时列不出来",
// 不是平台故障,也不该把内部错误细节当 500 抛出去。
writeJSON(w, http.StatusServiceUnavailable, map[string]string{
"error": "dingtalk_discovery_unavailable",
"hint": "dws 未登录或缺少租户目录读权限;请先刷新钉钉会话,或让管理员检查 dws 凭据。",
})
return
}
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "could not list dingtalk sources"})
writeDWSSourceError(w, err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"data": result})
+1 -1
View File
@@ -53,7 +53,7 @@ func TestListDingtalkSourcesReportsEnvironmentProblemsAsUnavailable(t *testing.T
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("unavailable = %d, want 503 body=%s", rec.Code, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "dingtalk_discovery_unavailable") {
if !strings.Contains(rec.Body.String(), "dws_unavailable") {
t.Fatalf("body = %s", rec.Body.String())
}
}
+18 -35
View File
@@ -2,59 +2,42 @@ package httpapi
import (
"context"
"knowledge-governance/internal/dingtalkdiscovery"
postgresstore "knowledge-governance/internal/platform/postgres"
"net/http"
"net/http/httptest"
"testing"
"time"
"knowledge-governance/internal/dingtalkdiscovery"
)
func TestDWSNameConflictIsNotASuccessfulRegistration(t *testing.T) {
store := &fakeDingtalkRegistrar{fakeJobStore: &fakeJobStore{}, err: postgresstore.ErrDingtalkSpaceNameTaken}
lister := fakeDingtalkSources{result: dingtalkdiscovery.Result{KnowledgeBases: []dingtalkdiscovery.KnowledgeBase{{Name: "wiki", SpaceID: "testspace"}}}}
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/api/v1/dingtalk-spaces", nil)
registerSpaceThroughDWS(rec, req, Config{Store: store, DingtalkSources: lister, Now: time.Now}, "testspace", "test-op")
if rec.Code != http.StatusConflict {
req := httptest.NewRequest("POST", "/api/v1/dingtalk-spaces", nil)
registerDWSOnlySpace(rec, req, Config{Store: store, DingtalkSources: lister, Now: time.Now}, "testspace", "test-op")
if rec.Code != 409 {
t.Fatalf("name conflict must not claim success: %d", rec.Code)
}
}
// 登记走 dws 不是侧车:解析出来的键、名称、根节点直接决定新来源的确定性 id。
func TestResolveSpaceViaDWSUsesTheSpaceCredentialsAlreadyHeld(t *testing.T) {
lister := fakeDingtalkSources{result: dingtalkdiscovery.Result{
KnowledgeBases: []dingtalkdiscovery.KnowledgeBase{
{Name: "国内技术支持管理", SpaceID: "9JOGOLBQndOyaz4Q"},
{Name: "视觉文档", SpaceID: "9JOGO9EEOMLMWX4Q"},
},
DriveSpaces: []dingtalkdiscovery.DriveSpace{
{Name: "全员文件夹", SpaceID: "102571304", RootFolderID: "g1YGl6Pm2Db8Dr3OayClVxLq0Ee4p79a"},
},
}}
rootKey, name, rootID, spaceID, ok := resolveSpaceViaDWS(
context.Background(), lister, dingtalkdiscovery.SpaceURL("9JOGOLBQndOyaz4Q"))
if !ok || rootKey != "space-9JOGOLBQndOyaz4Q" || name != "国内技术支持管理" || rootID != "9JOGOLBQndOyaz4Q" || spaceID != "9JOGOLBQndOyaz4Q" {
t.Fatalf("knowledge base resolved as key=%q name=%q root=%q space=%q ok=%v", rootKey, name, rootID, spaceID, ok)
result := dingtalkdiscovery.Result{KnowledgeBases: []dingtalkdiscovery.KnowledgeBase{{Name: "国内技术支持管理", SpaceID: "9JOGOLBQndOyaz4Q"}}, DriveSpaces: []dingtalkdiscovery.DriveSpace{{Name: "全员文件夹", SpaceID: "102571304", RootFolderID: "drive-root"}}}
input, err := resolveDWS(result, dingtalkdiscovery.SpaceURL("9JOGOLBQndOyaz4Q"))
if err != nil || input.RootKey != "space-9JOGOLBQndOyaz4Q" || input.Name != "国内技术支持管理" || input.RootID != "9JOGOLBQndOyaz4Q" || input.SpaceID != "9JOGOLBQndOyaz4Q" {
t.Fatalf("wiki resolution=%+v %v", input, err)
}
// 钉盘的根是 rootFolderId,不是空间 id:抓取侧按它遍历。
rootKey, _, rootID, _, ok = resolveSpaceViaDWS(context.Background(), lister, dingtalkdiscovery.SpaceURL("102571304"))
if !ok || rootID != "g1YGl6Pm2Db8Dr3OayClVxLq0Ee4p79a" || rootKey != "space-102571304" {
t.Fatalf("drive space resolved as key=%q root=%q ok=%v", rootKey, rootID, ok)
input, err = resolveDWS(result, dingtalkdiscovery.SpaceURL("102571304"))
if err != nil || input.RootID != "drive-root" || input.RootKey != "space-102571304" {
t.Fatalf("drive resolution=%+v %v", input, err)
}
}
func TestResolveSpaceViaDWSLeavesUnknownSpacesToTheFallback(t *testing.T) {
lister := fakeDingtalkSources{result: dingtalkdiscovery.Result{}}
if _, _, _, _, ok := resolveSpaceViaDWS(context.Background(), lister, dingtalkdiscovery.SpaceURL("unknownspace1")); ok {
t.Fatal("an unknown space must not resolve")
func TestResolveDWSRejectsUnknownAndUnwiredWithoutBrowserFallback(t *testing.T) {
if _, err := resolveDWS(dingtalkdiscovery.Result{}, dingtalkdiscovery.SpaceURL("unknownspace1")); err == nil {
t.Fatal("unknown space resolved")
}
if _, _, _, _, ok := resolveSpaceViaDWS(context.Background(), nil, dingtalkdiscovery.SpaceURL("9JOGOLBQndOyaz4Q")); ok {
t.Fatal("without a lister nothing can resolve")
if _, err := availableDWS(context.Background(), Config{}); err == nil {
t.Fatal("unwired service resolved")
}
if _, _, _, _, ok := resolveSpaceViaDWS(context.Background(), lister, "not-a-space-url"); ok {
t.Fatal("a URL without a space id must not resolve")
if _, err := resolveDWS(dingtalkdiscovery.Result{}, "not-a-space-url"); err == nil {
t.Fatal("invalid space resolved")
}
}
+4 -154
View File
@@ -2,9 +2,7 @@ package httpapi
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"log"
"net/http"
@@ -12,10 +10,6 @@ import (
"regexp"
"strings"
"time"
"knowledge-governance/internal/governance"
"knowledge-governance/internal/jobs"
postgresstore "knowledge-governance/internal/platform/postgres"
)
// DingTalk login proxy. The scan-code login itself lives in the
@@ -102,86 +96,15 @@ func registerDingtalkSpace(w http.ResponseWriter, r *http.Request, config Config
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "url is required"})
return
}
// dws 已经持有有效凭据(access 自动续期、refresh 一个月),登记也该走它:
// 只有 dws 解析不出这个空间时,才回落到老的扫码侧车。
if registered, ok := registerSpaceThroughDWS(w, r, config, strings.TrimSpace(input.URL), operationID); ok || registered {
return
}
payload, _ := json.Marshal(map[string]string{"url": strings.TrimSpace(input.URL)})
status, data := proxyDingtalk(http.MethodPost, "/spaces/add", payload)
if status != http.StatusOK {
if status == 0 {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "dingtalk_login_unavailable"})
return
}
// Relay the sidecar's error (login_required / space_not_found /
// name_conflict) verbatim; bodies are machine codes, no secrets.
writeProxyResult(w, status, data)
return
}
var resolved struct {
Key string `json:"key"`
Name string `json:"name"`
Root string `json:"root"`
}
if err := json.Unmarshal(data, &resolved); err != nil || resolved.Key == "" || resolved.Name == "" || resolved.Root == "" {
writeJSON(w, http.StatusBadGateway, map[string]string{"error": "invalid sidecar registration response"})
return
}
store, ok := config.Store.(interface {
RegisterDingtalkSpace(context.Context, postgresstore.RegisterDingtalkSpaceInput) (string, error)
})
if !ok {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "registration_unavailable"})
return
}
spaceID := extractDingtalkSpaceID(input.URL)
if spaceID == "" {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid space url"})
return
}
connectionID, err := store.RegisterDingtalkSpace(r.Context(), postgresstore.RegisterDingtalkSpaceInput{
RootKey: resolved.Key, Name: resolved.Name, SpaceID: spaceID, RootID: resolved.Root, Now: config.Now(), Actor: actorFor(r),
})
switch {
case errors.Is(err, postgresstore.ErrDingtalkSpaceNameTaken):
writeJSON(w, http.StatusConflict, map[string]string{
"error": "space_name_conflict",
"hint": "已有同名来源仍在使用(未下架):请先在该来源卡片上「去掉来源」,或确认选的是同一个知识库空间。",
})
return
case errors.Is(err, postgresstore.ErrDingtalkSpaceIdentityConflict):
writeJSON(w, http.StatusConflict, map[string]string{
"error": "space_identity_conflict",
"hint": "这个登记键已指向另一个知识库空间:请刷新列表后重试;若仍失败,把该来源「去掉来源」后再登记。",
})
return
case err != nil:
// 真正的数据库/内部故障不能伪装成"冲突":换个 500 才有人去查日志。
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "space_registration_failed"})
return
}
// First scan is enqueued right away: the new connection has no catalog
// head yet, so the generic sources/{id}/refresh endpoint would 404 until
// the first snapshot exists.
if enqueuer, ok := config.Store.(interface {
EnqueueCatalogRefreshWithOperation(context.Context, governance.CatalogSource, string, time.Time) (jobs.Job, bool, error)
}); ok {
if _, _, enqueueErr := enqueuer.EnqueueCatalogRefreshWithOperation(r.Context(), governance.CatalogSource{
ConnectionID: connectionID, SourceType: governance.SourceTypeDingTalk, Name: resolved.Name,
}, operationID+":first", config.Now().UTC()); enqueueErr != nil {
log.Printf("dingtalk space %s registered but first refresh enqueue failed: %v", connectionID, enqueueErr)
}
}
log.Printf("dingtalk space %q registered as connection %s by %s", resolved.Name, connectionID, actorFor(r))
writeJSON(w, http.StatusCreated, map[string]any{"data": map[string]string{
"connection_id": connectionID, "name": resolved.Name,
}})
registerDWSOnlySpace(w, r, config, strings.TrimSpace(input.URL), operationID)
}
var dingtalkSpacePattern = regexp.MustCompile(`/spaces/([A-Za-z0-9_-]+)`)
func extractDingtalkSpaceID(raw string) string {
if !validDingtalkURL(raw) {
return ""
}
match := dingtalkSpacePattern.FindStringSubmatch(raw)
if len(match) == 2 {
return match[1]
@@ -244,76 +167,3 @@ func proxyDingtalk(method, path string, body []byte) (int, []byte) {
}
return resp.StatusCode, data
}
// resolveSpaceViaDWS 用 dws 列举结果定位一个空间:返回确定性登记键、名称与根节点。
// 认不出就 ok=false,调用方回落到老的扫码侧车(历史空间仍能被登记)。
func resolveSpaceViaDWS(ctx context.Context, lister dingtalkSourcesLister, rawURL string) (rootKey, name, rootID, spaceID string, ok bool) {
spaceID = extractDingtalkSpaceID(rawURL)
if lister == nil || spaceID == "" {
return "", "", "", "", false
}
discovered, err := lister.ListAvailableDingtalkSources(ctx)
if err != nil {
return "", "", "", "", false
}
for _, base := range discovered.KnowledgeBases {
if base.SpaceID == spaceID {
// 知识库:URL 里的 id 就是空间 id,抓取侧按 spaces/<id> 匹配。
return "space-" + spaceID, base.Name, spaceID, spaceID, true
}
}
for _, space := range discovered.DriveSpaces {
if space.SpaceID == spaceID && space.RootFolderID != "" {
return "space-" + spaceID, space.Name, space.RootFolderID, spaceID, true
}
}
return "", "", "", "", false
}
// registerSpaceThroughDWS 用 dws 凭据直接登记来源,不再要求扫码会话。
func registerSpaceThroughDWS(w http.ResponseWriter, r *http.Request, config Config, rawURL, operationID string) (bool, bool) {
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
defer cancel()
rootKey, name, rootID, spaceID, ok := resolveSpaceViaDWS(ctx, config.DingtalkSources, rawURL)
if !ok {
return false, false
}
store, wired := config.Store.(interface {
RegisterDingtalkSpace(context.Context, postgresstore.RegisterDingtalkSpaceInput) (string, error)
})
if !wired {
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "registration_unavailable"})
return true, false
}
connectionID, err := store.RegisterDingtalkSpace(ctx, postgresstore.RegisterDingtalkSpaceInput{
RootKey: rootKey, Name: name, SpaceID: spaceID, RootID: rootID, Now: config.Now(), Actor: actorFor(r),
})
switch {
case errors.Is(err, postgresstore.ErrDingtalkSpaceNameTaken):
writeJSON(w, http.StatusConflict, map[string]string{"error": "space_name_conflict", "hint": "已有同名来源,但空间标识不一致;请核对已有来源,不能按名称判定添加成功。"})
return true, false
case errors.Is(err, postgresstore.ErrDingtalkSpaceIdentityConflict):
writeJSON(w, http.StatusConflict, map[string]string{
"error": "space_identity_conflict",
"hint": "这个登记键已指向另一个知识库空间:请刷新列表后重试。",
})
return true, false
case err != nil:
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": "space_registration_failed"})
return true, false
}
if enqueuer, wired := config.Store.(interface {
EnqueueCatalogRefreshWithOperation(context.Context, governance.CatalogSource, string, time.Time) (jobs.Job, bool, error)
}); wired {
if _, _, enqueueErr := enqueuer.EnqueueCatalogRefreshWithOperation(ctx, governance.CatalogSource{
ConnectionID: connectionID, SourceType: governance.SourceTypeDingTalk, Name: name,
}, operationID+":first", config.Now().UTC()); enqueueErr != nil {
log.Printf("dingtalk space %s registered but first refresh enqueue failed: %v", connectionID, enqueueErr)
}
}
log.Printf("dingtalk space %q registered as connection %s by %s (dws)", name, connectionID, actorFor(r))
writeJSON(w, http.StatusCreated, map[string]any{"data": map[string]string{
"connection_id": connectionID, "name": name,
}})
return true, true
}
+2 -1
View File
@@ -9,6 +9,7 @@ import (
"strings"
"testing"
"knowledge-governance/internal/dingtalkdiscovery"
postgresstore "knowledge-governance/internal/platform/postgres"
)
@@ -123,7 +124,7 @@ func TestDingtalkSpaceRegistrationErrorClassification(t *testing.T) {
call := func(storeErr error) (int, string) {
store := &fakeDingtalkRegistrar{fakeJobStore: &fakeJobStore{}, err: storeErr}
handler, err := NewHandler(Config{Store: store})
handler, err := NewHandler(Config{Store: store, DingtalkSources: fakeDingtalkSources{result: dingtalkdiscovery.Result{KnowledgeBases: []dingtalkdiscovery.KnowledgeBase{{Name: "公司级技术支持知识库", SpaceID: "9JOK9EEOMLMWX4Q"}}}}})
if err != nil {
t.Fatalf("NewHandler() error = %v", err)
}
+9
View File
@@ -63,6 +63,15 @@ func handleDWSLogin(w http.ResponseWriter, r *http.Request, config Config, actio
}
return
}
if config.PPConnection != nil && (action != "get" || view.State == "succeeded" || view.State == "failed" || view.State == "expired") {
if err := config.PPConnection.AuditOnce(r.Context(), "dws-"+view.ID+"-"+string(view.State), "钉钉账号(DWS)", "dws_authorization", identity.Username, "DWS 授权状态:"+string(view.State)); err != nil {
if action == "start" {
_, _ = config.DWSLogin.Cancel(owner, view.ID)
}
writeJSON(w, 503, map[string]string{"error": "授权审计写入失败,请重试"})
return
}
}
if action != "get" {
log.Printf("DWS authorization action=%s id=%s actor=%s", action, view.ID, identity.Username)
}
+212
View File
@@ -0,0 +1,212 @@
package httpapi
import (
"context"
"errors"
"knowledge-governance/internal/dingtalkdiscovery"
"knowledge-governance/internal/governance"
"knowledge-governance/internal/jobs"
postgresstore "knowledge-governance/internal/platform/postgres"
"net/http"
"net/url"
"strings"
"time"
)
type dwsSourceError struct {
Status int
Code, Hint string
}
func (e *dwsSourceError) Error() string { return e.Hint }
func writeDWSSourceError(w http.ResponseWriter, err error) {
var e *dwsSourceError
if errors.As(err, &e) {
writeJSON(w, e.Status, map[string]string{"error": e.Code, "hint": e.Hint})
return
}
writeJSON(w, 500, map[string]string{"error": "dws_source_failed", "hint": "读取钉钉来源失败,请重试"})
}
func availableDWS(ctx context.Context, config Config) (dingtalkdiscovery.Result, error) {
identity := ""
if config.DingtalkSources == nil {
return dingtalkdiscovery.Result{}, &dwsSourceError{501, "dws_unwired", "DWS 来源服务未接通"}
}
if reader, ok := config.DingtalkSources.(interface {
ReadDWSMetadata(context.Context) (dingtalkdiscovery.CredentialStatus, error)
}); ok {
status, err := reader.ReadDWSMetadata(ctx)
if err != nil || !status.StatusKnown {
return dingtalkdiscovery.Result{}, &dwsSourceError{503, "dws_status_unavailable", "暂时无法读取 DWS 授权状态,请重试"}
}
identity = status.Identity
if !status.Authenticated && !status.RefreshTokenValid {
return dingtalkdiscovery.Result{}, &dwsSourceError{401, "dws_login_required", "DWS 未登录或授权已失效,请重新授权"}
}
}
result, err := config.DingtalkSources.ListAvailableDingtalkSources(ctx)
if err != nil {
if errors.Is(err, dingtalkdiscovery.ErrCredentials) {
return result, &dwsSourceError{401, "dws_login_required", "DWS 授权已失效,请重新授权"}
}
if errors.Is(err, dingtalkdiscovery.ErrPermission) {
return result, &dwsSourceError{403, "dws_permission_denied", "当前 DWS 账号缺少目录读取权限"}
}
if errors.Is(err, dingtalkdiscovery.ErrUnavailable) {
return result, &dwsSourceError{503, "dws_unavailable", "DWS 目录暂时无法读取,请检查连接后重试"}
}
return result, err
}
if identity != "" {
reader := config.DingtalkSources.(interface {
ReadDWSMetadata(context.Context) (dingtalkdiscovery.CredentialStatus, error)
})
fresh, e := reader.ReadDWSMetadata(ctx)
if e != nil {
return result, &dwsSourceError{503, "dws_status_unavailable", "无法确认当前账号,请重试"}
}
if fresh.Identity != identity {
return result, &dwsSourceError{409, "dws_identity_changed", "DWS 账号或组织已变化,请重新确认来源"}
}
}
result.Identity = identity
if config.Store != nil {
if page, e := config.Store.ListSourceConnections(ctx); e == nil {
seen := map[string]bool{}
for _, c := range page.Items {
seen[c.ID] = true
}
for i, kb := range result.KnowledgeBases {
in := postgresstore.RegisterDingtalkSpaceInput{RootKey: "space-" + kb.SpaceID}
result.KnowledgeBases[i].Registered = seen[in.ConnectionID()]
}
for i, space := range result.DriveSpaces {
in := postgresstore.RegisterDingtalkSpaceInput{RootKey: "space-" + space.SpaceID}
result.DriveSpaces[i].Registered = seen[in.ConnectionID()]
}
}
}
if registry, ok := config.Store.(interface {
RegisteredDingtalkSpaceIDs(context.Context) (map[string]bool, error)
}); ok {
ids, e := registry.RegisteredDingtalkSpaceIDs(ctx)
if e != nil {
return result, &dwsSourceError{503, "dws_registration_state_unavailable", "无法核对已接入空间,请重试"}
}
for i, kb := range result.KnowledgeBases {
result.KnowledgeBases[i].Registered = ids[kb.SpaceID]
}
for i, space := range result.DriveSpaces {
result.DriveSpaces[i].Registered = ids[space.SpaceID]
}
}
return result, nil
}
func resolveDWS(result dingtalkdiscovery.Result, raw string) (postgresstore.RegisterDingtalkSpaceInput, error) {
spaceID := extractDingtalkSpaceID(raw)
if spaceID == "" {
return postgresstore.RegisterDingtalkSpaceInput{}, &dwsSourceError{400, "invalid_space_url", "请输入有效的钉钉空间地址"}
}
for _, base := range result.KnowledgeBases {
if base.SpaceID == spaceID {
return postgresstore.RegisterDingtalkSpaceInput{RootKey: "space-" + spaceID, Name: base.Name, RootID: spaceID, SpaceID: spaceID}, nil
}
}
for _, space := range result.DriveSpaces {
if space.SpaceID == spaceID && space.RootFolderID != "" {
return postgresstore.RegisterDingtalkSpaceInput{RootKey: "space-" + spaceID, Name: space.Name, RootID: space.RootFolderID, SpaceID: spaceID}, nil
}
}
if result.KnowledgeNotice != "" || result.DriveNotice != "" {
return postgresstore.RegisterDingtalkSpaceInput{}, &dwsSourceError{503, "dws_scope_incomplete", "当前目录结果不完整,无法确认所选空间;请检查相应目录权限后重试"}
}
return postgresstore.RegisterDingtalkSpaceInput{}, &dwsSourceError{409, "dws_space_not_visible", "当前 DWS 账号已看不到所选空间,请刷新范围或核对账号权限"}
}
func registerDWSOnlySpace(w http.ResponseWriter, r *http.Request, config Config, raw, operationID string) {
if extractDingtalkSpaceID(raw) == "" {
writeDWSSourceError(w, &dwsSourceError{400, "invalid_space_url", "请输入有效的钉钉空间地址"})
return
}
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
defer cancel()
result, err := availableDWS(ctx, config)
if err != nil {
writeDWSSourceError(w, err)
return
}
if wanted := r.Header.Get("X-DWS-Identity"); wanted != "" && wanted != result.Identity {
writeDWSSourceError(w, &dwsSourceError{409, "dws_identity_changed", "DWS 账号或组织已变化,请重新确认来源"})
return
}
input, err := resolveDWS(result, raw)
if err != nil {
writeDWSSourceError(w, err)
return
}
input.Now = config.Now()
input.Actor = actorFor(r)
store, ok := config.Store.(interface {
RegisterDingtalkSpace(context.Context, postgresstore.RegisterDingtalkSpaceInput) (string, error)
})
if !ok {
writeJSON(w, 503, map[string]string{"error": "registration_unavailable"})
return
}
connectionID, err := store.RegisterDingtalkSpace(ctx, input)
switch {
case errors.Is(err, postgresstore.ErrDingtalkSpaceNameTaken):
writeJSON(w, 409, map[string]string{"error": "space_name_conflict", "hint": "已有同名来源仍在使用:请核对空间身份,必要时先移除旧连接(原去掉来源操作)"})
return
case errors.Is(err, postgresstore.ErrDingtalkSpaceIdentityConflict):
writeJSON(w, 409, map[string]string{"error": "space_identity_conflict", "hint": "登记键已指向另一个空间,请刷新后核对来源"})
return
case err != nil:
writeJSON(w, 500, map[string]string{"error": "space_registration_failed"})
return
}
queued := false
if queue, ok := config.Store.(interface {
EnqueueCatalogRefreshWithOperation(context.Context, governance.CatalogSource, string, time.Time) (jobs.Job, bool, error)
}); ok {
_, _, err = queue.EnqueueCatalogRefreshWithOperation(ctx, governance.CatalogSource{ConnectionID: connectionID, SourceType: governance.SourceTypeDingTalk, Name: input.Name}, "dws-source-first:"+connectionID, config.Now().UTC())
queued = err == nil
}
writeJSON(w, 201, map[string]any{"data": map[string]any{"connection_id": connectionID, "name": input.Name, "first_scan_queued": queued}})
}
func validateDWSSpace(w http.ResponseWriter, r *http.Request, config Config) {
if !credentialAdmin(w, r, "dingtalk-space") {
return
}
var input struct {
URLs []string `json:"urls"`
}
if decodeJSONStrict(r, &input) != nil || len(input.URLs) == 0 || len(input.URLs) > 100 {
writeJSON(w, 400, map[string]string{"error": "请选择有效的空间范围"})
return
}
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
defer cancel()
result, err := availableDWS(ctx, config)
if err != nil {
writeDWSSourceError(w, err)
return
}
if wanted := r.Header.Get("X-DWS-Identity"); wanted != "" && wanted != result.Identity {
writeDWSSourceError(w, &dwsSourceError{409, "dws_identity_changed", "DWS 账号或组织已变化,请重新确认来源"})
return
}
for _, raw := range input.URLs {
if _, err := resolveDWS(result, raw); err != nil {
writeDWSSourceError(w, err)
return
}
}
writeJSON(w, 200, map[string]any{"data": map[string]any{"valid": true, "checked_at": config.Now().UTC(), "count": len(input.URLs)}})
}
func validDingtalkURL(raw string) bool {
if !strings.Contains(raw, "://") {
return true
}
u, err := url.Parse(raw)
return err == nil && u.Scheme == "https" && u.Host == "alidocs.dingtalk.com" && u.User == nil && u.Fragment == ""
}
+30
View File
@@ -0,0 +1,30 @@
package httpapi
import (
"context"
"knowledge-governance/internal/dingtalkdiscovery"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestDWSRegistrationNeverFallsBackToBrowserSession(t *testing.T) {
calls := 0
sidecar := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { calls++; w.WriteHeader(401) }))
defer sidecar.Close()
t.Setenv("DINGTALK_LOGIN_SIDECAR_URL", sidecar.URL)
h, err := NewHandler(Config{Store: &fakeDingtalkRegistrar{fakeJobStore: &fakeJobStore{}}, DingtalkSources: fakeDingtalkSources{err: dingtalkdiscovery.ErrUnavailable}})
if err != nil {
t.Fatal(err)
}
req := httptest.NewRequest("POST", "/api/v1/dingtalk-spaces", strings.NewReader(`{"url":"https://alidocs.dingtalk.com/i/spaces/valid-space-id/overview"}`))
req.Header.Set("Idempotency-Key", "op-dws-only")
req.Header.Set("X-Governance-Action", "dingtalk-space")
req = req.WithContext(context.WithValue(req.Context(), sessionKey{}, sessionIdentity{Username: "tester", Role: roleAdmin}))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if calls != 0 || rec.Code != 503 || !strings.Contains(rec.Body.String(), "dws_unavailable") {
t.Fatalf("DWS failed: legacy calls=%d status=%d body=%s", calls, rec.Code, rec.Body.String())
}
}
+1 -1
View File
@@ -52,7 +52,7 @@ func TestOpenAPIContractMatchesRegisteredRoutesAndOmitsSecrets(t *testing.T) {
switch value := node.(type) {
case map[string]any:
for key, child := range value {
if key == "base_url" && path != "components.schemas.QAAISafeSettings.properties" && path != "components.schemas.QAAISaveSettings.properties" {
if key == "base_url" && path != "components.schemas.QAAISafeSettings.properties" && path != "components.schemas.QAAISaveSettings.properties" && path != "components.schemas.PPSafeMetadata.properties" && path != "components.schemas.PPConnectionDraft.properties" {
t.Fatalf("unapproved endpoint exposure at %s", path)
}
next := key
+94
View File
@@ -0,0 +1,94 @@
package httpapi
import (
"context"
"errors"
"knowledge-governance/internal/ppauth"
"net/http"
"net/url"
"time"
)
func credentialAdmin(w http.ResponseWriter, r *http.Request, action string) bool {
w.Header().Set("Cache-Control", "no-store")
identity, ok := requireSessionIdentity(w, r)
if !ok {
return false
}
if identity.Role != roleAdmin {
writeJSON(w, 403, map[string]string{"error": "需要管理员身份管理来源凭据"})
return false
}
if r.Method != "GET" {
originOK := true
if raw := r.Header.Get("Origin"); raw != "" {
u, err := url.Parse(raw)
originOK = err == nil && u.Host == r.Host && (u.Scheme == "http" || u.Scheme == "https")
}
if !originOK || r.Header.Get("Sec-Fetch-Site") == "cross-site" || r.Header.Get("X-Governance-Action") != action {
writeJSON(w, 403, map[string]string{"error": "请从本平台的数据源配置页操作"})
return false
}
}
return true
}
func writePPError(w http.ResponseWriter, err error) {
status := http.StatusBadRequest
if errors.Is(err, ppauth.ErrConflict) {
status = http.StatusConflict
}
writeJSON(w, status, map[string]string{"error": err.Error()})
}
func registerPPConnectionRoutes(mux *http.ServeMux, config Config) {
for _, pattern := range []string{"GET /api/v1/pp-connection", "PUT /api/v1/pp-connection", "POST /api/v1/pp-connection/test"} {
mux.HandleFunc(pattern, func(w http.ResponseWriter, r *http.Request) {
if !credentialAdmin(w, r, "pp-connection") {
return
}
if config.PPConnection == nil {
writeJSON(w, 503, map[string]string{"error": "PP 配置服务未接通,请联系管理员"})
return
}
if r.Method == "GET" {
v, err := config.PPConnection.View()
if err != nil {
writeJSON(w, 503, map[string]string{"error": err.Error()})
return
}
writeJSON(w, 200, map[string]any{"data": v})
return
}
var draft ppauth.Draft
r.Body = http.MaxBytesReader(w, r.Body, 16384)
if decodeJSONStrict(r, &draft) != nil {
writeJSON(w, 400, map[string]string{"error": "PP 配置请求格式不正确"})
return
}
ctx, cancel := context.WithTimeout(r.Context(), 50*time.Second)
defer cancel()
if r.Method == "POST" {
result, err := config.PPConnection.Test(ctx, draft, actorFor(r))
if err != nil {
writePPError(w, err)
return
}
writeJSON(w, 200, map[string]any{"data": result})
return
}
result, err := config.PPConnection.Save(ctx, draft, actorFor(r))
if err != nil {
status := 400
if errors.Is(err, ppauth.ErrConflict) {
status = 409
}
if errors.Is(err, ppauth.ErrRejected) {
status = 422
}
writeJSON(w, status, map[string]any{"error": err.Error(), "data": result})
return
}
writeJSON(w, 200, map[string]any{"data": result})
})
}
}
+142
View File
@@ -0,0 +1,142 @@
package httpapi
import (
"context"
"encoding/json"
"fmt"
"knowledge-governance/internal/jobs"
postgresstore "knowledge-governance/internal/platform/postgres"
"knowledge-governance/internal/ppauth"
"net/http/httptest"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"time"
)
type noPPJobStore struct {
*fakeJobStore
queued int
}
func (s *noPPJobStore) Enqueue(ctx context.Context, request jobs.EnqueueRequest, now time.Time) (*jobs.Job, error) {
s.queued++
return s.fakeJobStore.Enqueue(ctx, request, now)
}
func TestPPHTTPRoundTripRechecksSaveAndNeverEnqueues(t *testing.T) {
probes := 0
m := &ppauth.Manager{Path: t.TempDir() + "/credentials/settings.json", Probe: func(_ context.Context, c ppauth.Credentials) ppauth.Observation {
probes++
return ppauth.Observation{State: "valid", Message: "认证有效 · 目录可读"}
}}
store := &noPPJobStore{fakeJobStore: &fakeJobStore{}}
h, err := NewHandler(Config{Store: store, PPConnection: m})
if err != nil {
t.Fatal(err)
}
call := func(method, path, body string) *httptest.ResponseRecorder {
req := httptest.NewRequest(method, path, strings.NewReader(body))
req.Header.Set("X-Governance-Action", "pp-connection")
req = req.WithContext(context.WithValue(req.Context(), sessionKey{}, sessionIdentity{Username: "tester", Role: roleAdmin}))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
return rec
}
draft := `{"base_url":"https://api.example.invalid","username":"example","password":"example-private","revision":0}`
for _, op := range [][2]string{{"POST", "/api/v1/pp-connection/test"}, {"PUT", "/api/v1/pp-connection"}, {"GET", "/api/v1/pp-connection"}} {
rec := call(op[0], op[1], draft)
if rec.Code != 200 {
t.Fatalf("%s status=%d body=%s", op[0], rec.Code, rec.Body)
}
if strings.Contains(rec.Body.String(), "example-private") || strings.Contains(rec.Body.String(), `"password":`) {
t.Fatal("credential leaked through response")
}
}
if probes != 2 || store.queued != 0 {
t.Fatalf("save must recheck and must not enqueue: checks=%d jobs=%d", probes, store.queued)
}
rec := call("PUT", "/api/v1/pp-connection", draft)
if rec.Code != 409 {
t.Fatalf("stale revision accepted: %d", rec.Code)
}
}
func TestPPHTTPRefusesCrossOriginBeforeProbe(t *testing.T) {
probes := 0
m := &ppauth.Manager{Path: t.TempDir() + "/credentials/settings.json", Probe: func(_ context.Context, _ ppauth.Credentials) ppauth.Observation {
probes++
return ppauth.Observation{State: "valid"}
}}
h, _ := NewHandler(Config{Store: &fakeJobStore{}, PPConnection: m})
req := httptest.NewRequest("POST", "/api/v1/pp-connection/test", strings.NewReader(`{}`))
req.Header.Set("Origin", "https://foreign.example.invalid")
req.Header.Set("X-Governance-Action", "pp-connection")
req = req.WithContext(context.WithValue(req.Context(), sessionKey{}, sessionIdentity{Username: "tester", Role: roleAdmin}))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != 403 || probes != 0 {
t.Fatal("cross-origin request reached credential probe")
}
}
type combinedAuditStore struct {
*fakeJobStore
items []postgresstore.SourceAuditItem
}
func (s *combinedAuditStore) ListSourceAudit(_ context.Context, limit, offset int) (postgresstore.SourceAuditPage, error) {
start := offset
if start > len(s.items) {
start = len(s.items)
}
end := start + limit
if end > len(s.items) {
end = len(s.items)
}
return postgresstore.SourceAuditPage{Items: s.items[start:end], Total: len(s.items), Limit: limit, Offset: offset}, nil
}
func TestPPSourceAuditSharesTheExistingPagedView(t *testing.T) {
ctx := context.Background()
m := &ppauth.Manager{Path: t.TempDir() + "/credentials/settings.json"}
if err := os.Mkdir(filepath.Dir(m.Path), 0700); err != nil {
t.Fatal(err)
}
now := time.Now().UTC()
store := &combinedAuditStore{fakeJobStore: &fakeJobStore{}}
expected := []postgresstore.SourceAuditItem{}
for i := 0; i < 8; i++ {
item := postgresstore.SourceAuditItem{ID: fmt.Sprintf("db-%02d", i), CreatedAt: now.Add(-time.Duration(i*2) * time.Minute), SourceName: "existing source"}
store.items = append(store.items, item)
expected = append(expected, item)
}
state := ppauth.State{}
for i := 0; i < 4; i++ {
event := ppauth.AuditEvent{ID: fmt.Sprintf("auth-%02d", i), CreatedAt: now.Add(-time.Duration(i*2+1) * time.Minute), SourceName: "PP", Action: "credential_test", Actor: "tester", Detail: "只读连接检查"}
state.Audit = append(state.Audit, event)
expected = append(expected, postgresstore.SourceAuditItem{ID: event.ID, CreatedAt: event.CreatedAt, SourceName: event.SourceName})
}
raw, _ := json.Marshal(state)
if err := os.WriteFile(m.Path, raw, 0600); err != nil {
t.Fatal(err)
}
sort.Slice(expected, func(i, j int) bool { return expected[i].CreatedAt.After(expected[j].CreatedAt) })
for offset := 0; offset <= len(expected); offset++ {
page, err := combinedSourceAudit(ctx, store, m, 3, offset)
if err != nil || page.Total != len(expected) {
t.Fatalf("offset=%d total=%d err=%v", offset, page.Total, err)
}
end := offset + 3
if end > len(expected) {
end = len(expected)
}
if len(page.Items) != end-offset {
t.Fatalf("wrong page length at %d", offset)
}
for i, item := range page.Items {
if item.ID != expected[offset+i].ID {
t.Fatalf("audit order mismatch at offset %d", offset)
}
}
}
}
+42
View File
@@ -0,0 +1,42 @@
package httpapi
import (
"context"
"net/http/httptest"
"testing"
)
func TestPPAuthRoutesRequireAdminAndFailClosedWhenUnwired(t *testing.T) {
h, err := NewHandler(Config{Store: &fakeJobStore{}})
if err != nil {
t.Fatal(err)
}
for _, method := range []string{"GET", "PUT", "POST"} {
path := "/api/v1/pp-connection"
if method == "POST" {
path += "/test"
}
for _, role := range []string{"", roleViewer, roleEngineer, roleAdmin} {
req := httptest.NewRequest(method, path, nil)
if role != "" {
req = req.WithContext(context.WithValue(req.Context(), sessionKey{}, sessionIdentity{UserID: "admin", Username: "tester", Role: role, SessionID: "session"}))
}
req.Header.Set("X-Governance-Action", "pp-connection")
res := httptest.NewRecorder()
h.ServeHTTP(res, req)
want := 403
if role == "" {
want = 401
}
if role == roleAdmin {
want = 503
}
if res.Code != want {
t.Errorf("%s role=%q status=%d want=%d", method, role, res.Code, want)
}
}
if got := classifyRole(method, path); got != roleAdmin {
t.Errorf("%s classified=%q; credential metadata and tests must be admin only", method, got)
}
}
}
+4
View File
@@ -20,6 +20,7 @@ import (
"knowledge-governance/internal/jobs"
"knowledge-governance/internal/monitoring"
postgresstore "knowledge-governance/internal/platform/postgres"
"knowledge-governance/internal/ppauth"
"knowledge-governance/internal/qaopsai"
"knowledge-governance/internal/qaopsmirror"
"knowledge-governance/internal/qaopsorg"
@@ -224,6 +225,7 @@ type Config struct {
// DingtalkDocuments 列在线文档/表格并校验可读(只读,dws)。
DingtalkDocuments dingtalkDocLister
DWSLogin *dingtalkdiscovery.DeviceLoginManager
PPConnection *ppauth.Manager
NewID func() (string, error)
Auth *AuthConfig
}
@@ -259,6 +261,8 @@ func NewHandler(config Config) (http.Handler, error) {
}
mux := http.NewServeMux()
registerDWSLoginRoutes(mux, config)
registerPPConnectionRoutes(mux, config)
mux.HandleFunc("POST /api/v1/dingtalk-spaces/validate", func(w http.ResponseWriter, r *http.Request) { validateDWSSpace(w, r, config) })
registerQAAIRoutes(mux, config)
mux.HandleFunc("POST /api/v1/qa-ops/departments/statistics", func(w http.ResponseWriter, r *http.Request) { qaDepartmentStatistics(w, r, config) })
mux.HandleFunc("POST /api/v1/qa-ops/dashboard", func(w http.ResponseWriter, r *http.Request) { qaOperationsDashboard(w, r, config) })
+1 -1
View File
@@ -35,7 +35,7 @@ func listSourceAudit(w http.ResponseWriter, r *http.Request, config Config) {
writeJSON(w, 501, map[string]string{"error": "audit not configured"})
return
}
page, err := reader.ListSourceAudit(r.Context(), limit, offset)
page, err := combinedSourceAudit(r.Context(), reader, config.PPConnection, limit, offset)
if err != nil {
writeJSON(w, 500, map[string]string{"error": "无法读取数据源操作记录"})
return
@@ -0,0 +1,36 @@
package postgres
import (
"context"
"net/url"
"strings"
)
// RegisteredDingtalkSpaceIDs also covers historical connections whose root key
// predates DWS; a source name is never an identity.
func (s *Store) RegisteredDingtalkSpaceIDs(ctx context.Context) (map[string]bool, error) {
rows, err := s.db.Query(ctx, "SELECT base_url FROM source_connections WHERE source_type='dingtalk' AND archived_at IS NULL")
if err != nil {
return nil, err
}
defer rows.Close()
result := map[string]bool{}
for rows.Next() {
var raw string
if err := rows.Scan(&raw); err != nil {
return nil, err
}
u, err := url.Parse(raw)
if err != nil {
continue
}
space := u.Query().Get("space")
if space == "" && strings.HasPrefix(u.Path, "spaces/") {
space = strings.TrimPrefix(u.Path, "spaces/")
}
if space != "" {
result[space] = true
}
}
return result, rows.Err()
}
+356
View File
@@ -0,0 +1,356 @@
package ppauth
import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"net/url"
"os"
"path/filepath"
"strings"
"syscall"
"time"
)
type Credentials struct {
ID string `json:"id"`
BaseURL string `json:"base_url"`
Username string `json:"username"`
Password string `json:"password"`
}
type Draft struct {
BaseURL string `json:"base_url"`
Username string `json:"username"`
Password string `json:"password,omitempty"`
PendingID string `json:"pending_id,omitempty"`
Revision int `json:"revision"`
}
type Observation struct {
State string `json:"state"`
Message string `json:"message"`
CheckedAt string `json:"checked_at"`
Previous *Observation `json:"previous,omitempty"`
}
type Metadata struct {
ID string `json:"id"`
BaseURL string `json:"base_url"`
Username string `json:"username"`
PasswordConfigured bool `json:"password_configured"`
}
type View struct {
Revision int `json:"revision"`
Active *Metadata `json:"active"`
Pending *Metadata `json:"pending"`
Observation *Observation `json:"observation"`
PendingObservation *Observation `json:"pending_observation"`
}
type AuditEvent struct {
ID string `json:"id"`
SourceName string `json:"source_name"`
Action string `json:"action"`
Actor string `json:"actor"`
CreatedAt time.Time `json:"created_at"`
Detail string `json:"detail"`
}
type State struct {
Revision int `json:"revision"`
Active *Credentials `json:"active"`
Pending *Credentials `json:"pending"`
Observation *Observation `json:"observation"`
PendingObservation *Observation `json:"pending_observation"`
Audit []AuditEvent `json:"audit"`
}
type Manager struct {
Path string
LegacyPath string
Probe func(context.Context, Credentials) Observation
}
var ErrRejected = errors.New("PP 配置校验未通过,原配置保留")
var ErrInvalid = errors.New("API 地址必须为 HTTPS 根地址,且不能包含凭据、查询参数或片段")
var ErrConflict = errors.New("配置已被更新,请刷新后重新提交")
func ValidateBaseURL(raw string) bool {
u, err := url.Parse(raw)
return err == nil && len(raw) <= 300 && strings.TrimSpace(raw) == raw && u.Scheme == "https" && u.Host != "" && u.Hostname() != "" && u.Opaque == "" && u.User == nil && u.RawQuery == "" && !u.ForceQuery && u.Fragment == "" && !strings.Contains(raw, "#") && (u.Path == "" || u.Path == "/")
}
func metadata(c *Credentials) *Metadata {
if c == nil {
return nil
}
return &Metadata{ID: c.ID, BaseURL: c.BaseURL, Username: c.Username, PasswordConfigured: c.Password != ""}
}
func view(s State) View {
return View{Revision: s.Revision, Active: metadata(s.Active), Pending: metadata(s.Pending), Observation: s.Observation, PendingObservation: s.PendingObservation}
}
func randomID() string {
var b [16]byte
if _, err := rand.Read(b[:]); err != nil {
panic("secure random unavailable")
}
return hex.EncodeToString(b[:])
}
func (m *Manager) locked(ctx context.Context, fn func(*State) (bool, error)) error {
if !filepath.IsAbs(m.Path) {
return errors.New("PP credential storage path is invalid")
}
dir := filepath.Dir(m.Path)
if err := os.MkdirAll(dir, 0700); err != nil {
return errors.New("PP 配置存储不可用")
}
info, err := os.Lstat(dir)
if err != nil || !info.IsDir() || info.Mode().Perm()&0077 != 0 {
return errors.New("PP 配置目录权限必须为 0700")
}
lockPath := m.Path + ".lock"
if info, e := os.Lstat(lockPath); e == nil && (info.Mode()&os.ModeSymlink != 0 || info.Mode().Perm()&0077 != 0) {
return errors.New("PP 配置锁权限不安全")
}
lock, err := os.OpenFile(lockPath, os.O_CREATE|os.O_RDWR, 0600)
if err != nil {
return errors.New("PP 配置存储不可用")
}
defer lock.Close()
for {
err = syscall.Flock(int(lock.Fd()), syscall.LOCK_EX|syscall.LOCK_NB)
if err == nil {
break
}
if err != syscall.EWOULDBLOCK {
return errors.New("PP 配置锁不可用")
}
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(20 * time.Millisecond):
}
}
defer syscall.Flock(int(lock.Fd()), syscall.LOCK_UN)
var s State
info, err = os.Lstat(m.Path)
if err == nil {
if info.Mode()&os.ModeSymlink != 0 || !info.Mode().IsRegular() || info.Mode().Perm()&0077 != 0 || info.Size() > 16<<20 {
return errors.New("PP 凭据文件权限或格式不安全")
}
raw, e := os.ReadFile(m.Path)
if e != nil || json.Unmarshal(raw, &s) != nil {
return errors.New("无法读取 PP 配置")
}
} else if !errors.Is(err, os.ErrNotExist) {
return errors.New("无法读取 PP 配置")
} else if m.LegacyPath != "" {
raw, e := os.ReadFile(m.LegacyPath)
if e == nil {
var c Credentials
if json.Unmarshal(raw, &c) != nil {
return errors.New("旧 PP 配置格式不正确")
}
if c.BaseURL == "" {
c.BaseURL = "https://api.dobot.cn"
}
c.ID = "legacy"
s.Active = &c
} else if !errors.Is(e, os.ErrNotExist) {
return errors.New("无法读取旧 PP 配置")
}
}
changed, err := fn(&s)
if err != nil || !changed {
return err
}
raw, err := json.Marshal(s)
if err != nil {
return errors.New("PP 配置保存失败")
}
f, err := os.CreateTemp(dir, ".pp-next-")
if err != nil {
return errors.New("PP 配置保存失败")
}
tmp := f.Name()
defer os.Remove(tmp)
if err = f.Chmod(0600); err == nil {
_, err = f.Write(raw)
}
if err == nil {
err = f.Sync()
}
closeErr := f.Close()
if err == nil {
err = closeErr
}
if err == nil {
err = os.Rename(tmp, m.Path)
}
if err != nil {
return errors.New("PP 配置保存失败")
}
d, e := os.Open(dir)
if e != nil {
return errors.New("PP 配置持久化未确认")
}
defer d.Close()
if d.Sync() != nil {
return errors.New("PP 配置持久化未确认")
}
return nil
}
func (m *Manager) View() (v View, err error) {
err = m.locked(context.Background(), func(s *State) (bool, error) { v = view(*s); return false, nil })
return
}
func (m *Manager) snapshot(ctx context.Context) (s State, err error) {
err = m.locked(ctx, func(state *State) (bool, error) { s = *state; return false, nil })
return
}
func candidate(s State, d Draft) (Credentials, error) {
if !ValidateBaseURL(d.BaseURL) {
return Credentials{}, ErrInvalid
}
if len(d.Username) > 255 || strings.ContainsAny(d.Username, "\n\r\x00") || len(d.Password) > 8192 {
return Credentials{}, errors.New("账号或密码格式不正确")
}
c := Credentials{ID: randomID(), BaseURL: strings.TrimRight(d.BaseURL, "/"), Username: strings.TrimSpace(d.Username), Password: d.Password}
if c.Password == "" && d.PendingID != "" {
if s.Pending == nil || s.Pending.ID != d.PendingID {
return Credentials{}, ErrConflict
}
c.Password = s.Pending.Password
} else if c.Password == "" && s.Active != nil {
c.Password = s.Active.Password
}
return c, nil
}
func same(a, b *Credentials) bool {
return a != nil && b != nil && a.BaseURL == b.BaseURL && a.Username == b.Username && a.Password == b.Password
}
func (m *Manager) probe(ctx context.Context, c Credentials) Observation {
o := Observation{State: "incomplete", Message: "配置未完成,请填写账号和密码"}
if c.Username != "" && c.Password != "" {
if m.Probe == nil {
o = Observation{State: "unavailable", Message: "PP 检查服务未接通"}
} else {
o = m.Probe(ctx, c)
}
}
switch o.State {
case "valid", "auth_failed", "permission_denied", "incomplete", "unavailable", "invalid_response":
default:
o = Observation{State: "unavailable", Message: "检查结果不可识别,请重试"}
}
if o.CheckedAt == "" {
o.CheckedAt = time.Now().UTC().Format(time.RFC3339Nano)
}
return o
}
func addAudit(s *State, source, action, actor, detail string) {
s.Audit = append(s.Audit, AuditEvent{ID: "auth-" + randomID(), SourceName: source, Action: action, Actor: actor, CreatedAt: time.Now().UTC(), Detail: detail})
}
func retainObservation(old *Observation, next Observation) *Observation {
if next.State == "unavailable" && old != nil {
if old.State == "unavailable" && old.Previous != nil {
next.Previous = old.Previous
} else {
next.Previous = old
}
}
return &next
}
func (m *Manager) Save(ctx context.Context, d Draft, actor string) (v View, err error) {
s, err := m.snapshot(ctx)
if err != nil {
return v, err
}
if s.Revision != d.Revision {
return v, ErrConflict
}
c, err := candidate(s, d)
if err != nil {
return v, err
}
o := m.probe(ctx, c)
rejected := o.State == "auth_failed" || o.State == "permission_denied" || o.State == "invalid_response"
err = m.locked(ctx, func(current *State) (bool, error) {
if current.Revision != s.Revision {
return false, ErrConflict
}
if rejected {
if same(current.Active, &c) {
current.Observation = retainObservation(current.Observation, o)
}
addAudit(current, "PP 资料下载中心", "credential_rejected", actor, o.Message)
v = view(*current)
return true, nil
}
current.Revision++
if o.State == "valid" {
current.Active = &c
current.Pending = nil
current.Observation = retainObservation(current.Observation, o)
current.PendingObservation = nil
} else {
current.Pending = &c
current.PendingObservation = &o
}
detail := "认证配置已保存并生效"
if o.State != "valid" {
detail = "修改已保存为待验证,未替换生效配置"
}
if d.Password != "" {
detail += ";密码已更新"
}
addAudit(current, "PP 资料下载中心", "credential_configured", actor, detail)
v = view(*current)
return true, nil
})
if err == nil && rejected {
err = fmt.Errorf("%w:%s", ErrRejected, o.Message)
}
return
}
func (m *Manager) Test(ctx context.Context, d Draft, actor string) (o Observation, err error) {
s, err := m.snapshot(ctx)
if err != nil {
return o, err
}
if s.Revision != d.Revision {
return o, ErrConflict
}
c, err := candidate(s, d)
if err != nil {
return o, err
}
o = m.probe(ctx, c)
err = m.locked(ctx, func(current *State) (bool, error) {
if current.Revision != s.Revision {
return false, ErrConflict
}
if same(current.Active, &c) {
current.Observation = retainObservation(current.Observation, o)
}
addAudit(current, "PP 资料下载中心", "credential_test", actor, o.Message)
return true, nil
})
return
}
func (m *Manager) Audit(ctx context.Context, source, action, actor, detail string) error {
return m.locked(ctx, func(s *State) (bool, error) { addAudit(s, source, action, actor, detail); return true, nil })
}
func (m *Manager) AuditOnce(ctx context.Context, id, source, action, actor, detail string) error {
return m.locked(ctx, func(s *State) (bool, error) {
for _, e := range s.Audit {
if e.ID == id {
return false, nil
}
}
addAudit(s, source, action, actor, detail)
s.Audit[len(s.Audit)-1].ID = id
return true, nil
})
}
func (m *Manager) Events(ctx context.Context) (events []AuditEvent, err error) {
err = m.locked(ctx, func(s *State) (bool, error) { events = append(events, s.Audit...); return false, nil })
return
}
+35
View File
@@ -0,0 +1,35 @@
package ppauth
import (
"context"
"errors"
"testing"
)
func TestSaveValidatesBeforeActivationAndTransientKeepsActive(t *testing.T) {
state := "valid"
seen := ""
manager := &Manager{Path: t.TempDir() + "/credentials/settings.json", Probe: func(_ context.Context, c Credentials) Observation {
seen = c.Password
return Observation{State: state, Message: state}
}}
view, err := manager.Save(context.Background(), Draft{BaseURL: "https://api.example.invalid", Username: "example", Password: "example-password", Revision: 0}, "tester")
if err != nil || view.Active == nil || view.Active.Username != "example" || !view.Active.PasswordConfigured || seen != "example-password" {
t.Fatalf("validated save did not activate: %+v %v", view, err)
}
activeID := view.Active.ID
state = "unavailable"
view, err = manager.Save(context.Background(), Draft{BaseURL: "https://changed.example.invalid", Username: "changed", Revision: view.Revision}, "tester")
if err != nil || view.Active == nil || view.Active.ID != activeID || view.Pending == nil || view.Pending.Username != "changed" || seen != "example-password" {
t.Fatalf("transient save replaced active or lost password: %+v %v", view, err)
}
state = "auth_failed"
_, err = manager.Save(context.Background(), Draft{BaseURL: "https://changed.example.invalid", Username: "changed", Password: "bad-example", Revision: view.Revision}, "tester")
if !errors.Is(err, ErrRejected) {
t.Fatalf("explicit rejection allowed activation: %v", err)
}
after, _ := manager.View()
if after.Active.ID != activeID {
t.Fatal("rejection changed active config")
}
}
+104
View File
@@ -0,0 +1,104 @@
package ppauth
import (
"context"
"encoding/json"
"errors"
"os"
"os/exec"
"strings"
"sync"
"sync/atomic"
"testing"
)
func TestPendingPasswordSurvivesRevalidationWithoutReturningSecrets(t *testing.T) {
state := "valid"
seen := ""
m := &Manager{Path: t.TempDir() + "/credentials/settings.json", Probe: func(_ context.Context, c Credentials) Observation {
seen = c.Password
return Observation{State: state, Message: state}
}}
v, err := m.Save(context.Background(), Draft{BaseURL: "https://api.example.invalid", Username: "example", Password: "example-old"}, "tester")
if err != nil {
t.Fatal(err)
}
state = "unavailable"
v, err = m.Save(context.Background(), Draft{BaseURL: "https://api.example.invalid", Username: "example", Password: "example-new", Revision: v.Revision}, "tester")
if err != nil || v.Pending == nil {
t.Fatalf("pending save: %v", err)
}
state = "valid"
v, err = m.Save(context.Background(), Draft{BaseURL: v.Pending.BaseURL, Username: v.Pending.Username, PendingID: v.Pending.ID, Revision: v.Revision}, "tester")
if err != nil || seen != "example-new" || v.Pending != nil {
t.Fatal("saved pending password was not reused")
}
raw, _ := json.Marshal(v)
if strings.Contains(string(raw), "example-new") || strings.Contains(string(raw), `"password":`) {
t.Fatal("metadata leaked secret")
}
info, err := os.Stat(m.Path)
if err != nil || info.Mode().Perm() != 0600 {
t.Fatal("secret file not mode 0600")
}
}
func TestConcurrentSavesCannotOverwriteANewerRevision(t *testing.T) {
var calls atomic.Int32
barrier := make(chan struct{})
m := &Manager{Path: t.TempDir() + "/credentials/settings.json", Probe: func(_ context.Context, c Credentials) Observation {
if calls.Add(1) == 2 {
close(barrier)
}
<-barrier
return Observation{State: "valid", Message: "valid"}
}}
outcomes := make(chan error, 2)
var wg sync.WaitGroup
for _, name := range []string{"example-one", "example-two"} {
wg.Add(1)
go func(user string) {
defer wg.Done()
_, err := m.Save(context.Background(), Draft{BaseURL: "https://api.example.invalid", Username: user, Password: "example-password"}, "tester")
outcomes <- err
}(name)
}
wg.Wait()
close(outcomes)
success, conflict := 0, 0
for err := range outcomes {
if err == nil {
success++
} else if errors.Is(err, ErrConflict) {
conflict++
} else {
t.Fatal(err)
}
}
if success != 1 || conflict != 1 {
t.Fatalf("success=%d conflict=%d", success, conflict)
}
}
func TestFailedObservationRetainsPreviousSuccess(t *testing.T) {
state := "valid"
m := &Manager{Path: t.TempDir() + "/credentials/settings.json", Probe: func(_ context.Context, c Credentials) Observation { return Observation{State: state, Message: state} }}
v, err := m.Save(context.Background(), Draft{BaseURL: "https://api.example.invalid", Username: "example", Password: "example-password"}, "tester")
if err != nil {
t.Fatal(err)
}
state = "unavailable"
_, err = m.Test(context.Background(), Draft{BaseURL: v.Active.BaseURL, Username: v.Active.Username, Revision: v.Revision}, "tester")
if err != nil {
t.Fatal(err)
}
v, err = m.View()
if err != nil || v.Observation.Previous == nil || v.Observation.Previous.State != "valid" {
t.Fatal("failed check erased previous successful observation")
}
}
func TestPPPythonProtocolAndConfigurationConsumers(t *testing.T) {
command := exec.Command("python3", "-m", "unittest", "discover", "-s", "../../scripts", "-p", "test_pp_client.py", "-v")
output, err := command.CombinedOutput()
if err != nil {
t.Fatalf("PP protocol regression: %v\n%s", err, output)
}
}
+31
View File
@@ -0,0 +1,31 @@
package ppauth
import (
"bytes"
"context"
"encoding/json"
"os"
"os/exec"
"time"
)
// CommandProbe reuses the worker's read-only PP adapter; secrets travel on stdin.
func CommandProbe(python, script string) func(context.Context, Credentials) Observation {
return func(ctx context.Context, c Credentials) Observation {
ctx, cancel := context.WithTimeout(ctx, 45*time.Second)
defer cancel()
raw, _ := json.Marshal(c)
command := exec.CommandContext(ctx, python, script)
command.Stdin = bytes.NewReader(raw)
command.Env = []string{"PATH=" + os.Getenv("PATH"), "PYTHONUNBUFFERED=1"}
out, err := command.Output()
if err != nil || len(out) > 8192 {
return Observation{State: "unavailable", Message: "PP 检查服务暂不可用,请重试"}
}
var o Observation
if json.Unmarshal(out, &o) != nil {
return Observation{State: "unavailable", Message: "PP 检查结果不可识别"}
}
return o
}
}
+1 -1
View File
@@ -181,7 +181,7 @@ func NewCommandRunner(pythonPath, scriptPath, spoolRoot, spoolPrefix string, max
// ——钉钉取件在真实链路上会整体失败(2026-10-10 端到端验收实测;脚本单跑正常,
// 因为那只绕过了这个白名单)。
var dwsEnvNames = []string{
"ARTIFACT_DINGTALK_FETCHER", "DWS_BIN", "DWS_CONFIG_DIR", "DWS_HOME", "DWS_DISABLE_KEYCHAIN",
"ARTIFACT_DINGTALK_FETCHER", "PP_AUTH_STATE_FILE", "PP_LEGACY_CREDENTIALS_FILE", "DWS_BIN", "DWS_CONFIG_DIR", "DWS_HOME", "DWS_DISABLE_KEYCHAIN",
}
func forwardedDWSEnv() []string {
+18 -4
View File
@@ -278,7 +278,7 @@ def pp_find_entry(manifest):
def pp_download(manifest, max_bytes):
entry = pp_find_entry(manifest)
if entry is None:
raise MissingSource("PP ledger entry for this locator no longer exists")
entry = {}
# Ledger titles may omit the extension; recover the true file name from the
# local path or source URL before type checks run.
manifest_name = manifest.setdefault("name", "")
@@ -301,9 +301,23 @@ def pp_download(manifest, max_bytes):
if expected_hash and len(expected_hash) == 64 and actual != expected_hash:
raise FetchError("PP local copy hash mismatch (ledger changed?)", retryable=True, code="local_mismatch")
return str(candidate), size, actual, True
url = entry.get("url", "")
if not url:
raise MissingSource("PP ledger entry has no local file and no download URL")
from pp_client import active_client, record_observation, observation, PPError
client = None
try:
client = active_client()
inventory = client.inventory(client.login())
record_observation(client.config, observation('valid'))
except PPError as error:
if client is not None:
record_observation(client.config, observation(error.state))
raise FetchError(str(error), retryable=error.state == 'unavailable', code='pp_' + error.state) from None
locator = manifest.get('locator_hash', '')
matches = [row for row in inventory['items'] if row.get('online') == 1 and (row.get('potalGroup') or {}).get('groupName') == '公开' and digest_of(str(row.get('fileUrl') or '').strip()) == locator]
if not matches:
raise MissingSource('PP file is no longer visible in the configured portal')
url = str(matches[0].get('fileUrl') or '').strip()
if not pathlib.PurePosixPath(manifest['name']).suffix:
manifest['name'] += pathlib.PurePosixPath(urllib.parse.unquote(urllib.parse.urlsplit(url).path)).suffix
path, size, actual = http_get(url, {}, 180, max_bytes)
expected_hash = str(entry.get("sha256", "")).lower()
if expected_hash and len(expected_hash) == 64 and actual != expected_hash:
+10 -5
View File
@@ -8,6 +8,7 @@ import argparse,csv,io,json,os,pathlib,re,subprocess,sys,tempfile
from collections import deque
sys.path.insert(0,'/app/scripts')
from source_catalog_import import build_alist_source,build_dingtalk_sources,build_pp_live_source,catalog_sql,validate_source
from pp_client import active_client,record_observation,observation,PPError
class RefreshBlocked(RuntimeError):pass
_source_name='';_last_directory=''
@@ -39,12 +40,16 @@ def run_psql(sql):
if result.returncode:raise RuntimeError('catalog activation failed')
def refresh_pp():
sys.path.insert(0,'/sources/pp');from download_scope_a import login,fetch_inventory
emit('connecting',10,'正在登录PP资料下载中心')
try:token=login('/run/source-secrets/pp_credentials.json')
except Exception as error:raise RefreshBlocked('PP登录失败,请在来源链接更新凭据') from error
emit('scanning',35,'正在实时读取PP公开资料清单')
inventory=fetch_inventory(token)
client=None
try:
client=active_client();token=client.login()
emit('scanning',35,'正在实时读取PP公开资料清单')
inventory=client.inventory(token)
record_observation(client.config,observation('valid'))
except PPError as error:
if client is not None:record_observation(client.config,observation(error.state))
raise RefreshBlocked(str(error)+',请在数据源配置检查PP连接') from error
emit('building',75,'正在构建PP八类目录快照',len(inventory.get('items',[])),len(inventory.get('items',[])))
manifest={}
try:
+205
View File
@@ -0,0 +1,205 @@
"""PP read-only API client; the only PP authentication protocol implementation.
API checks and new worker tasks load the same atomically activated configuration.
Secrets enter the check subprocess on stdin, never through argv or stdout.
"""
import contextlib
import datetime
import fcntl
import json
import os
import pathlib
import stat
import sys
import tempfile
import urllib.error
import urllib.parse
import urllib.request
DEFAULT_BASE = "https://api.dobot.cn"
DEFAULT_STATE = "/data/source-auth/pp/settings.json"
MESSAGES = {
"valid": "认证有效 · 目录可读",
"incomplete": "配置未完成,请填写账号和密码",
"auth_failed": "PP 拒绝认证,请检查账号和密码",
"permission_denied": "已认证,但目录权限不足",
"unavailable": "PP 暂时无法检查,请稍后重试",
"invalid_response": "PP 接口地址或返回格式不兼容",
}
class PPError(Exception):
def __init__(self, state):
self.state = state
super().__init__(MESSAGES[state])
def observation(state):
return {"state": state, "message": MESSAGES[state], "checked_at": datetime.datetime.now(datetime.timezone.utc).isoformat()}
def validate_base(raw, allow_http=False):
if not isinstance(raw, str) or len(raw) > 300 or raw.strip() != raw:
raise PPError("invalid_response")
try:
u = urllib.parse.urlsplit(raw)
if (u.scheme not in (("https", "http") if allow_http else ("https",))
or not u.hostname or u.username is not None or u.password is not None
or u.path not in ("", "/") or u.query or u.fragment or "?" in raw or "#" in raw):
raise PPError("invalid_response")
_ = u.port
except ValueError:
raise PPError("invalid_response") from None
return raw.rstrip("/")
def load_config(path=None):
p = pathlib.Path(path or os.environ.get("PP_AUTH_STATE_FILE") or DEFAULT_STATE)
if p.exists():
info = p.lstat()
if not stat.S_ISREG(info.st_mode) or stat.S_IMODE(info.st_mode) & 0o077:
raise PPError("incomplete")
state = json.loads(p.read_text(encoding="utf-8"))
config = state.get("active")
if not isinstance(config, dict):
raise PPError("incomplete")
else:
legacy = pathlib.Path(os.environ.get("PP_LEGACY_CREDENTIALS_FILE") or "/run/source-secrets/pp_credentials.json")
try:
config = json.loads(legacy.read_text(encoding="utf-8"))
except (OSError, ValueError):
raise PPError("incomplete") from None
config = {**config, "base_url": config.get("base_url") or DEFAULT_BASE, "id": "legacy"}
validate_base(config.get("base_url"))
if not config.get("username") or not config.get("password"):
raise PPError("incomplete")
return config
class NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
return None
class Client:
def __init__(self, config, allow_http=False, timeout=20):
self.config = dict(config)
self.base = validate_base(config.get("base_url"), allow_http)
self.timeout = timeout
self.opener = urllib.request.build_opener(NoRedirect())
def request(self, path, token="", body=None, login=False):
headers = {"Accept": "application/json", "User-Agent": "KnowledgeGovernance/PPReadOnly"}
if token:
headers["Authorization"] = "Bearer " + token
if body is not None:
headers["Content-Type"] = "application/x-www-form-urlencoded"
req = urllib.request.Request(self.base + path, data=body, headers=headers)
try:
with self.opener.open(req, timeout=self.timeout) as response:
raw = response.read((8 << 20) + 1)
if len(raw) > 8 << 20:
raise PPError("invalid_response")
payload = json.loads(raw)
except urllib.error.HTTPError as error:
error.close()
if error.code == 401 or (error.code == 403 and login):
raise PPError("auth_failed") from None
if error.code == 403:
raise PPError("permission_denied") from None
if error.code in (408, 425, 429) or error.code >= 500:
raise PPError("unavailable") from None
raise PPError("invalid_response") from None
except (TimeoutError, OSError):
raise PPError("unavailable") from None
except (ValueError, TypeError):
raise PPError("invalid_response") from None
if not isinstance(payload, dict) or not isinstance(payload.get("code"), int) or isinstance(payload.get("code"), bool):
raise PPError("invalid_response")
if payload["code"] != 0:
code = payload["code"]
if code in (408, 429) or 500 <= code < 600:
raise PPError("unavailable")
if login or code == 401:
raise PPError("auth_failed")
if code == 403:
raise PPError("permission_denied")
raise PPError("invalid_response")
return payload.get("data")
def login(self):
if not self.config.get("username") or not self.config.get("password"):
raise PPError("incomplete")
body = urllib.parse.urlencode({"username": self.config["username"], "password": self.config["password"]}).encode()
data = self.request("/admin/login/token", body=body, login=True)
token = data.get("access_token") if isinstance(data, dict) else None
if not isinstance(token, str) or not token:
raise PPError("invalid_response")
return token
def page(self, token, current=1, size=200):
query = urllib.parse.urlencode({"current": current, "size": size})
data = self.request("/potal/file/back/page?" + query, token=token)
if (not isinstance(data, dict) or not isinstance(data.get("list"), list)
or not isinstance(data.get("total"), int) or isinstance(data.get("total"), bool)
or data["total"] < 0 or not isinstance(data.get("hasNextPage"), bool)):
raise PPError("invalid_response")
return data
def check(self):
try:
self.page(self.login(), size=1)
return observation("valid")
except PPError as error:
return observation(error.state)
def inventory(self, token):
items = []
for current in range(1, 10001):
page = self.page(token, current)
items.extend(page["list"])
if not page["hasNextPage"]:
if len(items) != page["total"]:
raise PPError("invalid_response")
return {"reported_total": page["total"], "pages": current, "items": items}
raise PPError("invalid_response")
def record_observation(config, result, path=None):
"""An old task may update only the observation of its own active revision."""
p = pathlib.Path(path or os.environ.get("PP_AUTH_STATE_FILE") or DEFAULT_STATE)
if not p.is_file():
return
with os.fdopen(os.open(str(p) + ".lock", os.O_CREAT | os.O_RDWR, 0o600), "r+", encoding="utf-8") as lock:
fcntl.flock(lock, fcntl.LOCK_EX)
state = json.loads(p.read_text(encoding="utf-8"))
if not state.get("active") or state["active"].get("id") != config.get("id"):
return
old = state.get("observation")
if result.get("state") == "unavailable" and isinstance(old, dict):
result = {**result, "previous": old.get("previous") or old}
state["observation"] = result
fd, temp = tempfile.mkstemp(prefix=".pp-next-", dir=p.parent)
try:
with os.fdopen(fd, "w", encoding="utf-8") as out:
json.dump(state, out, ensure_ascii=False)
out.flush()
os.fsync(out.fileno())
os.replace(temp, p)
directory = os.open(p.parent, os.O_RDONLY)
try:
os.fsync(directory)
finally:
os.close(directory)
finally:
with contextlib.suppress(FileNotFoundError):
os.unlink(temp)
def active_client():
return Client(load_config())
def main():
try:
config = json.load(sys.stdin)
result = Client(config).check()
except PPError as error:
result = observation(error.state)
except Exception:
result = observation("unavailable")
print(json.dumps(result, ensure_ascii=False))
if __name__ == "__main__":
main()
+56
View File
@@ -0,0 +1,56 @@
import json
import contextlib
import hashlib
import io
import os
from unittest.mock import patch
import pathlib
import tempfile
import threading
import unittest
from http.server import BaseHTTPRequestHandler, HTTPServer
import pp_client
class PPClientTests(unittest.TestCase):
def setUp(self):
self.calls=[];self.login_code=0;self.list_status=200;self.items=[]
owner=self
class Handler(BaseHTTPRequestHandler):
def log_message(self,*args):pass
def do_POST(self):
owner.calls.append((self.command,self.path));self.rfile.read(int(self.headers.get('Content-Length','0')))
self.send_response(200);self.end_headers();self.wfile.write(json.dumps({'code':owner.login_code,'data':{'access_token':'example-token'} if owner.login_code==0 else {}}).encode())
def do_GET(self):
owner.calls.append((self.command,self.path))
if self.path=='/file.pdf':
content=b'%PDF-1.4\nExample PP download\n';self.send_response(200);self.send_header('Content-Length',str(len(content)));self.end_headers();self.wfile.write(content);return
self.send_response(owner.list_status);self.end_headers();self.wfile.write(json.dumps({'code':0,'data':{'list':owner.items,'total':len(owner.items),'hasNextPage':False}}).encode())
self.server=HTTPServer(('127.0.0.1',0),Handler);self.thread=threading.Thread(target=self.server.serve_forever,daemon=True);self.thread.start()
self.client=pp_client.Client({'base_url':'http://127.0.0.1:'+str(self.server.server_port),'username':'example','password':'example-password'},allow_http=True)
def tearDown(self):self.server.shutdown();self.server.server_close();self.thread.join()
def test_check_really_logs_in_and_reads_empty_directory(self):
result=self.client.check();self.assertEqual(result['state'],'valid');self.assertEqual([x[0] for x in self.calls],['POST','GET']);self.assertTrue(self.calls[0][1].endswith('/admin/login/token'));self.assertTrue(self.calls[1][1].startswith('/potal/file/back/page?'))
def test_explicit_rejection_never_reads_catalog(self):
self.login_code=401;result=self.client.check();self.assertEqual(result['state'],'auth_failed');self.assertEqual(len(self.calls),1)
def test_permission_failure_and_transient_remain_distinct(self):
self.list_status=403;self.assertEqual(self.client.check()['state'],'permission_denied');self.list_status=429;self.assertEqual(self.client.check()['state'],'unavailable')
def test_latest_saved_active_config_owns_each_new_task(self):
with tempfile.TemporaryDirectory() as root:
p=pathlib.Path(root)/'settings.json';p.write_text(json.dumps({'active':{'id':'one','base_url':'https://one.example.invalid','username':'example','password':'example-password'}}));p.chmod(0o600)
self.assertEqual(pp_client.load_config(str(p))['id'],'one');p.write_text(json.dumps({'active':{'id':'two','base_url':'https://two.example.invalid','username':'example','password':'example-password'}}));self.assertEqual(pp_client.load_config(str(p))['base_url'],'https://two.example.invalid')
def test_refresh_consumer_uses_the_shared_client(self):
import live_catalog_refresh
with patch('live_catalog_refresh.active_client',return_value=self.client),patch('live_catalog_refresh.build_pp_live_source',side_effect=lambda inventory,manifest:inventory),contextlib.redirect_stdout(io.StringIO()):
result=live_catalog_refresh.refresh_pp()
self.assertEqual(result['items'],[]);self.assertEqual([x[0] for x in self.calls],['POST','GET'])
def test_download_consumer_uses_current_portal_and_real_file_bytes(self):
import artifact_fetch
url=self.client.base+'/file.pdf'
self.items=[{'online':1,'potalGroup':{'groupName':'公开'},'fileUrl':url}]
with patch('pp_client.active_client',return_value=self.client),patch('artifact_fetch.pp_find_entry',return_value=None):
path,size,digest,local=artifact_fetch.pp_download({'locator_hash':hashlib.sha256(url.encode()).hexdigest(),'name':'example.pdf'},1024)
try:
self.assertFalse(local);self.assertGreater(size,0);self.assertEqual(hashlib.sha256(pathlib.Path(path).read_bytes()).hexdigest(),digest)
self.assertEqual([x[0] for x in self.calls],['POST','GET','GET'])
finally:os.unlink(path)
if __name__=='__main__':unittest.main()
+29 -2
View File
@@ -12,7 +12,20 @@ import type { DWSCredentials, DWSLogin } from './dwsLogin'
const messageOf = (cause: unknown) => (cause instanceof Error ? cause.message : '请求失败,请重试')
const moment = (value: string) => new Date(value).toLocaleString('zh-CN', { hour12: false })
export function DWSAccount() {
export function DWSAccount({
autoStart = false,
onAuthorized,
onCredentials,
onCancel,
}: {
autoStart?: boolean
onAuthorized?: (credentials: DWSCredentials) => void
onCredentials?: (credentials: DWSCredentials) => void
onCancel?: () => void
} = {}) {
const callbacks = useRef({ onAuthorized, onCredentials, onCancel })
callbacks.current = { onAuthorized, onCredentials, onCancel }
const startedAutomatically = useRef(false)
const [credentials, setCredentials] = useState<DWSCredentials | null>(null)
const [loading, setLoading] = useState(true)
const [error, setError] = useState('')
@@ -34,7 +47,10 @@ export function DWSAccount() {
mounted.current = true
const controller = new AbortController()
void fetchDWSCredentials(controller.signal)
.then(setCredentials)
.then((fresh) => {
setCredentials(fresh)
callbacks.current.onCredentials?.(fresh)
})
.catch((cause) => {
if (!controller.signal.aborted) setError(messageOf(cause))
})
@@ -101,6 +117,13 @@ export function DWSAccount() {
}
}
useEffect(() => {
if (autoStart && !startedAutomatically.current) {
startedAutomatically.current = true
void start()
}
}, [autoStart])
const close = async () => {
const current = generation.current
const sessionID = id.current
@@ -110,6 +133,7 @@ export function DWSAccount() {
++generation.current
setOpen(false)
setStarting(false)
callbacks.current.onCancel?.()
return
}
if (sessionID && session && dwsLoginActive(session.state)) {
@@ -125,6 +149,7 @@ export function DWSAccount() {
id.current = ''
setOpen(false)
setSession(null)
callbacks.current.onCancel?.()
}
}
@@ -159,7 +184,9 @@ export function DWSAccount() {
if (!mounted.current || current !== generation.current) return
setCredentials(fresh)
setNotice('DWS 登录成功')
callbacks.current.onCredentials?.(fresh)
void check()
callbacks.current.onAuthorized?.(fresh)
})
.catch((cause) => {
if (mounted.current && current === generation.current) setError(`授权成功,状态刷新失败:${messageOf(cause)}`)
+9 -2
View File
@@ -8,11 +8,18 @@ describe('DingtalkSourceDiscovery', () => {
const html = renderToStaticMarkup(
<DingtalkSourceDiscovery registeredNames={['ATOM知识库']} onRegistered={vi.fn()} />
)
for (const text of ['dws 导入(钉钉)', '刷新列表', '加入为来源(选中 0)', '已是来源']) {
for (const text of ['添加数据源向导', '类型', '授权', '范围', '校验', '保存', '钉钉知识库', '钉盘']) {
expect(html).toContain(text)
}
// 需求(2026-10-10):数据源配置这里不再做文件格式筛选——格式归目录级标注决定。
for (const removed of ['文件格式筛选', '未勾选的一律不取件', '保存保留格式', '恢复继承上级规则']) {
for (const removed of [
'文件格式筛选',
'未勾选的一律不取件',
'保存保留格式',
'恢复继承上级规则',
'扫码会话',
'弹二维码',
]) {
expect(html).not.toContain(removed)
}
})
+3 -184
View File
@@ -1,21 +1,6 @@
import { useCallback, useEffect, useState } from 'react'
import { DingtalkRequestError, registerDingtalkSpace } from './dingtalkSession'
import { DingtalkSessionControl } from './DingtalkSessionControl'
import {
discoveryCandidates,
fetchAvailableDingtalkSources,
type AvailableDingtalkSources,
type DiscoveryCandidate,
} from './dingtalkSources'
// 「系统设置 → 数据源配置」里的数据源发现面板(管理员):
// ① 列出当前钉钉账号(dws)能看到的钉钉知识库;
// ② 列出钉盘空间(我的文件 / 组织空间);
// ③ 勾选后加入为来源(登记走既有路径,首次目录扫描自动排队);
// ④ 登记需要有效的钉钉扫码会话:会话失效时直接弹二维码,扫完自动重试本次登记。
// 这里不显示文件格式筛选:格式由目录级标注决定,不在这张卡上做。
import { SourceWizard } from './SourceWizard'
// Keep historical callers on the same DWS authorization and recovery path.
export function DingtalkSourceDiscovery({
registeredNames,
onRegistered,
@@ -23,171 +8,5 @@ export function DingtalkSourceDiscovery({
registeredNames: string[]
onRegistered: () => void
}) {
const [list, setList] = useState<AvailableDingtalkSources | null>(null)
const [loading, setLoading] = useState(false)
const [loadError, setLoadError] = useState<string | null>(null)
const [checked, setChecked] = useState<string[]>([])
const [busy, setBusy] = useState(false)
const [message, setMessage] = useState<string | null>(null)
const [failure, setFailure] = useState<string | null>(null)
const [loginRequired, setLoginRequired] = useState(false)
const load = useCallback(async (signal?: AbortSignal) => {
setLoading(true)
setLoadError(null)
try {
const next = await fetchAvailableDingtalkSources(signal)
setList(next)
// 已登记的默认不再勾选:避免重复登记。
setChecked([])
} catch (cause) {
if (cause instanceof DOMException && cause.name === 'AbortError') return
setLoadError(cause instanceof Error ? cause.message : '列举数据源失败')
setList(null)
} finally {
setLoading(false)
}
}, [])
useEffect(() => {
const controller = new AbortController()
void load(controller.signal)
return () => controller.abort()
}, [load])
const candidates = discoveryCandidates(list, registeredNames)
const knowledgeBases = candidates.filter((item) => item.kind === 'knowledge_base')
const driveSpaces = candidates.filter((item) => item.kind === 'drive_space')
const selected = candidates.filter((item) => checked.includes(item.key))
const toggle = (key: string) =>
setChecked((current) => (current.includes(key) ? current.filter((item) => item !== key) : [...current, key]))
const submit = async (retryAfterLogin = false) => {
setBusy(true)
setFailure(null)
setMessage(null)
const added: string[] = []
const failed: string[] = []
let needLogin = false
for (const candidate of selected) {
try {
const result = await registerDingtalkSpace(candidate.url)
added.push(result.name || candidate.name)
} catch (cause) {
// 扫码会话失效:登记本身没问题,直接弹二维码,扫完自动重试剩下的。
if (cause instanceof DingtalkRequestError && cause.code === 'login_required' && !retryAfterLogin) {
needLogin = true
break
}
failed.push(`${candidate.name}:${cause instanceof Error ? cause.message : '登记失败'}`)
}
}
setBusy(false)
if (needLogin) {
setLoginRequired(true)
setFailure('钉钉扫码会话已失效:请扫码登录,登录成功后会自动继续这次登记。')
return
}
if (added.length) {
setMessage(`已加入 ${added.length} 个来源:${added.join('、')}。首次目录扫描已在后台排队。`)
onRegistered()
void load()
}
if (failed.length) setFailure(failed.join(';'))
}
const resumeAfterLogin = () => {
setLoginRequired(false)
setFailure(null)
void submit(true)
}
const renderRow = (candidate: DiscoveryCandidate) => {
const already = candidate.alreadyRegistered
return (
<li key={candidate.key} className={already ? 'discovery-row registered' : 'discovery-row'}>
<label>
<input
type="checkbox"
aria-label={`选择${candidate.name}`}
checked={checked.includes(candidate.key)}
disabled={already || busy}
onChange={() => toggle(candidate.key)}
/>
<span>
<strong>{candidate.name}</strong>
<small>{candidate.detail}</small>
</span>
</label>
{already ? <span className="safe-badge">已是来源</span> : null}
</li>
)
}
return (
<section className="dingtalk-discovery" aria-label="可加入的数据源">
<div className="dingtalk-discovery-heading">
<div>
<strong>dws 导入(钉钉)</strong>
<small>按当前钉钉账号列出能访问的知识库与钉盘空间;勾选后加入为来源,首次目录扫描自动排队。</small>
</div>
<button type="button" className="secondary" onClick={() => void load()} disabled={loading}>
{loading ? '正在列举…' : '刷新列表'}
</button>
</div>
{loadError ? (
<p className="error-message" role="alert">
{loadError}
</p>
) : null}
{list ? (
<div className="dingtalk-discovery-groups">
<div className="discovery-group">
<h4>知识库({knowledgeBases.length})</h4>
{list.knowledge_notice ? (
<small className="empty-inline">知识库列表可能不全:{list.knowledge_notice}</small>
) : null}
{knowledgeBases.length ? (
<ul>{knowledgeBases.map(renderRow)}</ul>
) : (
<small className="empty-inline">当前账号下没有可访问的知识库。</small>
)}
</div>
<div className="discovery-group">
<h4>钉盘({driveSpaces.length})</h4>
{list.drive_notice ? <small className="empty-inline">钉盘这次列不出来:{list.drive_notice}</small> : null}
{!list.drive_notice && !driveSpaces.length ? (
<small className="empty-inline">当前账号下没有可见的钉盘空间。</small>
) : null}
{driveSpaces.length ? <ul>{driveSpaces.map(renderRow)}</ul> : null}
</div>
</div>
) : null}
<div className="dingtalk-discovery-actions">
<button type="button" disabled={!selected.length || busy} onClick={() => void submit()}>
{busy ? '正在加入…' : `加入为来源(选中 ${selected.length})`}
</button>
<small>勾选的知识库会登记为来源连接;已登记的条目标为「已是来源」,不会重复登记。</small>
</div>
{message ? (
<p className="success-message" role="status">
{message}
</p>
) : null}
{failure ? (
<p className="error-message" role="alert">
{failure}
</p>
) : null}
{loginRequired ? (
<DingtalkSessionControl
autoStart
hideTrigger
space={selected[0]?.url}
onSuccess={resumeAfterLogin}
onClose={() => setLoginRequired(false)}
/>
) : null}
</section>
)
return <SourceWizard registeredNames={registeredNames} ppConnected={false} onRegistered={onRegistered} initialOpen />
}
+93
View File
@@ -0,0 +1,93 @@
.pp-connection {
border: 1px solid var(--border, #dce3eb);
border-radius: 10px;
padding: 16px;
margin: 12px 0;
background: var(--surface, #fff);
}
.pp-heading,
.pp-actions,
.pp-password {
display: flex;
align-items: center;
gap: 10px;
flex-wrap: wrap;
}
.pp-heading {
justify-content: space-between;
}
.pp-heading h3 {
margin: 0;
font-size: 15px;
}
.pp-state {
color: var(--text-muted, #66758a);
font-size: 13px;
}
.pp-connection dl {
margin: 12px 0;
}
.pp-connection dl > div {
display: flex;
gap: 12px;
margin: 6px 0;
font-size: 13px;
}
.pp-connection dt {
flex: 0 0 76px;
color: var(--text-muted, #66758a);
}
.pp-connection dd {
margin: 0;
overflow-wrap: anywhere;
}
.pp-connection form {
display: grid;
gap: 12px;
margin-top: 14px;
}
.pp-connection label {
display: grid;
gap: 5px;
font-size: 13px;
}
.pp-connection input {
width: 100%;
max-width: 540px;
box-sizing: border-box;
padding: 9px 11px;
border: 1px solid var(--border, #cdd7e4);
border-radius: 6px;
background: var(--surface, #fff);
color: inherit;
}
.pp-connection button {
padding: 7px 12px;
border: 1px solid var(--border, #cdd7e4);
border-radius: 6px;
background: var(--surface, #fff);
color: inherit;
cursor: pointer;
}
.pp-connection button.primary {
background: #2563eb;
color: #fff;
border-color: #2563eb;
}
.pp-connection button:disabled {
opacity: 0.5;
cursor: default;
}
.pp-connection [role='alert'] {
color: #a93835;
}
.pp-connection [role='status'] {
font-size: 13px;
}
.pp-connection.compact {
margin-top: 0;
}
.pp-test-result {
padding-top: 8px;
border-top: 1px solid var(--border, #dce3eb);
}
+76
View File
@@ -0,0 +1,76 @@
// @vitest-environment jsdom
import { act } from 'react'
import { createRoot, type Root } from 'react-dom/client'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { PPConnection } from './PPConnection'
let root: Root, host: HTMLDivElement
let fetcher: ReturnType<typeof vi.fn<(path: string, init?: RequestInit) => Promise<Response>>>
const active = {
id: 'active-one',
base_url: 'https://api.example.invalid',
username: 'example',
password_configured: true,
}
const view = {
revision: 1,
active,
pending: null,
observation: { state: 'valid', message: '认证有效 · 目录可读', checked_at: '2026-11-17T10:00:00Z' },
pending_observation: null,
}
const json = (data: unknown) => new Response(JSON.stringify({ data }))
const click = async (label: string) => {
const b = [...host.querySelectorAll('button')].find((x) => x.textContent?.trim() === label)
expect(b, label).toBeTruthy()
await act(async () => b!.click())
}
const type = async (label: string, value: string) => {
const input = [...host.querySelectorAll('label')]
.find((x) => x.textContent?.startsWith(label))
?.querySelector('input')
expect(input, label).toBeTruthy()
await act(async () => {
Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value')!.set!.call(input, value)
input!.dispatchEvent(new Event('input', { bubbles: true }))
})
}
beforeEach(() => {
Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true })
host = document.createElement('div')
document.body.append(host)
root = createRoot(host)
fetcher = vi.fn(async (_p: string, init?: RequestInit) =>
json(init?.method === 'PUT' ? { ...view, revision: 2 } : view)
)
vi.stubGlobal('fetch', fetcher)
})
afterEach(async () => {
await act(async () => root.unmount())
host.remove()
vi.unstubAllGlobals()
})
describe('PP configuration editor', () => {
it('keeps the stored password without sending a masked replacement or launching scans', async () => {
await act(async () => root.render(<PPConnection />))
await click('配置')
expect(host.querySelector('input[type=password]')).toBeNull()
await click('保存并校验')
const writes = fetcher.mock.calls.filter(([, i]) => i?.method === 'PUT')
expect(writes).toHaveLength(1)
const data = JSON.parse(String(writes[0][1]?.body))
expect(data).toEqual({ base_url: active.base_url, username: 'example', revision: 1 })
expect(host.textContent).toContain('已保存并生效')
expect(fetcher.mock.calls.every(([p]) => p.startsWith('/api/v1/pp-connection'))).toBe(true)
})
it('sends a newly entered password only on explicit replacement and clears it after save', async () => {
await act(async () => root.render(<PPConnection />))
await click('配置')
await click('更新密码')
await type('新密码', 'example-replacement')
await click('保存并校验')
const write = fetcher.mock.calls.find(([, i]) => i?.method === 'PUT')
expect(JSON.parse(String(write?.[1]?.body)).password).toBe('example-replacement')
expect(host.querySelector('input[type=password]')).toBeNull()
expect(host.textContent).not.toContain('example-replacement')
})
})
+13
View File
@@ -0,0 +1,13 @@
import { renderToStaticMarkup } from 'react-dom/server'
import { describe, expect, it } from 'vitest'
import { PPConnection } from './PPConnection'
describe('PP connection card', () => {
it('starts with an observation gap and exposes configuration without inventing validity', () => {
const html = renderToStaticMarkup(<PPConnection />)
expect(html).toContain('PP 连接')
expect(html).toContain('读取中')
expect(html).toContain('配置')
expect(html).not.toContain('认证有效')
expect(html).not.toContain('公开只读可达')
})
})
+288
View File
@@ -0,0 +1,288 @@
import { useCallback, useEffect, useRef, useState } from 'react'
import {
readPPConnection,
savePPConnection,
testPPConnection,
type PPDraft,
type PPObservation,
type PPView,
} from './ppConnection'
import './PPConnection.css'
const errorText = (e: unknown) => (e instanceof Error ? e.message : '请求失败,请重试')
const when = (s: string) => new Date(s).toLocaleString('zh-CN', { hour12: false })
export function PPConnection({ compact = false }: { compact?: boolean }) {
const [view, setView] = useState<PPView | null>(null)
const [loading, setLoading] = useState(true)
const [error, setError] = useState('')
const [notice, setNotice] = useState('')
const [editing, setEditing] = useState(false)
const [draft, setDraft] = useState<PPDraft>({ base_url: 'https://api.dobot.cn', username: '', revision: 0 })
const [password, setPassword] = useState('')
const [replacing, setReplacing] = useState(false)
const [busy, setBusy] = useState('')
const [tested, setTested] = useState<PPObservation | null>(null)
const mounted = useRef(false)
const sequence = useRef(0)
const action = useRef<AbortController | null>(null)
const formState = useRef({ editing: false, busy: '' })
formState.current = { editing, busy }
const load = useCallback(async () => {
const current = ++sequence.current
try {
const next = await readPPConnection()
if (mounted.current && sequence.current === current) {
setView(next)
setError('')
}
} catch (e) {
if (mounted.current && sequence.current === current) setError(errorText(e))
} finally {
if (mounted.current && sequence.current === current) setLoading(false)
}
}, [])
useEffect(() => {
mounted.current = true
void load()
const refresh = () => {
if (!formState.current.editing && !formState.current.busy) void load()
}
window.addEventListener('pp-config-updated', refresh)
return () => {
mounted.current = false
sequence.current++
action.current?.abort()
window.removeEventListener('pp-config-updated', refresh)
}
}, [load])
const open = () => {
if (!view) return
const source = view.pending ?? view.active
setDraft({
base_url: source?.base_url ?? 'https://api.dobot.cn',
username: source?.username ?? '',
revision: view.revision,
...(view.pending ? { pending_id: view.pending.id } : {}),
})
setPassword('')
setReplacing(false)
setTested(null)
setNotice('')
setError('')
setEditing(true)
}
const resetTest = () => {
setTested(null)
setNotice('')
}
const run = async (save: boolean) => {
if (!view) return
setBusy(save ? 'saving' : 'testing')
setError('')
setNotice('')
setTested(null)
const controller = new AbortController()
action.current = controller
const current = ++sequence.current
const payload: PPDraft = editing
? { ...draft, ...(password ? { password } : {}) }
: {
base_url: view.active?.base_url ?? 'https://api.dobot.cn',
username: view.active?.username ?? '',
revision: view.revision,
}
try {
if (save) {
const next = await savePPConnection(payload, controller.signal)
if (!mounted.current || sequence.current !== current) return
setView(next)
setPassword('')
setReplacing(false)
setEditing(false)
setNotice(next.pending ? '修改已保存为待验证;原生效配置保留。' : 'PP 配置已保存并生效。')
window.dispatchEvent(new Event('pp-config-updated'))
} else {
const result = await testPPConnection(payload, controller.signal)
if (!mounted.current || sequence.current !== current) return
setTested(result)
const next = await readPPConnection(controller.signal)
if (mounted.current && sequence.current === current) setView(next)
}
} catch (e) {
if (!controller.signal.aborted && mounted.current && sequence.current === current) {
setError(errorText(e))
try {
const next = await readPPConnection(controller.signal)
if (mounted.current && sequence.current === current) setView(next)
} catch {
/* Keep the last observation with the visible error. */
}
}
} finally {
if (mounted.current && sequence.current === current) setBusy('')
}
}
const observation = view?.observation
const status = loading
? '读取中…'
: error && !view
? '状态读取失败'
: !view?.active
? '未配置'
: !observation
? '未检查'
: observation.state === 'valid'
? '认证有效 · 目录可读'
: observation.message
return (
<section className={`pp-connection${compact ? ' compact' : ''}`} aria-label="PP 连接">
<div className="pp-heading">
<h3>PP 连接</h3>
<span className={observation?.state === 'valid' ? 'safe-badge' : 'pp-state'}>{status}</span>
</div>
{!editing && (
<>
<dl>
<div>
<dt>API 地址</dt>
<dd>{view?.active?.base_url || '未配置'}</dd>
</div>
<div>
<dt>账号</dt>
<dd>{view?.active?.username || '未配置'}</dd>
</div>
<div>
<dt>密码</dt>
<dd>{view?.active?.password_configured ? '已配置' : '未配置'}</dd>
</div>
<div>
<dt>最近检查</dt>
<dd>{observation ? `${when(observation.checked_at)} · ${observation.message}` : '尚无检查记录'}</dd>
</div>
</dl>
{observation?.previous && (
<p className="hint">
上次结果(旧):{observation.previous.message} · {when(observation.previous.checked_at)}
</p>
)}
{view?.pending && (
<p className="hint">存在待验证修改;{view.active ? '当前仍使用原生效配置。' : '尚无生效配置。'}</p>
)}
<div className="pp-actions">
<button type="button" disabled={!view || !!busy} onClick={open}>
配置
</button>
<button type="button" disabled={!view || !!busy || !view.active} onClick={() => void run(false)}>
{busy === 'testing' ? '正在检查…' : '测试连接'}
</button>
{error && (
<button type="button" disabled={!!busy} onClick={() => void load()}>
刷新状态
</button>
)}
</div>
</>
)}
{editing && (
<form
onSubmit={(e) => {
e.preventDefault()
void run(true)
}}
>
<label>
API 地址
<input
type="url"
value={draft.base_url}
disabled={!!busy}
autoComplete="off"
onChange={(e) => {
setDraft({ ...draft, base_url: e.target.value })
resetTest()
}}
required
/>
</label>
<label>
账号
<input
value={draft.username}
disabled={!!busy}
autoComplete="off"
onChange={(e) => {
setDraft({ ...draft, username: e.target.value })
resetTest()
}}
/>
</label>
<div className="pp-password">
<span>密码:{(view?.pending ?? view?.active)?.password_configured ? '已配置' : '未配置'}</span>
<button
type="button"
disabled={!!busy}
onClick={() => {
setReplacing(!replacing)
setPassword('')
resetTest()
}}
>
{replacing ? '保留已存密码' : '更新密码'}
</button>
</div>
{replacing && (
<label>
新密码
<input
type="password"
autoComplete="new-password"
value={password}
disabled={!!busy}
placeholder="留空保留已存密码"
onChange={(e) => {
setPassword(e.target.value)
resetTest()
}}
/>
</label>
)}
<div className="pp-actions">
<button
type="button"
disabled={!!busy}
onClick={() => {
setEditing(false)
setPassword('')
setTested(null)
setError('')
}}
>
取消修改
</button>
<button type="button" disabled={!!busy} onClick={() => void run(false)}>
{busy === 'testing' ? '正在检查…' : '测试连接'}
</button>
<button type="submit" className="primary" disabled={!!busy}>
{busy === 'saving' ? '正在校验…' : '保存并校验'}
</button>
</div>
</form>
)}
{tested && (
<p role="status" className="pp-test-result">
{editing ? '未保存配置测试:' : '当前配置检查:'}
{tested.message}({when(tested.checked_at)})
</p>
)}
{notice && <p role="status">{notice}</p>}
{error && (
<p role="alert">
{error}
{view && ';上次读取的配置与检查结果仍保留。'}
</p>
)}
</section>
)
}
export function PPAuthorizationStep() {
return <PPConnection compact />
}
+168
View File
@@ -0,0 +1,168 @@
// @vitest-environment jsdom
import { act } from 'react'
import { createRoot, type Root } from 'react-dom/client'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { SourceWizard } from './SourceWizard'
let root: Root, host: HTMLDivElement
let fetcher: ReturnType<typeof vi.fn<(path: string, init?: RequestInit) => Promise<Response>>>
const reply = (data: unknown, status = 200) => new Response(JSON.stringify({ data }), { status })
const account = {
authenticated: true,
refresh_token_valid: true,
status_known: true,
identity: 'account-a',
corp_name: '示例组织',
user_name: '示例账号',
}
const spaces = {
knowledge_bases: [
{ name: '资料一', space_id: 'space-one', url: 'https://alidocs.dingtalk.com/i/spaces/space-one/overview' },
{ name: '资料二', space_id: 'space-two', url: 'https://alidocs.dingtalk.com/i/spaces/space-two/overview' },
],
drive_spaces: [],
fetched_at: '2026-11-17T10:00:00Z',
}
const click = async (text: string) => {
const b = [...host.querySelectorAll('button')].find(
(x) => x.textContent?.trim() === text || x.textContent?.trim().includes(text)
)
expect(b, text).toBeTruthy()
await act(async () => b!.click())
}
const flush = async () => {
await act(async () => {
await Promise.resolve()
})
}
beforeEach(() => {
Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true })
vi.useFakeTimers()
host = document.createElement('div')
document.body.append(host)
root = createRoot(host)
fetcher = vi.fn(async (path: string) => {
if (path.endsWith('credential-status')) return reply(account)
if (path.endsWith('/available')) return reply(spaces)
if (path.endsWith('/validate')) return reply({ valid: true, count: 2, checked_at: '2026-11-17T10:00:00Z' })
throw new Error('unexpected ' + path)
})
vi.stubGlobal('fetch', fetcher)
})
afterEach(async () => {
await act(async () => root.unmount())
host.remove()
vi.useRealTimers()
vi.unstubAllGlobals()
})
async function selectTwo() {
await act(async () => root.render(<SourceWizard registeredNames={[]} ppConnected onRegistered={() => {}} />))
await click('+ 添加数据源')
await click('钉钉知识库')
await click('下一步')
await flush()
await click('下一步')
await act(async () => {
for (const box of host.querySelectorAll<HTMLInputElement>('input[type=checkbox]')) box.click()
})
await click('下一步')
await flush()
}
describe('DWS source wizard', () => {
it('uses shared DWS identity and really validates the selected spaces', async () => {
await selectTwo()
expect(host.textContent).not.toContain('扫码会话')
expect(fetcher.mock.calls.some(([p]) => String(p).endsWith('/validate'))).toBe(true)
expect(host.textContent).toContain('校验通过')
})
it('resumes only the unregistered space after inline DWS authorization', async () => {
const fallback = fetcher.getMockImplementation()!
let tries = 0
fetcher.mockImplementation(async (path: string, init?: RequestInit) => {
if (path === '/api/v1/dingtalk-spaces') {
const url = JSON.parse(String(init?.body)).url
if (url.includes('space-two') && ++tries === 1)
return new Response(JSON.stringify({ error: 'dws_login_required', hint: '授权已失效' }), { status: 401 })
return reply(
{
connection_id: url.includes('space-one') ? 'one' : 'two',
name: url.includes('space-one') ? '资料一' : '资料二',
first_scan_queued: true,
},
201
)
}
if (path === '/api/v1/dws-login')
return reply(
{
id: 'a'.repeat(32),
state: 'waiting',
expires_at: '2099-01-01T00:00:00Z',
verification_url: 'https://login.dingtalk.com/oauth2/device/verify.htm?user_code=EXAMPLE',
user_code: 'EXAMPLE',
},
202
)
if (path.startsWith('/api/v1/dws-login/'))
return reply({
id: 'a'.repeat(32),
state: init?.method === 'DELETE' ? 'cancelled' : 'succeeded',
expires_at: '2099-01-01T00:00:00Z',
verification_url: '',
user_code: '',
})
return fallback(path, init)
})
await selectTwo()
await click('保存并首次扫描')
await flush()
expect(host.textContent).toContain('DWS 登录授权')
await act(async () => vi.advanceTimersByTimeAsync(1500))
await flush()
const calls = fetcher.mock.calls.filter(([p]) => p === '/api/v1/dingtalk-spaces')
expect(calls.filter(([, i]) => String((i as RequestInit).body).includes('space-one'))).toHaveLength(1)
const second = calls.filter(([, i]) => String((i as RequestInit).body).includes('space-two'))
expect(second).toHaveLength(2)
expect((second[0][1] as RequestInit).headers).toEqual((second[1][1] as RequestInit).headers)
expect(fetcher.mock.calls.some(([p]) => String(p).includes('/dingtalk-session'))).toBe(false)
})
it('requires a new selection when authorization changes the account', async () => {
const fallback = fetcher.getMockImplementation()!
let authorized = false
fetcher.mockImplementation(async (path: string, init?: RequestInit) => {
if (path.endsWith('credential-status'))
return reply({ ...account, identity: authorized ? 'account-b' : 'account-a' })
if (path === '/api/v1/dingtalk-spaces')
return new Response(JSON.stringify({ error: 'dws_login_required', hint: '授权失效' }), { status: 401 })
if (path === '/api/v1/dws-login')
return reply(
{
id: 'b'.repeat(32),
state: 'waiting',
expires_at: '2099-01-01T00:00:00Z',
verification_url: 'https://login.dingtalk.com/oauth2/device/verify.htm?user_code=EXAMPLE',
user_code: 'EXAMPLE',
},
202
)
if (path.startsWith('/api/v1/dws-login/')) {
authorized = true
return reply({
id: 'b'.repeat(32),
state: 'succeeded',
expires_at: '2099-01-01T00:00:00Z',
verification_url: '',
user_code: '',
})
}
return fallback(path, init)
})
await selectTwo()
await click('保存并首次扫描')
await act(async () => vi.advanceTimersByTimeAsync(1500))
await flush()
expect(host.textContent).toContain('账号或组织已变化,请重新确认来源')
expect(fetcher.mock.calls.filter(([p]) => p === '/api/v1/dingtalk-spaces')).toHaveLength(1)
expect([...host.querySelectorAll<HTMLInputElement>('input[type=checkbox]')].every((x) => !x.checked)).toBe(true)
})
})
+340 -196
View File
@@ -1,147 +1,294 @@
import { useCallback, useEffect, useMemo, useState } from 'react'
import { DingtalkRequestError, registerDingtalkSpace } from './dingtalkSession'
import { DingtalkSessionControl } from './DingtalkSessionControl'
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
import { DWSAccount } from './DWSAccount'
import { PPAuthorizationStep } from './PPConnection'
import { DingtalkRequestError, registerDingtalkSpace, type RegisteredDingtalkSpace } from './dingtalkSession'
import { fetchDWSCredentials, type DWSCredentials } from './dwsLogin'
import {
discoveryCandidates,
fetchAvailableDingtalkSources,
validateDingtalkSources,
type AvailableDingtalkSources,
type DiscoveryCandidate,
} from './dingtalkSources'
import { newOperationID } from './operationKey'
import './sourceWizard.css'
// 「系统设置 → 数据源配置」的统一添加向导。
// 一个入口、五步固定:类型 → 授权 → 范围 → 校验 → 保存。
// 类型只决定「候选从哪来」与「授权怎么说明」,版式和动作词完全一致。
// 对内/对外不在这里配置:策略只在「来源与目录」标注。
// 单篇钉钉文档/表格不进这里——把链接发给我,我用 dws 导入。
type SourceKind = 'wiki' | 'drive' | 'pp'
const kinds: { key: SourceKind; title: string; detail: string; icon: string }[] = [
{ key: 'wiki', title: '钉钉知识库', detail: '组织 / 我的知识库', icon: '库' },
{ key: 'drive', title: '钉盘', detail: '我的文件 / 组织空间', icon: '盘' },
{ key: 'pp', title: 'Partner Portal', detail: '公开只读资料中心', icon: '↧' },
const kinds = [
{ key: 'wiki' as const, title: '钉钉知识库', detail: '组织 / 我的知识库', icon: '库' },
{ key: 'drive' as const, title: '钉盘', detail: '我的文件 / 组织空间', icon: '盘' },
{ key: 'pp' as const, title: 'Partner Portal', detail: '资料中心 · 只读访问', icon: '↧' },
]
const steps = ['类型', '授权', '范围', '校验', '保存'] as const
const steps = ['类型', '授权', '范围', '校验', '保存']
const text = (e: unknown) => (e instanceof Error ? e.message : '请求失败,请重试')
const needsLogin = (e: unknown) =>
e instanceof DingtalkRequestError && ['dws_login_required', 'login_required'].includes(e.code)
const validAccount = (c: DWSCredentials | null) =>
Boolean(c?.status_known && (c.authenticated || c.refresh_token_valid))
export function SourceWizard({
registeredNames,
ppConnected,
onRegistered,
initialOpen = false,
}: {
registeredNames: string[]
// PP 是一个门户一条来源:已接入时向导只做说明,不重复登记。
ppConnected: boolean
onRegistered: () => void
initialOpen?: boolean
}) {
const [open, setOpen] = useState(false)
const [open, setOpen] = useState(initialOpen)
const [step, setStep] = useState(0)
const [kind, setKind] = useState<SourceKind | null>(null)
const [list, setList] = useState<AvailableDingtalkSources | null>(null)
const [credentials, setCredentials] = useState<DWSCredentials | null>(null)
const [loading, setLoading] = useState(false)
const [loadError, setLoadError] = useState<string | null>(null)
const [loadError, setLoadError] = useState('')
const [checked, setChecked] = useState<string[]>([])
const [busy, setBusy] = useState(false)
const [failure, setFailure] = useState<string | null>(null)
const [message, setMessage] = useState<string | null>(null)
const [loginRequired, setLoginRequired] = useState(false)
const load = useCallback(async (signal?: AbortSignal) => {
setLoading(true)
setLoadError(null)
try {
setList(await fetchAvailableDingtalkSources(signal))
} catch (cause) {
if (cause instanceof DOMException && cause.name === 'AbortError') return
setLoadError(cause instanceof Error ? cause.message : '列举数据源失败')
setList(null)
} finally {
setLoading(false)
const [failure, setFailure] = useState('')
const [message, setMessage] = useState('')
const [authNeeded, setAuthNeeded] = useState(false)
const [autoAuthorize, setAutoAuthorize] = useState(false)
const [verified, setVerified] = useState('')
const alive = useRef(true)
const epoch = useRef(0)
const completed = useRef(new Map<string, RegisteredDingtalkSpace>())
const pending = useRef<DiscoveryCandidate[]>([])
const operationIDs = useRef(new Map<string, string>())
const identity = useRef('')
const submitted = useRef(false)
const recovering = useRef(false)
const callbacks = useRef({ onRegistered })
callbacks.current = { onRegistered }
useEffect(() => {
alive.current = true
return () => {
alive.current = false
epoch.current++
}
}, [])
const load = useCallback(async () => {
const current = ++epoch.current
setLoading(true)
setLoadError('')
try {
const account = await fetchDWSCredentials()
if (!alive.current || current !== epoch.current) return null
setCredentials(account)
if (identity.current && account.identity && identity.current !== account.identity && !submitted.current) {
setChecked([])
setVerified('')
setFailure('DWS 账号或组织已变化,请重新确认来源。')
identity.current = account.identity
}
if (!identity.current && account.identity) identity.current = account.identity
if (!validAccount(account)) {
setList(null)
setAuthNeeded(true)
setLoadError('请先完成 DWS 登录授权')
return null
}
const result = await fetchAvailableDingtalkSources()
if (!alive.current || current !== epoch.current) return null
setList(result)
return result
} catch (e) {
if (alive.current && current === epoch.current) {
setList(null)
setLoadError(text(e))
if (needsLogin(e)) setAuthNeeded(true)
}
return null
} finally {
if (alive.current && current === epoch.current) setLoading(false)
}
}, [])
useEffect(() => {
if (!open || (kind !== 'wiki' && kind !== 'drive')) return
const controller = new AbortController()
void load(controller.signal)
return () => controller.abort()
if (open && (kind === 'wiki' || kind === 'drive')) void load()
return () => {
epoch.current++
}
}, [open, kind, load])
const candidates = useMemo(() => discoveryCandidates(list, registeredNames), [list, registeredNames])
const visible = useMemo(
() =>
candidates.filter((item) =>
kind === 'wiki' ? item.kind === 'knowledge_base' : kind === 'drive' ? item.kind === 'drive_space' : false
),
[candidates, kind]
const visible = candidates.filter((x) =>
kind === 'wiki' ? x.kind === 'knowledge_base' : kind === 'drive' ? x.kind === 'drive_space' : false
)
const selected = visible.filter((item) => checked.includes(item.key))
const selected = visible.filter((x) => checked.includes(x.key) && !x.alreadyRegistered)
const close = () => {
if (busy) return
epoch.current++
setOpen(false)
setStep(0)
setKind(null)
setChecked([])
setFailure(null)
setMessage(null)
setLoginRequired(false)
setFailure('')
setMessage('')
setAuthNeeded(false)
setAutoAuthorize(false)
setVerified('')
identity.current = ''
submitted.current = false
pending.current = []
completed.current.clear()
operationIDs.current.clear()
}
const toggle = (key: string) =>
setChecked((current) => (current.includes(key) ? current.filter((item) => item !== key) : [...current, key]))
const submit = async (retryAfterLogin = false) => {
const report = () => {
const values = [...completed.current.values()]
const queued = values.filter((x) => x.first_scan_queued === true).length
setMessage(
values.length
? `已加入 ${values.length} 个来源:${values.map((x) => x.name).join('、')}。${queued ? `首次目录扫描已入队 ${queued} 项。` : ''}${values.some((x) => x.first_scan_queued === false) ? '部分首次扫描尚未入队,请检查来源状态。' : ''}`
: ''
)
}
const submit = async (items: DiscoveryCandidate[] = selected) => {
if (busy && !recovering.current) return
setBusy(true)
setFailure(null)
const added: string[] = []
const failed: string[] = []
let needLogin = false
for (const candidate of selected) {
setFailure('')
submitted.current = true
pending.current = items.filter((x) => !completed.current.has(x.key))
const remaining = [...pending.current]
let changed = false
for (const candidate of remaining) {
if (!alive.current) break
let operation = operationIDs.current.get(candidate.key)
if (!operation) {
operation = newOperationID()
operationIDs.current.set(candidate.key, operation)
}
try {
const result = await registerDingtalkSpace(candidate.url)
added.push(result.name || candidate.name)
} catch (cause) {
if (cause instanceof DingtalkRequestError && cause.code === 'login_required' && !retryAfterLogin) {
needLogin = true
break
const result = await registerDingtalkSpace(candidate.url, operation, identity.current || undefined)
if (!alive.current) return
completed.current.set(candidate.key, result)
pending.current = pending.current.filter((x) => x.key !== candidate.key)
changed = true
} catch (e) {
if (!alive.current) return
if (needsLogin(e)) {
setAuthNeeded(true)
setAutoAuthorize(!recovering.current)
setFailure('DWS 授权已失效;重新授权后继续剩余来源。')
setBusy(false)
report()
if (changed) callbacks.current.onRegistered()
return
}
failed.push(`${candidate.name}:${cause instanceof Error ? cause.message : '登记失败'}`)
if (e instanceof DingtalkRequestError && e.code === 'dws_identity_changed') {
pending.current = []
submitted.current = false
setChecked([])
setStep(2)
setVerified('')
setFailure('DWS 账号或组织已变化,请重新确认来源。')
void load()
} else setFailure(`${candidate.name}:${text(e)}。已成功项保留,可重试剩余来源。`)
setBusy(false)
report()
if (changed) callbacks.current.onRegistered()
return
}
}
setBusy(false)
if (needLogin) {
setLoginRequired(true)
setFailure('钉钉扫码会话已失效:扫码后会自动继续这次添加。')
if (alive.current) {
setBusy(false)
setAuthNeeded(false)
setAutoAuthorize(false)
submitted.current = false
report()
if (changed) callbacks.current.onRegistered()
}
}
const authorized = async (account: DWSCredentials) => {
if (!alive.current) return
setCredentials(account)
setAuthNeeded(false)
setAutoAuthorize(false)
const changed = Boolean(identity.current && account.identity !== identity.current)
if (!identity.current && account.identity) identity.current = account.identity
if (changed) {
identity.current = account.identity || ''
pending.current = []
submitted.current = false
completed.current.clear()
operationIDs.current.clear()
setChecked([])
setVerified('')
setStep(2)
setFailure('DWS 账号或组织已变化,请重新确认来源。')
await load()
return
}
if (added.length) {
setMessage(`已加入 ${added.length} 个来源:${added.join('、')}。首次目录扫描已在后台排队。`)
onRegistered()
void load()
const result = await load()
if (!result || !alive.current) return
if (submitted.current && pending.current.length) {
if (!identity.current || !account.identity) {
pending.current = []
submitted.current = false
setChecked([])
setStep(2)
setVerified('')
setFailure('无法确认授权前后为同一账号,请重新选择来源。')
return
}
const available = new Set(discoveryCandidates(result, []).map((x) => x.key))
if (pending.current.some((x) => !available.has(x.key))) {
setFailure('重新授权后部分来源已不可见,请重新确认范围。')
setStep(2)
setVerified('')
submitted.current = false
return
}
recovering.current = true
await submit([...pending.current])
recovering.current = false
}
if (failed.length) setFailure(failed.join(';'))
}
const canAdvance =
(step === 0 && kind !== null) ||
(step === 1 && kind !== null) ||
(step === 2 && (kind === 'pp' || selected.length > 0)) ||
(step === 3 && (kind === 'pp' || selected.length > 0))
const verify = async () => {
if (!selected.length) return
const current = epoch.current
setStep(3)
setBusy(true)
setVerified('')
setFailure('')
try {
const result = await validateDingtalkSources(
selected.map((x) => x.url),
identity.current || undefined
)
if (alive.current && current === epoch.current) setVerified(result.checked_at)
} catch (e) {
if (alive.current && current === epoch.current) {
setFailure(text(e))
if (needsLogin(e)) setAuthNeeded(true)
}
} finally {
if (alive.current && current === epoch.current) setBusy(false)
}
}
const authPanel = (
<DWSAccount
autoStart={autoAuthorize}
onCredentials={(c) => {
setCredentials(c)
if (!identity.current && c.identity) identity.current = c.identity
}}
onAuthorized={(c) => void authorized(c)}
onCancel={() => {
setAutoAuthorize(false)
setFailure('授权已取消,选择已保留。')
}}
/>
)
const scopeNotice = kind === 'wiki' ? list?.knowledge_notice : list?.drive_notice
return (
<section className="source-wizard" aria-label="添加数据源">
<div className="source-wizard-bar">
<div>
<h3>数据源</h3>
</div>
<h3>数据源</h3>
<button type="button" className="source-wizard-open" onClick={() => setOpen(true)}>
+ 添加数据源
</button>
</div>
{open ? (
{open && (
<div className="source-wizard-panel" role="dialog" aria-label="添加数据源向导">
<ol className="source-wizard-steps">
{steps.map((label, index) => (
@@ -150,8 +297,7 @@ export function SourceWizard({
</li>
))}
</ol>
{step === 0 ? (
{step === 0 && (
<div className="source-wizard-types">
{kinds.map((item) => (
<button
@@ -161,6 +307,9 @@ export function SourceWizard({
onClick={() => {
setKind(item.key)
setChecked([])
setVerified('')
setFailure('')
setAuthNeeded(false)
}}
>
<span aria-hidden="true">{item.icon}</span>
@@ -169,136 +318,131 @@ export function SourceWizard({
</button>
))}
</div>
) : null}
{step === 1 ? (
<div className="source-wizard-auth">
{kind === 'pp' ? (
<>
<div className="row">
<span>授权方式</span>
<b>公开只读</b>
</div>
<div className="row">
<span>凭据</span>
<span>由服务端持有,无需填写</span>
</div>
<p className="hint">资料下载中心的登录凭据由运维放在服务端,向导里不录入任何密码。</p>
</>
) : (
<>
<div className="row">
<span>授权方式</span>
<b>当前钉钉账号扫码会话</b>
</div>
<div className="row">
<span>读取权限</span>
<span>只读;不写回、不删除源文件</span>
</div>
<p className="hint">
用的是平台已登录的 dws 凭据,不需要每个来源单独扫码;会话失效时会在保存时弹二维码。
</p>
</>
)}
</div>
) : null}
{step === 2 ? (
)}
{step === 1 && (
<div className="source-wizard-auth">{kind === 'pp' ? <PPAuthorizationStep /> : authPanel}</div>
)}
{step === 2 && (
<div className="source-wizard-scope">
{kind === 'pp' ? (
<p className="hint">
{ppConnected
? '已接入 PP 资料下载中心(一个门户一条来源),无需重复添加;刷新与归档在下方清单里做。'
: 'PP 需要服务端放置只读凭据后才能接入;凭据到位后联系我开通,向导本身不录密码。'}
? 'PP 资料下载中心已接入,一个门户保留一条来源。'
: 'PP 来源尚未接入;认证配置与来源登记分别管理。'}
</p>
) : loading ? (
<p className="hint">正在列举当前账号可见的来源…</p>
<p>正在列举当前 DWS 账号可见的来源…</p>
) : loadError ? (
<p role="alert">{loadError}</p>
) : visible.length === 0 ? (
<p className="hint">
当前账号没有可添加的{kind === 'wiki' ? '知识库' : '钉盘空间'};已加入的不会重复列出。
</p>
<>
<p role="alert">{loadError}</p>
<button onClick={() => void load()}>重新读取目录</button>
</>
) : (
<>
<div className="source-wizard-check">
{visible.map((item) => (
<label key={item.key}>
<input type="checkbox" checked={checked.includes(item.key)} onChange={() => toggle(item.key)} />
<span>{item.name}</span>
</label>
))}
</div>
<p className="hint">
已选 {selected.length} 项。勾选空间即可,目录细节加入后再在「来源与目录」里标注。
</p>
{scopeNotice && <p role="alert">{scopeNotice}</p>}
{visible.length ? (
<div className="source-wizard-check">
{visible.map((item) => (
<label key={item.key}>
<input
type="checkbox"
checked={checked.includes(item.key)}
disabled={item.alreadyRegistered || busy}
onChange={() => {
setChecked((current) =>
current.includes(item.key)
? current.filter((x) => x !== item.key)
: [...current, item.key]
)
setVerified('')
}}
/>
<span>
{item.name}
{item.alreadyRegistered ? '(已接入)' : ''}
</span>
</label>
))}
</div>
) : (
<p>当前账号没有可添加的{kind === 'wiki' ? '知识库' : '钉盘空间'}。</p>
)}
<p className="hint">已选 {selected.length} 项;目录规则在加入后的“来源与目录”设置。</p>
</>
)}
</div>
) : null}
{step === 3 ? (
)}
{step === 3 && (
<div className="source-wizard-verify">
<ul>
{(kind === 'pp' ? [] : selected).map((item: DiscoveryCandidate) => (
<li key={item.key}>
<span>{item.name}</span>
<b className="ok">仍可见</b>
</li>
))}
{kind === 'pp' ? (
<li>
<span>Partner Portal 公开目录</span>
<b className="ok">公开只读可达</b>
</li>
) : null}
</ul>
<p className="hint">添加时会按当前账号再读一次;如果已看不到,会拒绝并提示刷新。</p>
{busy ? (
<p>正在校验所选来源…</p>
) : verified ? (
<>
<ul>
{selected.map((item) => (
<li key={item.key}>
<span>{item.name}</span>
<b className="ok">校验通过</b>
</li>
))}
</ul>
<small>检查于 {new Date(verified).toLocaleString('zh-CN', { hour12: false })}</small>
</>
) : (
<button onClick={() => void verify()}>重新校验</button>
)}
</div>
) : null}
{step === 4 ? (
)}
{step === 4 && (
<div className="source-wizard-save">
{message ? <p role="status">{message}</p> : null}
{failure ? <p role="alert">{failure}</p> : null}
{!message && !failure ? (
<p className="hint">
将加入{' '}
{(kind === 'pp' ? ['Partner Portal'] : selected.map((item) => item.name)).join('、') || '(未选择)'}
。
</p>
) : null}
{loginRequired ? (
<DingtalkSessionControl
autoStart
hideTrigger
space={selected[0]?.url}
onSuccess={() => {
setLoginRequired(false)
void submit(true)
}}
/>
) : null}
{message && <p role="status">{message}</p>}
{pending.current.length > 0 && !authNeeded && !busy && (
<button onClick={() => void submit([...pending.current])}>重试剩余来源</button>
)}
</div>
) : null}
)}
{failure && <p role="alert">{failure}</p>}
{authNeeded && step !== 1 && authPanel}
<div className="source-wizard-foot">
<button type="button" className="ghost" onClick={() => (step === 0 ? close() : setStep(step - 1))}>
<button
type="button"
className="ghost"
disabled={busy}
onClick={() => {
setVerified('')
setFailure('')
step === 0 ? close() : setStep(step - 1)
}}
>
{step === 0 ? '取消' : '上一步'}
</button>
{kind === 'pp' ? (
{kind === 'pp' && step === 1 ? (
<button type="button" className="primary" onClick={close}>
完成
</button>
) : step < 3 ? (
<button type="button" className="primary" disabled={!canAdvance} onClick={() => setStep(step + 1)}>
) : step < 2 ? (
<button
type="button"
className="primary"
disabled={busy || kind === null || (step === 1 && !validAccount(credentials))}
onClick={() => setStep(step + 1)}
>
下一步
</button>
) : step === 2 ? (
<button
type="button"
className="primary"
disabled={busy || !selected.length}
onClick={() => void verify()}
>
下一步
</button>
) : step === 3 ? (
<button
type="button"
className="primary"
disabled={busy || !canAdvance}
disabled={busy || !verified}
onClick={() => {
setStep(4)
void submit()
@@ -307,13 +451,13 @@ export function SourceWizard({
{busy ? '正在加入…' : '保存并首次扫描'}
</button>
) : (
<button type="button" className="primary" onClick={close}>
<button type="button" className="primary" disabled={busy} onClick={close}>
完成
</button>
)}
</div>
</div>
) : null}
)}
</section>
)
}
+7 -1
View File
@@ -8,6 +8,7 @@ import { DingtalkAccountStatus, SourceManagementDetails, SourceSettingsCard } fr
import { RoutingBindingsSection } from './RoutingBindingsSection'
import { ProductionBindingsSection } from './ProductionBindingsSection'
import { SourceWizard } from './SourceWizard'
import { PPConnection } from './PPConnection'
import { useOptionalAuth } from './auth'
import { fetchMonitorOverview } from './monitor'
import type { MonitorOverview } from './monitor'
@@ -297,7 +298,12 @@ export function SystemSettingsView({
onRegistered={onSourcesChanged}
/>
) : null}
{identity?.role === 'admin' ? <DingtalkAccountStatus /> : null}
{identity?.role === 'admin' ? (
<>
<DingtalkAccountStatus />
<PPConnection />
</>
) : null}
<div className="settings-source-group" id="settings-sources-pp">
{listedConnections.length ? (
<div className="settings-source-list">
+18 -4
View File
@@ -65,6 +65,7 @@ export async function cancelDingtalkSession(id: string): Promise<void> {
export interface RegisteredDingtalkSpace {
connection_id: string
name: string
first_scan_queued?: boolean
}
export class DingtalkRequestError extends Error {
@@ -81,14 +82,19 @@ export class DingtalkRequestError extends Error {
// Register a DingTalk knowledge-base space (URL or bare space id) as a new
// source connection. Errors carry the sidecar's machine codes so the UI can
// explain them: login_required, space_not_found, name_conflict.
export async function registerDingtalkSpace(url: string): Promise<RegisteredDingtalkSpace> {
export async function registerDingtalkSpace(
url: string,
operationID = newOperationID(),
identity?: string
): Promise<RegisteredDingtalkSpace> {
const response = await fetch('/api/v1/dingtalk-spaces', {
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-Governance-Action': 'dingtalk-space',
'Idempotency-Key': newOperationID(),
'Idempotency-Key': operationID,
...(identity ? { 'X-DWS-Identity': identity } : {}),
},
body: JSON.stringify({ url }),
})
@@ -106,7 +112,11 @@ export async function registerDingtalkSpace(url: string): Promise<RegisteredDing
/* status-only */
}
const friendly: Record<string, string> = {
login_required: '钉钉扫码会话无效或已过期,请先在钉钉来源卡片上「扫码刷新会话」',
login_required: 'DWS 授权已失效,请重新授权',
dws_login_required: 'DWS 授权已失效,请重新授权',
dws_unavailable: 'DWS 目录暂时不可用,请重试',
dws_permission_denied: '当前 DWS 账号缺少目录权限',
dws_identity_changed: 'DWS 账号或组织已变化,请重新确认来源',
space_not_found: '未找到该知识库空间:请确认空间链接正确且当前账号有阅读权限',
name_conflict: '该空间与已登记的来源同名',
// 登记冲突分两种:同名来源还在用 vs 同一个登记键指向了别的空间。原因要说清,
@@ -127,7 +137,11 @@ export async function registerDingtalkSpace(url: string): Promise<RegisteredDing
const data = value.data
if (typeof data.connection_id !== 'string' || typeof data.name !== 'string')
throw new Error('invalid dingtalk space response')
return { connection_id: data.connection_id, name: data.name }
return {
connection_id: data.connection_id,
name: data.name,
...(typeof data.first_scan_queued === 'boolean' ? { first_scan_queued: data.first_scan_queued } : {}),
}
}
function parseSession(value: unknown): DingtalkSessionView {
+1
View File
@@ -7,6 +7,7 @@ const payload = {
knowledge_bases: [
{
name: 'ATOM知识库',
registered: true,
space_id: '9JOGOQneRgkneX4Q',
space_type: 'orgWikiSpace',
url: 'https://alidocs.dingtalk.com/i/spaces/9JOGOQneRgkneX4Q/overview',
+49 -10
View File
@@ -1,7 +1,10 @@
// 「系统设置 → 数据源配置」的数据源发现客户端:列出当前 dws 账号能看到的钉钉知识库与钉盘空间。
// 只读;勾选后的登记仍走既有的 /api/v1/dingtalk-spaces(由扫码会话解析空间根目录)。
import { DingtalkRequestError } from './dingtalkSession'
export interface AvailableKnowledgeBase {
registered?: boolean
name: string
space_id: string
space_type?: string
@@ -10,6 +13,7 @@ export interface AvailableKnowledgeBase {
}
export interface AvailableDriveSpace {
registered?: boolean
name: string
space_id: string
space_type: string
@@ -38,12 +42,17 @@ function isRecord(value: unknown): value is Record<string, unknown> {
export async function fetchAvailableDingtalkSources(signal?: AbortSignal): Promise<AvailableDingtalkSources> {
const response = await fetch('/api/v1/dingtalk-spaces/available', { headers: { Accept: 'application/json' }, signal })
if (!response.ok) {
// 503 是环境问题(dws 未登录/权限不足),501 是后端没接线:都要说清楚"不是没有来源"。
if (response.status === 503)
throw new Error('暂时列不出来:dws 未登录或缺少租户目录读权限。先刷新钉钉会话,或让管理员检查 dws 凭据。')
if (response.status === 501) throw new Error('后端未接线:数据源发现接口不可用。')
if (response.status === 401 || response.status === 403) throw new Error('需要管理员身份才能列举数据源。')
throw new Error(`列举数据源失败(HTTP ${response.status})`)
const body: unknown = await response.json().catch(() => null)
const code = isRecord(body) && typeof body.error === 'string' ? body.error : 'request_failed'
const hint =
isRecord(body) && typeof body.hint === 'string'
? body.hint
: response.status === 501
? '后端未接线:DWS 来源服务不可用。'
: response.status === 401 || response.status === 403
? '需要管理员身份才能列举数据源。'
: `目录暂时列不出来(HTTP ${response.status})`
throw new DingtalkRequestError(hint, code, response.status)
}
const payload = (await response.json()) as unknown
const data = isRecord(payload) && isRecord(payload.data) ? payload.data : {}
@@ -60,6 +69,7 @@ export async function fetchAvailableDingtalkSources(signal?: AbortSignal): Promi
? [
{
name,
registered: item.registered === true,
space_id: spaceID,
space_type: spaceType,
url,
@@ -77,6 +87,7 @@ export async function fetchAvailableDingtalkSources(signal?: AbortSignal): Promi
? [
{
name,
registered: item.registered === true,
space_id: spaceID,
space_type: spaceType,
root_folder_id: typeof item.root_folder_id === 'string' ? item.root_folder_id : '',
@@ -103,9 +114,8 @@ export interface DiscoveryCandidate {
// 避免重复登记。
export function discoveryCandidates(
list: AvailableDingtalkSources | null,
registeredNames: string[]
_registeredNames: string[]
): DiscoveryCandidate[] {
const registered = new Set(registeredNames.map((name) => name.trim()).filter(Boolean))
const knowledgeBases = (list?.knowledge_bases ?? []).map((item): DiscoveryCandidate => ({
key: `kb:${item.space_id}`,
kind: 'knowledge_base',
@@ -113,7 +123,7 @@ export function discoveryCandidates(
spaceID: item.space_id,
url: item.url || driveSpaceURL(item.space_id),
detail: item.space_type === 'myWikiSpace' ? '钉钉知识库 · 我的知识库' : '钉钉知识库 · 组织知识库',
alreadyRegistered: registered.has(item.name.trim()),
alreadyRegistered: item.registered === true,
}))
const driveSpaces = (list?.drive_spaces ?? []).map((item): DiscoveryCandidate => ({
key: `drive:${item.space_id}`,
@@ -122,7 +132,36 @@ export function discoveryCandidates(
spaceID: item.space_id,
url: driveSpaceURL(item.space_id),
detail: item.space_type === 'mySpace' ? '钉盘 · 我的文件' : '钉盘 · 组织空间',
alreadyRegistered: registered.has(item.name.trim()),
alreadyRegistered: item.registered === true,
}))
return [...knowledgeBases, ...driveSpaces]
}
export async function validateDingtalkSources(urls: string[], identity?: string): Promise<{ checked_at: string }> {
const response = await fetch('/api/v1/dingtalk-spaces/validate', {
method: 'POST',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
'X-Governance-Action': 'dingtalk-space',
...(identity ? { 'X-DWS-Identity': identity } : {}),
},
body: JSON.stringify({ urls }),
})
const body: unknown = await response.json().catch(() => null)
if (!response.ok) {
const code = isRecord(body) && typeof body.error === 'string' ? body.error : 'validation_failed'
const hint = isRecord(body) && typeof body.hint === 'string' ? body.hint : '来源校验失败,请重新读取范围'
throw new DingtalkRequestError(hint, code, response.status)
}
if (
!isRecord(body) ||
!isRecord(body.data) ||
body.data.valid !== true ||
body.data.count !== urls.length ||
typeof body.data.checked_at !== 'string' ||
!Number.isFinite(Date.parse(body.data.checked_at))
)
throw new Error('未取得有效的来源校验结果')
return { checked_at: body.data.checked_at }
}
+3 -1
View File
@@ -1,4 +1,5 @@
export interface DWSCredentials {
identity?: string
authenticated: boolean
refresh_token_valid: boolean
status_known: boolean
@@ -45,7 +46,7 @@ async function request(path: string, signal?: AbortSignal, method = 'GET') {
const reason = record(payload) && typeof payload.error === 'string' ? payload.error : ''
if (reason === 'dingtalk_discovery_unavailable')
throw new Error('DWS 未登录、授权已过期或缺少目录权限;请先登录授权,再检查连接。')
throw new Error(reason || `DWS 请求失败(HTTP ${response.status})`)
throw new Error((record(payload) ? text(payload.hint) : '') || reason || `DWS 请求失败(HTTP ${response.status})`)
}
if (!record(payload) || !record(payload.data)) throw new Error('DWS 返回的数据格式不正确,请重试')
return payload.data
@@ -55,6 +56,7 @@ export async function fetchDWSCredentials(signal?: AbortSignal): Promise<DWSCred
if (typeof data.authenticated !== 'boolean' || typeof data.refresh_token_valid !== 'boolean')
throw new Error('无法判断 DWS 登录状态')
return {
identity: text(data.identity),
authenticated: data.authenticated,
refresh_token_valid: data.refresh_token_valid,
status_known: data.status_known !== false,
+45
View File
@@ -0,0 +1,45 @@
import { describe, expect, it, vi, afterEach } from 'vitest'
import { readPPConnection, testPPConnection, savePPConnection } from './ppConnection'
afterEach(() => vi.unstubAllGlobals())
describe('PP connection transport', () => {
it('uses current draft on POST without saving or sending a placeholder password', async () => {
const fetcher = vi.fn(
async () =>
new Response(
JSON.stringify({ data: { state: 'valid', message: '目录可读', checked_at: '2026-11-17T10:00:00Z' } })
)
)
vi.stubGlobal('fetch', fetcher)
const result = await testPPConnection({ base_url: 'https://api.example.invalid', username: 'example', revision: 3 })
expect(result.state).toBe('valid')
const [path, init] = fetcher.mock.calls[0] as unknown as [string, RequestInit]
expect(path).toBe('/api/v1/pp-connection/test')
expect(init.method).toBe('POST')
expect(JSON.parse(String(init.body))).toEqual({
base_url: 'https://api.example.invalid',
username: 'example',
revision: 3,
})
})
it('preserves real response errors and reads metadata only', async () => {
vi.stubGlobal(
'fetch',
vi.fn(async () => new Response(JSON.stringify({ error: '原配置保留' }), { status: 422 }))
)
await expect(
savePPConnection({ base_url: 'https://api.example.invalid', username: 'example', revision: 1 })
).rejects.toThrow('原配置保留')
vi.stubGlobal(
'fetch',
vi.fn(
async () =>
new Response(
JSON.stringify({
data: { revision: 1, active: null, pending: null, observation: null, pending_observation: null },
})
)
)
)
expect((await readPPConnection()).revision).toBe(1)
})
})
+86
View File
@@ -0,0 +1,86 @@
export type PPObservation = { state: string; message: string; checked_at: string; previous?: PPObservation | null }
export type PPMetadata = { id: string; base_url: string; username: string; password_configured: boolean }
export type PPView = {
revision: number
active: PPMetadata | null
pending: PPMetadata | null
observation: PPObservation | null
pending_observation: PPObservation | null
}
export type PPDraft = { base_url: string; username: string; password?: string; revision: number; pending_id?: string }
const record = (v: unknown): v is Record<string, unknown> => typeof v === 'object' && v !== null
const states = new Set(['valid', 'auth_failed', 'permission_denied', 'incomplete', 'unavailable', 'invalid_response'])
async function request(path: string, method: string, draft?: PPDraft, signal?: AbortSignal) {
const response = await fetch(path, {
method,
signal,
cache: 'no-store',
headers: {
Accept: 'application/json',
...(draft ? { 'Content-Type': 'application/json', 'X-Governance-Action': 'pp-connection' } : {}),
},
...(draft ? { body: JSON.stringify(draft) } : {}),
})
const payload: unknown = await response.json().catch(() => null)
if (!response.ok)
throw new Error(
record(payload) && typeof payload.error === 'string' ? payload.error : `PP 请求失败(HTTP ${response.status})`
)
if (!record(payload) || !record(payload.data)) throw new Error('PP 返回的数据格式不正确')
return payload.data
}
function parseObservation(data: unknown): PPObservation | null {
if (data === null || data === undefined) return null
if (
!record(data) ||
!states.has(String(data.state)) ||
typeof data.message !== 'string' ||
typeof data.checked_at !== 'string' ||
!Number.isFinite(Date.parse(data.checked_at))
)
throw new Error('PP 检查状态无法识别')
return {
state: String(data.state),
message: data.message,
checked_at: data.checked_at,
...(record(data.previous) ? { previous: parseObservation({ ...data.previous, previous: null }) } : {}),
}
}
function parseMetadata(data: unknown): PPMetadata | null {
if (data === null) return null
if (
!record(data) ||
typeof data.id !== 'string' ||
typeof data.base_url !== 'string' ||
typeof data.username !== 'string' ||
typeof data.password_configured !== 'boolean'
)
throw new Error('PP 配置元数据不正确')
return {
id: data.id,
base_url: data.base_url,
username: data.username,
password_configured: data.password_configured,
}
}
function parseView(data: Record<string, unknown>): PPView {
if (!Number.isSafeInteger(data.revision) || Number(data.revision) < 0) throw new Error('PP 配置版本不正确')
return {
revision: Number(data.revision),
active: parseMetadata(data.active),
pending: parseMetadata(data.pending),
observation: parseObservation(data.observation),
pending_observation: parseObservation(data.pending_observation),
}
}
export async function readPPConnection(signal?: AbortSignal): Promise<PPView> {
return parseView(await request('/api/v1/pp-connection', 'GET', undefined, signal))
}
export async function testPPConnection(draft: PPDraft, signal?: AbortSignal): Promise<PPObservation> {
const result = parseObservation(await request('/api/v1/pp-connection/test', 'POST', draft, signal))
if (!result) throw new Error('未取得 PP 检查结果')
return result
}
export async function savePPConnection(draft: PPDraft, signal?: AbortSignal): Promise<PPView> {
return parseView(await request('/api/v1/pp-connection', 'PUT', draft, signal))
}
+4
View File
@@ -38,6 +38,10 @@ export async function fetchSourceAudit(offset = 0, signal?: AbortSignal): Promis
return data
}
export const sourceAuditActions: Record<string, string> = {
credential_configured: '更新认证配置',
credential_rejected: '认证配置校验失败',
credential_test: '测试连接',
dws_authorization: '钉钉授权',
created: '添加连接',
settings_updated: '修改配置',
paused: '暂停自动刷新',