feat: add knowledge parse cancellation with finalizing post-process state

Lets users stop an in-flight document parse to free up LLM / worker
resources without losing the chunks and index already written. The
core insight is that the previous parse_status=completed flipped as
soon as primary chunks landed, while the most expensive subtasks
(graph extract = N LLM calls per chunk, plus summary, question
generation) were still running in the background — so "completed"
wasn't actually terminal from a resource standpoint.

State machine

  pending -> processing -> finalizing -> completed
                              |
                              +-> cancelled (any of the three
                                            in-flight states)
                              +-> failed
                              +-> deleting

`finalizing` is the new post-process fan-out window. parse_status
only promotes to `completed` once pending_subtasks_count (a new
column tracking summary + question + per-chunk graph extract)
drains to zero via atomic FinalizeSubtask. Wiki ingest is
intentionally excluded from the counter — it's a KB-scoped
debounced batch and would otherwise pin parse_status in
`finalizing` for the wiki batch window.

Backend

- New ParseStatusFinalizing + pending_subtasks_count column with
  migration 000056.
- knowledgeRepository.SetFinalizing transitions processing -> finalizing
  conditionally so a racing cancel cannot be clobbered.
- knowledgeRepository.FinalizeSubtask atomically decrements the
  counter and self-promotes the row to completed when it hits zero.
- KnowledgePostProcess restructured to compute expected subtask
  count up front, flip to finalizing (or completed when no
  enrichment is enabled), and only then fan out subtasks. Subtask
  handlers (summary, question, graph extract) defer-decrement on
  terminal exit using the existing isFinalAsynqAttempt convention.
- New POST /api/v1/knowledge/{id}/cancel-parse handler accepting
  pending / processing / finalizing. Marks the row cancelled,
  zeroes the counter, best-effort dequeues asynq tasks via a new
  TaskInspector abstraction (asynq-mode walks pending/scheduled/
  retry queues; Lite-mode noop), and scrubs wiki ingest pending op.
- SpanTracker.AbortAttempt flat-sweeps every still-running span
  for the attempt via a new repo.CancelAllOpenSpans helper so the
  trace viewer's striped bars all flip to cancelled, even leaf
  generations whose parent stage already EndSpan'd (multimodal
  fan-out pattern). knowledge_post_process closes its postSpan
  via SkipSpan on the cancel/deleting entry guard so a worker
  that opens a span AFTER the cancel sweep doesn't leak it.
- Housekeeping and resetPendingTasks sweep finalizing rows
  identically to processing so a crash/restart can't strand them.
- DeleteKnowledge/DeleteKnowledgeList proactively dequeue
  downstream tasks via the same TaskInspector path.
- ChunkExtractService gets a cancel entry guard so the most
  expensive enrichment (graph extract) bails immediately when the
  parent knowledge is aborted.

Frontend

- New cancelKnowledgeParse API client + "Stop parsing" entry in
  both list view and card view more menus, gated on
  pending/processing/finalizing.
- Polling predicate refactored to a shared isParseInFlight helper
  that recognises `finalizing` (previously the doc list silently
  stopped polling once parse_status flipped from processing).
- Knowledge processing timeline: isPolling includes finalizing,
  new isHardTerminal short-circuits LIVE for cancelled/failed/
  completed so stranded child spans cannot pin LIVE on.
- DocumentListView.computeStatus distinguishes finalizing
  ("增强中") from completed and shows the previous "生成摘要中"
  copy when summary_status is still pending under finalizing.
  Added cancelled badge as well.
- i18n: statusFinalizing / statusCancelled / cancelParse* keys
  across zh-CN, en-US, ko-KR, ru-RU.

Docs / SDK

- docs/api/knowledge.md: documents the new finalizing state,
  cancel-parse semantics, and which statuses accept cancel.
- client (Go SDK): CancelKnowledgeParse with docstring listing
  the cancellable statuses.
This commit is contained in:
wizardchen
2026-05-28 18:26:56 +08:00
committed by lyingbug
parent c29d36238b
commit 44d6175559
32 changed files with 1312 additions and 86 deletions
+46 -2
View File
@@ -17,6 +17,7 @@
| DELETE | `/knowledge/:id` | 删除单条知识 |
| PUT | `/knowledge/manual/:id` | 更新手工 Markdown 知识 |
| POST | `/knowledge/:id/reparse` | 重新解析知识(异步) |
| POST | `/knowledge/:id/cancel-parse` | 取消正在进行的解析任务 |
| GET | `/knowledge/:id/download` | 下载原始文件(attachment) |
| GET | `/knowledge/:id/preview` | 内联预览文件(按扩展名设置 Content-Type) |
| PUT | `/knowledge/image/:id/:chunk_id` | 更新分块图像信息 |
@@ -28,8 +29,10 @@
> **公共说明**:
> - 路径中的 `:id`(知识库路径下)为**知识库 ID**,`/knowledge/:id` 中的 `:id` 为**知识 ID**。
> - 所有写操作(创建、更新、删除、迁移、重新解析)需要当前用户在知识库所属组织内具有 `editor` 或 `admin` 权限;清空知识库内容仅 KB **所有者**(admin 且租户匹配)可操作。
> - 关键状态字段:`parse_status` 取值 `pending` / `processing` / `completed` / `failed`;`enable_status` 取值 `enabled` / `disabled`。
> - 所有写操作(创建、更新、删除、迁移、重新解析、取消解析)需要当前用户在知识库所属组织内具有 `editor` 或 `admin` 权限;清空知识库内容仅 KB **所有者**(admin 且租户匹配)可操作。
> - 关键状态字段:`parse_status` 取值 `pending` / `processing` / `finalizing` / `completed` / `failed` / `cancelled`;`enable_status` 取值 `enabled` / `disabled`。
> - `processing` 指 DocReader / 分块 / 向量化阶段;`finalizing` 指主解析已完成、但摘要 / 问题生成 / 图谱抽取等异步增强子任务仍在执行;只有当全部子任务到达终态后才进入 `completed`。
> - `cancelled` 表示解析被用户主动取消,可通过 `reparse` 重新触发。`pending` / `processing` / `finalizing` 这三种状态都可通过 `cancel-parse` 终止。
## POST `/knowledge-bases/:id/knowledge/file` - 上传文件创建知识
@@ -562,6 +565,47 @@ curl --location --request POST 'http://localhost:8080/api/v1/knowledge/4c4e7c1a-
调用后 `parse_status` 会先变为 `pending`,再由后台 worker 转为 `processing` → `completed`/`failed`。
## POST `/knowledge/:id/cancel-parse` - 取消解析
中止正在进行的解析任务,常用于资源紧张时主动放弃当前文档的解析过程。
**行为**:
- 将 `parse_status` 置为 `cancelled`,`error_message` 写入「用户已取消解析」,并把 `pending_subtasks_count` 清零。
- 已写入数据库的分块 / 索引保留,可通过 `reparse` 接口在同一记录上重新触发解析。
- 后台异步会 best-effort 从队列中删除该知识对应的下游任务(多模态、问题生成、摘要、图谱抽取、Post-Process 等),并对正在执行的 worker 发出停止信号;worker 在下一个检查点退出。
- **可取消的状态**:`pending` / `processing` / `finalizing`。`finalizing` 表示主解析已完成、但摘要 / 问题生成 / 图谱抽取等增强子任务仍在执行;在该状态取消可以及时止血最昂贵的增强阶段(图谱抽取按 chunk 调 LLM,开销最大)。
- 已经完成 (`completed`) 或失败 (`failed`) 的知识不允许取消;正在删除 (`deleting`) 的知识不允许取消。
- 接口幂等:对已经 `cancelled` 的记录重复调用直接返回当前状态。
**请求**:
```curl
curl --location --request POST 'http://localhost:8080/api/v1/knowledge/4c4e7c1a-09cf-485b-a7b5-24b8cdc5acf5/cancel-parse' \
--header 'X-API-Key: sk-xxxxx'
```
**响应**:
```json
{
"success": true,
"message": "Knowledge parse cancelled",
"data": {
"id": "4c4e7c1a-09cf-485b-a7b5-24b8cdc5acf5",
"tenant_id": 1,
"knowledge_base_id": "kb-00000001",
"type": "file",
"title": "彗星.txt",
"parse_status": "cancelled",
"error_message": "用户已取消解析",
"enable_status": "disabled",
"created_at": "2025-08-12T11:52:36.168632+08:00",
"updated_at": "2025-08-12T13:05:00.000000+08:00"
}
}
```
## GET `/knowledge/:id/download` - 下载原始文件
以 `attachment` 方式下载知识对应的原始文件。