feat: add knowledge parse cancellation with finalizing post-process state

Lets users stop an in-flight document parse to free up LLM / worker
resources without losing the chunks and index already written. The
core insight is that the previous parse_status=completed flipped as
soon as primary chunks landed, while the most expensive subtasks
(graph extract = N LLM calls per chunk, plus summary, question
generation) were still running in the background — so "completed"
wasn't actually terminal from a resource standpoint.

State machine

  pending -> processing -> finalizing -> completed
                              |
                              +-> cancelled (any of the three
                                            in-flight states)
                              +-> failed
                              +-> deleting

`finalizing` is the new post-process fan-out window. parse_status
only promotes to `completed` once pending_subtasks_count (a new
column tracking summary + question + per-chunk graph extract)
drains to zero via atomic FinalizeSubtask. Wiki ingest is
intentionally excluded from the counter — it's a KB-scoped
debounced batch and would otherwise pin parse_status in
`finalizing` for the wiki batch window.

Backend

- New ParseStatusFinalizing + pending_subtasks_count column with
  migration 000056.
- knowledgeRepository.SetFinalizing transitions processing -> finalizing
  conditionally so a racing cancel cannot be clobbered.
- knowledgeRepository.FinalizeSubtask atomically decrements the
  counter and self-promotes the row to completed when it hits zero.
- KnowledgePostProcess restructured to compute expected subtask
  count up front, flip to finalizing (or completed when no
  enrichment is enabled), and only then fan out subtasks. Subtask
  handlers (summary, question, graph extract) defer-decrement on
  terminal exit using the existing isFinalAsynqAttempt convention.
- New POST /api/v1/knowledge/{id}/cancel-parse handler accepting
  pending / processing / finalizing. Marks the row cancelled,
  zeroes the counter, best-effort dequeues asynq tasks via a new
  TaskInspector abstraction (asynq-mode walks pending/scheduled/
  retry queues; Lite-mode noop), and scrubs wiki ingest pending op.
- SpanTracker.AbortAttempt flat-sweeps every still-running span
  for the attempt via a new repo.CancelAllOpenSpans helper so the
  trace viewer's striped bars all flip to cancelled, even leaf
  generations whose parent stage already EndSpan'd (multimodal
  fan-out pattern). knowledge_post_process closes its postSpan
  via SkipSpan on the cancel/deleting entry guard so a worker
  that opens a span AFTER the cancel sweep doesn't leak it.
- Housekeeping and resetPendingTasks sweep finalizing rows
  identically to processing so a crash/restart can't strand them.
- DeleteKnowledge/DeleteKnowledgeList proactively dequeue
  downstream tasks via the same TaskInspector path.
- ChunkExtractService gets a cancel entry guard so the most
  expensive enrichment (graph extract) bails immediately when the
  parent knowledge is aborted.

Frontend

- New cancelKnowledgeParse API client + "Stop parsing" entry in
  both list view and card view more menus, gated on
  pending/processing/finalizing.
- Polling predicate refactored to a shared isParseInFlight helper
  that recognises `finalizing` (previously the doc list silently
  stopped polling once parse_status flipped from processing).
- Knowledge processing timeline: isPolling includes finalizing,
  new isHardTerminal short-circuits LIVE for cancelled/failed/
  completed so stranded child spans cannot pin LIVE on.
- DocumentListView.computeStatus distinguishes finalizing
  ("增强中") from completed and shows the previous "生成摘要中"
  copy when summary_status is still pending under finalizing.
  Added cancelled badge as well.
- i18n: statusFinalizing / statusCancelled / cancelParse* keys
  across zh-CN, en-US, ko-KR, ru-RU.

Docs / SDK

- docs/api/knowledge.md: documents the new finalizing state,
  cancel-parse semantics, and which statuses accept cancel.
- client (Go SDK): CancelKnowledgeParse with docstring listing
  the cancellable statuses.
This commit is contained in:
wizardchen
2026-05-28 18:26:56 +08:00
committed by lyingbug
parent c29d36238b
commit 44d6175559
32 changed files with 1312 additions and 86 deletions
@@ -102,6 +102,14 @@ type SpanTracker interface {
// done; output/error are written verbatim.
FinalizeAttempt(ctx context.Context, knowledgeID string, attempt int, status string,
output types.JSONMap, errorCode, errorMessage string)
// AbortAttempt cascade-cancels every still-running descendant of the
// attempt's root span and then closes the root as cancelled. Used by
// the user-initiated cancel path so the trace viewer doesn't leave
// stranded subspans (multimodal images, postprocess subtasks)
// looking like they're still in flight forever after the user
// stopped the parse. Idempotent.
AbortAttempt(ctx context.Context, knowledgeID string, attempt int, errorCode, errorMessage, reason string)
}
type spanTracker struct {
@@ -724,6 +732,43 @@ func (t *spanTracker) FinalizeAttempt(ctx context.Context, knowledgeID string, a
t.touchKnowledgeHeartbeat(ctx, knowledgeID, types.SpanKindRoot)
}
// AbortAttempt is the user-cancel counterpart to FinalizeAttempt. It
// flips every still-running / still-pending span for this attempt to
// cancelled — regardless of tree position — and then closes the root.
//
// Why a flat sweep instead of CancelDescendants' BFS: fan-out stages
// (e.g. 多模态识别) call EndSpan on the stage as soon as they finish
// DISPATCHING their async per-image work, so by the time the user
// hits cancel the stage row is already status=done but its image[*]
// children are still status=running. A BFS that stops at terminal
// parents would orphan those leaves. The flat sweep doesn't care
// about the tree shape — anything not-yet-terminal gets flipped.
func (t *spanTracker) AbortAttempt(ctx context.Context, knowledgeID string, attempt int,
errorCode, errorMessage, reason string,
) {
if knowledgeID == "" || attempt <= 0 {
return
}
if reason == "" {
reason = "user cancelled"
}
if errorCode == "" {
errorCode = "USER_CANCELLED"
}
if n, err := t.repo.CancelAllOpenSpans(ctx, knowledgeID, attempt, errorCode, reason); err != nil {
logger.Warnf(ctx, "[SpanTracker] AbortAttempt sweep failed kid=%s attempt=%d: %v",
knowledgeID, attempt, err)
// Fall through to FinalizeAttempt anyway — closing the root
// is more important than perfectly closing every child.
} else if n > 0 {
logger.Infof(ctx,
"[SpanTracker] AbortAttempt swept %d open span(s) for kid=%s attempt=%d",
n, knowledgeID, attempt)
}
t.FinalizeAttempt(ctx, knowledgeID, attempt,
types.SpanStatusCancelled, nil, errorCode, errorMessage)
}
// noopSpanTracker collapses every method to a no-op for tests/lite.
type noopSpanTracker struct{}
@@ -743,3 +788,4 @@ func (noopSpanTracker) SkipSpan(_ context.Context, _ *Span, _ string)
func (noopSpanTracker) LookupStage(_ context.Context, _ string, _ int, _ string) *Span { return nil }
func (noopSpanTracker) FinalizeAttempt(_ context.Context, _ string, _ int, _ string, _ types.JSONMap, _, _ string) {
}
func (noopSpanTracker) AbortAttempt(_ context.Context, _ string, _ int, _, _, _ string) {}