7b2ef8bd8e5f7ee2967db3d2d45d45f125012f44
1914 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7b2ef8bd8e | fix: show agent id in editor | ||
|
|
43565c5d1b | fix: add model display name | ||
|
|
4e58dd42cc |
feat(knowledge-base): integrate trace availability checks and enhance UI interactions
This commit introduces a new utility function, `knowledgeSpansPayloadHasTrace`, to determine if the knowledge spans data contains a valid trace. Key changes include: 1. Updated the `KnowledgeBase` component to utilize the new trace availability checks, improving the logic for displaying trace-related UI elements. 2. Enhanced the `fetchSpans` function in the `knowledge-processing-timeline` component to emit trace availability based on the new utility. 3. Implemented caching for trace availability to optimize performance and reduce unnecessary API calls. These changes aim to improve user experience by providing accurate trace information and enhancing the overall responsiveness of the UI. |
||
|
|
a0547729b2 |
feat(trace-drawer): implement resizable trace drawer and enhance UI interactions
This commit introduces a resizable trace drawer in the `doc-content` component, allowing users to adjust the width for better visibility. Key changes include: 1. Added functionality to save and load the drawer width from local storage. 2. Implemented mouse events for resizing the drawer, enhancing user interaction. 3. Updated the UI to reflect the new drawer width dynamically. 4. Enhanced the trace entry button for improved accessibility and clarity. These changes aim to improve user experience by providing a more flexible and user-friendly interface for trace inspection. |
||
|
|
d12273255d |
refactor(timeline): improve polling logic and documentation for gracePoll
This commit enhances the polling mechanism in the KnowledgeProcessingTimeline component by introducing a new function, `shouldPollNow`, which clarifies the conditions under which polling should occur based on the `gracePoll` prop. The documentation for `gracePoll` has been expanded to provide clearer semantics for both user-visible and background mounts. Additionally, minor formatting adjustments were made to improve code readability. These changes aim to streamline the polling behavior and enhance the overall user experience. |
||
|
|
797f55c567 |
feat(timeline): enhance knowledge processing spans and UI elements
This commit introduces several improvements to the knowledge processing timeline and related components. Key changes include: 1. Added a `gracePoll` prop to the `KnowledgeProcessingTimeline` component to manage polling behavior more effectively. 2. Enhanced the UI by displaying the document title in the drawer, improving user visibility of the current document context. 3. Implemented new CSS classes for better styling of the drawer title bar, ensuring a more polished appearance. 4. Updated the backend to support the new `WikiSpan` tracking, allowing for detailed monitoring of document processing stages. These changes aim to improve user experience and provide better insights into the document processing workflow. |
||
|
|
e697eee07f |
feat(trace): enable direct access to trace drawer from card menu
This commit introduces a new method to open the trace drawer directly from the card menu, enhancing user experience by allowing immediate access to trace details without navigating through the document detail drawer. The implementation includes updates to the `handleViewTrace` function to ensure the correct knowledge ID and parse status are set before opening the trace drawer. Additionally, minor adjustments were made to the UI for better consistency and clarity. |
||
|
|
df3b72c0fe |
feat(migrations): add knowledge_processing_spans table and rollback script
This commit introduces the `knowledge_processing_spans` table to track the progress of document parsing stages, enabling better visibility and error handling in the frontend. The schema includes fields for span details, status, and timestamps, along with necessary indexes for efficient querying. A rollback script is also provided to drop the table and its associated indexes if needed. |
||
|
|
2a4d6f9019 |
fix: preserve span input/metadata, auto-expand subspans, force-arm poll
Three fixes in response to user feedback: 1. Span input disappearing on End/Fail The Upsert's DoUpdates always listed input/output/metadata, so calls that only set output (EndSpan) or only set error_* (FailSpan) wrote NULL into input/metadata, clobbering whatever Begin had recorded. Build the column list dynamically: skip input/output/metadata when the incoming row's value is nil. nil now means "preserve existing" (matches user's intuition "Begin recorded it, End shouldn't erase it"). 2. Subspans not auto-expanded Stages with children (multimodal.image[*], postprocess.summary, postprocess.question, postprocess.graph.chunk[*]) required a click on the ▸ caret to surface — easy to miss. On the FIRST successful fetch per (knowledgeId × attempt), auto-expand any stage that has children. Subsequent polls honor whatever the user has collapsed, so manual collapse mid-parse stays collapsed. 3. Auto-poll still not firing Force-arm the polling interval in onMounted regardless of state. The per-tick callback decides whether to actually fetch based on current parse_status — so the loop can never get stranded waiting on a status transition that already happened. Added a console.debug when the interval arms, so we can verify from DevTools console that polling is actually running. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
baa4e75d4a |
fix(timeline): switch to setInterval polling + watchdog watcher
User report: opened a parsing-in-progress doc, /spans was hit ONCE on mount and then never again — but parse_status said 'processing' and the LIVE badge was lit. The setTimeout-based recursive scheduler had a fragile property: any unexpected throw or skipped tick between the finally block and the schedule check could silently strand the loop with no way to re-arm. Refactor to a self-healing pattern: - setInterval polling: a single 2s tick checks "should we be polling? and is no fetch in flight?" — if both, fire fetchSpans. Decision is re-evaluated every tick from current reactive state, so flipping status from 'processing' to 'completed' between ticks naturally stops further fetches. - ensurePolling() helper is idempotent — calling it from fetchSpans's finally, from a parse_status watcher, and from onMounted all just arm the same single interval if not already running. - watch(data.value.parse_status) acts as a watchdog: any time the status flips into 'pending'/'processing', re-arm. Any time it flips out, tear down. This is the belt-and-suspenders that ensures the loop can't get stranded even if a future regression re-introduces a missed-schedule path. - fetchInFlight guard prevents overlapping fetches if a slow backend takes longer than POLL_INTERVAL_MS to respond. - Console warns on fetch failure — silent rejections is exactly what hid this bug. With this in place we can verify the loop's health from DevTools console. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
babf9139ec |
fix(timeline): hide "更新于" once trace is terminal
User report: opening an already-completed doc still shows "更新于 X 秒前" ticking up forever, which is misleading — the data IS final, no freshness concern exists. Re-gate the caption on isLive (was always on) so it only appears while pending/processing. Failures pill is still gated on the same isLive condition, which is the only state where polling-failure feedback is meaningful. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
9333869439 |
fix(timeline): track fetch attempts and surface silent polling failures
User reported: spinner spinning + "更新于 X 秒前" continuously growing without bound while parsing is in progress. The auto-poll loop IS firing every 2s, but lastFetchedAt only updated on success — so when the API returns success=false or errors, the caption silently aged while the visible state suggested live polling. - Move lastFetchedAt update into the finally block so it tracks attempts (success or failure). The caption now bounces back to "刚刚 / 1s 前 / 刚刚" every poll cycle, matching the spinner's visual cadence. - Track failedAttempts and lastFetchOk: when the last attempt failed the timestamp shows in muted italic; when 2+ consecutive attempts fail a small "⚠ 刷新失败" pill appears next to it with a tooltip explaining the situation. - New i18n keys fetchFailed / fetchFailedShort in zh-CN/en-US/ko-KR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
c4fba77a96 |
refactor(timeline): move Trace entry to header pill, simplify refresh UX
The Trace trigger was a giant card-style row with a green left strip
sitting above 文件名/摘要 — visually competed with document content
and felt out of place vs. the rest of the app's quiet density.
Trace entry:
- Replaces the big card with a compact rounded pill in the drawer
header, next to the file-type t-tag.
- Reads as a quiet secondary action — status dot + label + duration
+ chevron — at the same visual weight as the file-type tag.
- Status dot still pulses while parsing is in progress so the LIVE
signal survives the size cut.
- Hidden mount preserved so the pill's status/duration stays live
even before the user opens the secondary drawer.
Refresh UX:
- "更新于 X 秒前" caption now always visible (was hidden once polling
stopped). After parsing completes it tells the user how stale the
data is when they reopen the drawer hours later — the original
reason it was hidden ("ticks up forever") is exactly the staleness
signal one needs.
- LIVE badge tooltip simplified from "解析中,每 2 秒自动刷新({n}s 后)"
(cryptic countdown) to "解析进行中 — 每 2 秒自动刷新".
- Manual refresh icon now slow-spins in brand color while auto-poll
is active — users can tell at a glance "auto-refresh is on" without
needing the LIVE badge to interpret. Manual click still triggers
the fast spin.
- Drops the unused nextRefreshIn computed.
i18n: liveTooltip rewritten in zh-CN/en-US/ko-KR; new keys
autoRefreshOn / autoRefreshOff for the button title states.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
|
5dc0a49a9b |
feat(timeline): enrich per-image multimodal subspan output
The per-image multimodal subspan only captured image_url / enable_ocr / enable_caption on input and chunk_id on output, so the trace viewer could not answer "what did THIS image actually produce?" without joining back to the chunks table. Adds to the per-image span output: - vlm_model_id (or "legacy_inline" for inline-config KBs) - image_bytes (read size) - ocr_prompt: "default" | "scanned_pdf" - ocr_chars + ocr_preview (sanitized text, capped at 200 runes) - caption_chars + caption_preview - chunks_created (count of OCR/caption child chunks) - indexed (true after BatchIndex completes) - per-step error fields (read_error / ocr_error / caption_error / skipped reason) when something fails Also adds parent_chunk_id to the span input so the trace links back to the text chunk this image hangs off — useful when a doc has hundreds of inline images and you need to know WHERE in the text this one came from. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
f99bcb79e7 |
feat(timeline): scale axis to async tail, enrich span payloads, polish UI
The post-process stage closes in ~9ms (just enqueue work) but its async subspans (postprocess.summary, postprocess.question, postprocess.graph) keep producing rows for tens of seconds AFTER the root finalizes. The old timeline used trace.duration_ms as the time axis maximum, which clipped those subspan bars past the right edge. Timeline: - totalMs now always takes max(trace.duration_ms, observed-tail), so the axis stretches to fit the latest descendant end regardless of parse_status. - Render a faint dashed wrapping outline behind a parent span when its descendants extend past its own finished_at, so the postprocess stage row visibly spans the full window without overloading the 9ms self-time bar. - Tree expand/collapse caret bumped from 10 to 14px in a 16x16 hit area; copy icons in detail panel bumped from 11/14 to 14/18px; .kp-kv-copy button grown from 18 to 22px. - Short input/output payloads (<= 8 entries / <= 600 bytes JSON) auto-expand inline so users see the actual data without an extra click; longer payloads keep the click-to-expand summary. Span payloads (subspans only - root keeps the canonical identity, no duplicate knowledge_id/kb_id/tenant_id on every child): - extract.go: graph subspan output gains chunk_chars, chunk_preview, sample_nodes, sample_relations. - summary subspan output gains model_id, summary_preview. - question subspan output gains model_id and a sample_question captured from the first non-empty LLM response. i18n: new key knowledgeStages.detail.includingChildren for the wrapping-bar tooltip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
bd029b6d19 |
feat(knowledge): instrument postprocess subspans + polish timeline UI
Backend: summary, question, and graph-extract async tasks now record real processing time as subspans hanging off the (closed) postprocess stage, so the trace viewer no longer caps the postprocess row at the ~10 ms enqueue duration. Carries Attempt through SummaryGeneration / QuestionGeneration / ExtractChunk payloads so cross-process workers can resolve the right parent attempt. Frontend: drawer now uses attach="body" so the secondary 820px detail drawer escapes the 654px container of the main drawer; timeline timestamps include date prefix (MM-DD, or YYYY-MM-DD across years); "updated X ago" caption only shows during live polling. Tests: 4 new cases covering postprocess subspan attaching under a closed parent, missing-parent fallthrough, and Attempt JSON round-trip on the three task payloads. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
5ffc755e76 |
fix(knowledge): infer synthesized stage status from parse_status
Pre-tracker historical knowledge has zero rows in knowledge_processing_spans but parse_status correctly reads "completed" or "failed". The /spans handler was synthesizing five "pending" placeholders unconditionally, so legacy completed documents rendered as if they were stuck waiting in the queue forever. buildSpanTree now takes parse_status and chooses the placeholder status accordingly: - ParseStatusCompleted -> done - ParseStatusFailed -> failed - everything else -> pending (existing behaviour) Real rows always take precedence; this only changes what we put in the gaps. So healthy in-flight parses (parse_status=processing, some real rows, some still pending) keep showing pending placeholders exactly as before — the synthesized "completed" inference only fires when the parse already hit terminal state. Adds TestBuildSpanTree_LegacyCompletedRendersAsDone covering both the completed-legacy and failed-legacy branches. |
||
|
|
01d00c2a36 |
refactor(frontend): rebuild timeline UI with side drawer + attempt tabs
The previous Langfuse-style waterfall had concrete UX problems visible
in the field: the status text column wrapped Chinese characters across
two lines, the root span name truncated mid-identifier, the inline
detail expansion shifted sibling rows out of order, and the attempt
selector was a plain HTML <select>.
This commit rebuilds the visual layer around three corrections:
- Status column removed entirely. Status is conveyed by the 8 px dot
and the bar color; the redundant Chinese label that was wrapping
vertically is gone. Header subtitle now reads "总耗时 6.2s · ✓ 已完成"
via knowledgeStages.total + knowledgeStages.status.<value>.
- Inline row-detail replaced by a right-side drawer absolutely
positioned inside the timeline shell (overflow: hidden). Clicking a
row no longer reorders later rows beneath it. The drawer slides via
CSS transform, ESC and × close it, clicking the selected row toggles.
- Attempt <select> dropdown replaced by a horizontal pill tab strip
with a per-attempt status glyph (✓ / ✗ / ●). attemptStatuses Map
populates lazily — one /spans?attempt=n fetch per missing tab on
mount — so multi-attempt history doesn't block the initial render.
Drawer body shows the rich span data the backend just started
emitting: timing (started / finished / duration), an error block when
failed/cancelled, and Input / Output / Metadata as a 2-column
humanized key/value table. Numbers use toLocaleString, booleans render
as ✓/✗, arrays/objects collapse to a summary with a per-key "Show
JSON" toggle. Existing copySpan retains the raw JSON export path.
Root span name is now localized via knowledgeStages.root ("Knowledge
processing" / "知识处理") instead of leaking the raw identifier
"knowledge_processing" — which previously truncated as
"knowledge_process…".
i18n: added knowledgeStages.total and knowledgeStages.detail.{started,
finished, duration, input, output, metadata, error, showJson, hideJson}
to en-US, zh-CN, ko-KR, ru-RU. Compact mode (used by the card hover
popover) is untouched.
|
||
|
|
4db6e69b4e |
fix(knowledge): close root span on terminal state, enrich stage metadata
The root span created by OpenAttempt was never closed: PostProcess only
ended the postprocess stage, so the root row stayed at status=running
forever even after parse_status flipped to completed/failed. The
timeline rendered "进行中" indefinitely on the root, defeating the
whole "is the document done" question the timeline is meant to answer.
- SpanTracker.FinalizeAttempt(kid, attempt, status, output, code, msg):
closes the root row idempotently. Re-closing a terminal root no-ops
so success / cascade-fail / dead-letter paths can fire without
coordination.
- PostProcess.Handle calls FinalizeAttempt(done) after EndSpan(postprocess)
on the success path. Async downstream work (summary/question/wiki/
graph) still records its own spans; their completion extends the
trace's wall-clock end-time but doesn't reopen the root.
- FailSpan auto-closes the root when a MAIN pipeline stage fails
(docreader / chunking / embedding / multimodal / postprocess).
Cascade-cancelled siblings stay closed-with-the-cascade as before.
- Dead-letter callback (router/task.go) accepts the SpanTracker via
DI and calls FinalizeAttempt(failed, TASK_TIMEOUT) when a
document-related task exhausts retries. The probe payload now
extracts the Attempt field that Document/Manual/PostProcess
payloads already thread through.
Stage spans were also being recorded with nil input/output, leaving
the new detail panel with timestamps only. Each Begin/End site now
emits useful work metrics:
- DocReader input: file_name, file_type, is_url, url
output: text_length, images_found, is_audio, pages
- Chunking input: chunks_planned
output: chunks_written, total_text_chars
- Embedding input: chunks_to_embed, model_id, dim
output: vectors_written, storage_bytes
- Multimodal input: image_count, enable_ocr, enable_caption
- PostProcess output: chunks_total, enqueued_summary, _question, _wiki, _graph
i18n: add knowledgeStages.root ("Knowledge processing") so the UI
can render a localized name instead of the raw span identifier.
|
||
|
|
c5722234ef |
refactor(frontend): redesign parsing timeline as Langfuse-style waterfall
The five-circle pipeline conveyed only "which stage we're on" — operators asked for the actual span tree (root → stages → batches/images), the time each span took relative to the whole run, and the recorded JSON in/out. This commit rebuilds the visual layer to match Langfuse's trace view without touching the data layer (polling, attempt selector, retry, copy-span and watchers all unchanged). - Three-column grid (name | status+duration | bar): each span is one row, indented by depth. Status dot 8px, name monospace, duration right-aligned. Bars are absolutely-positioned inside a shared timeline lane, left/width computed from each span's started_at / duration_ms relative to trace.started_at and the run's end (or "now" while running). - Time ruler with five ticks (0% → 100%) above the bars; total duration label on the right. - Sub-spans (embedding.batch[i], multimodal.image[i]) collapsed under each stage by default behind a chevron toggle. Click any row to expand an inline detail panel showing ISO timestamps, status, error_code/message when failed, and input/output/metadata JSON blocks plus a "Copy span JSON" action. - Removed the parse_status gate in doc-content: the timeline is now always rendered when a knowledge detail is open, so completed documents also expose their recorded pipeline for audit. The KnowledgeBase hover popover stays compact-mode-only and still gates on parse_status (popover height matters there). - Running bars grow visibly via a 1s ticker (cleared on unmount). - Last-error block restricted to parse_status === failed. - Added knowledgeStages.totalDuration to en-US / zh-CN / ko-KR / ru-RU. |
||
|
|
1d8ac301c2 |
feat(frontend): visualize document parsing pipeline with span timeline
Renders a five-segment timeline (DocReader → Chunking → Embedding →
Multimodal → PostProcess) above each document so users can see exactly
which stage a document is in and which one failed, instead of a flat
"Parsing..." spinner.
- New KnowledgeProcessingTimeline component (full + compact mode):
full mode lives at the top of doc-content for any document not yet
in `completed`; compact mode replaces the parsing/failed lines in
the card hover popover with a five-dot mini-progress.
- Polls GET /api/v1/knowledge/:id/spans every 2s while
parse_status in {pending, processing}; stops on terminal status or
unmount. Hover popover instances opt out of polling to avoid
amplifying requests on grid hover.
- Failed / cancelled steps expand into an error card with the raw
error_message, an error_code -> localized title + suggestion mapping
for the twelve canonical codes (DOCREADER_TIMEOUT, EMBEDDING_RATE_LIMIT,
...), and a "Copy details" button that pastes the failed span JSON
for support tickets.
- Attempt selector renders only when latest_attempt > 1 so reparse
history is browsable; the latest attempt is selected by default.
- Last-error block under the timeline includes a Retry button that
calls reparseKnowledge() and immediately re-fetches.
- i18n strings added under knowledgeStages for en-US, zh-CN, ko-KR,
ru-RU. ja-JP locale file does not exist in this repository.
|
||
|
|
06f94a4811 |
feat(migrations): add knowledge_processing_spans table for tracking document parsing progress
- Introduced a new migration (000054) to create the knowledge_processing_spans table, which captures detailed progress of document parsing stages. - The table includes fields for span hierarchy, status tracking, and metadata to enhance visibility into the parsing pipeline. - Added indexes to optimize queries related to knowledge_id, attempt, and span relationships. - Implemented rollback migration to safely drop the table and associated indexes if needed. |
||
|
|
c82b098f44 |
fix(knowledge): address span-tree review feedback
- Collapse migrations 000052 (flat stages) + 000053 (span tree) into a single 000052_knowledge_processing_spans migration; the flat stages table never escaped this branch and the create-then-drop sequence had no value. - BeginStage: detect an existing (kid, attempt, stage) row before inserting and reuse its span_id with reset state, so re-entry from asynq retries or adjacent code paths no longer produces duplicate timeline segments. - FailSpan: when sibling-stage cascade flips a dependent stage to cancelled, also CancelDescendants on its subtree so already-running subspans (embedding.batch[i] etc.) cannot remain as orphan running rows under a cancelled parent. - Dead-letter callback: replace two sequential UpdateKnowledgeColumn writes with a single UpdateKnowledgeColumns map update so we cannot end up with parse_status=failed and stale error_message (or vice versa) when one of the two writes fails. - touchKnowledgeHeartbeat: skip subspan/generation transitions; only root and stage transitions poke knowledge.updated_at. Spans-table MAX(updated_at) already covers subspan progress for housekeeping, so this avoids 2*N+ UPDATE bursts on the same hot row when a multimodal stage fans out to many images. - Add regression tests for the BeginStage idempotency contract and the cascade-into-subspans behaviour. |
||
|
|
414749681f |
test(knowledge): add coverage and protect housekeeping from false-killing long stages
Addresses three review concerns from the prior PR:
1. No tests existed for any of the stuck-parsing fixes (PR① / PR②.5).
This commit adds coverage for the four most regression-prone
surfaces: span repo upsert/cascade/attempt isolation, span
tracker cascade-cancel and cross-process LookupStage,
housekeeping false-kill protection, and handler tree assembly.
2. Housekeeping was using only knowledge.updated_at as its staleness
signal, but knowledge.updated_at advances only at parse_status
transitions — a long DocReader call (or large embedding batch)
can run for an hour with no updated_at change, so a tight
DocumentProcessTimeout setting would falsely flip an actively
running parse to "failed".
The sweep now does a two-stage check: candidates by knowledge
updated_at, then filtered by MAX(spans.updated_at). Every
SpanTracker Begin/End/Fail/Skip now also pokes
knowledge.updated_at as a side-channel heartbeat, so the
filter sees recent activity even when no parse_status
transition has fired.
3. parseHeartbeatTime accepts the timestamp formats both Postgres
and SQLite emit for an aggregated MAX() column (the SQLite
driver doesn't auto-cast aggregates to time.Time the way
Postgres does), so the same code path works in Lite mode.
The new TestHousekeeping_NoFalseKill_ActiveSpan is the regression
test for the user-flagged scenario: a 3-hour-stale knowledge.updated_at
combined with a 2-minute-fresh span row must NOT be killed.
|
||
|
|
c9941a6688 |
refactor(knowledge): replace flat stage table with langfuse-style span tree
Addresses review feedback that the PR② design had four shortcomings:
1. The pipeline is a DAG, not a sequence — Embedding and Multimodal
are independent of each other, both downstream of Chunking, both
upstream of PostProcess. The flat (knowledge_id, stage) table
couldn't represent that, so a Chunking failure left dependents
stranded as "pending" forever instead of being marked as
impossible-to-run.
2. No history across attempts. A reparse erased the previous run's
status before the new run started, leaving operators with no way
to investigate "why did this fail twice?".
3. Stages had only status + duration. Operators want to know how big
the work was — pages parsed, chunks created, tokens embedded, VLM
calls made — to distinguish "slow because the file is huge" from
"slow because the docreader is wedged".
4. Multimodal fans out N image tasks; Embedding fans out M batches;
PostProcess fans out into Summary/Question/Wiki/Graph. Each unit
is interesting on its own (Langfuse already captures this for
LLM calls). The flat model couldn't express it.
The redesign mirrors Langfuse's trace/span/generation hierarchy:
* Migration 000053 supersedes 000052: knowledge_processing_spans
table with (knowledge_id, attempt, span_id) primary key, plus
parent_span_id, kind ∈ {root, stage, subspan, generation},
status ∈ {pending,running,done,failed,skipped,cancelled}, and
JSONB input/output/metadata fields.
* SpanTracker (replacing StageTracker) exposes OpenAttempt /
BeginStage / BeginSubSpan / EndSpan / FailSpan / SkipSpan /
LookupStage. Cross-process workers (image_multimodal) get the
parent's attempt + span via payload + LookupStage so subspans
attach correctly.
* StageDependencies declares the DAG; FailSpan now cascades —
descendants of the failed span and dependent stages are flipped
to "cancelled" with a UPSTREAM_FAILED code. The UI sees a clear
blast radius instead of orphan spinners.
* Reparse now calls OpenAttempt up front so the timeline reflects
"new attempt, all pending" instead of letting the previous run's
status linger until the worker picks up the task.
* Image_multimodal records each image as a generation subspan with
its own success/failure on the parent attempt's multimodal stage.
The finalize-on-last-attempt counter logic is preserved unchanged.
* GET /api/v1/knowledge/:id/spans (also kept /stages alias) returns
a tree shape with synthesized pending placeholders so the
frontend always renders five timeline segments. ?attempt=N
enables history navigation.
|
||
|
|
04f56f9cda |
feat(knowledge): track per-stage parsing progress with /stages API
Adds a five-segment progress model for the document parsing pipeline so the UI (PR③) can render a timeline showing where each document is (DocReader → Chunking → Embedding → Multimodal → PostProcess) and which stage failed with what error code. - New table `knowledge_processing_stages` (migration 000052) with one row per (knowledge_id, stage). UPSERT on Begin/Done/Fail bumps an attempt counter so re-parses don't lose history. - StageTracker service exposes Begin/Done/Fail/Skip; all calls are best-effort and never break the pipeline if persistence fails. - Stable error codes (DOCREADER_TIMEOUT / EMBEDDING_RATE_LIMIT / VECTORSTORE_WRITE_FAILED / ...) the UI can map to localized remediation hints. - Tracker call sites added at the four meaningful failure points: convert (DocReader), CreateChunks (Chunking), BatchIndex (Embedding), enqueueImageMultimodalTasks (Multimodal start), KnowledgePostProcess.Handle (Multimodal close + PostProcess). - New endpoint `GET /api/v1/knowledge/:id/stages` returns the five canonical stages — missing rows are synthesized as "pending" so the timeline always renders five segments. Includes current_stage and last_error block. |
||
|
|
3ae3ea97c5 |
fix(knowledge): prevent documents from getting stuck in "processing"
Several failure modes left Knowledge.parse_status pinned at "processing" forever, with no signal to users beyond a permanent spinner. This commit addresses the root causes and adds a safety net. - Asynq worker pool: explicit Concurrency (default 16, env-tunable via WEKNORA_ASYNQ_CONCURRENCY) so batch uploads don't queue behind a CPU-count-sized worker pool. Redis op timeouts raised to 500ms/1000ms (WEKNORA_REDIS_OP_TIMEOUT_MS) to absorb bursty multimodal counter ops. - DocReader RPC: cap each call with WEKNORA_DOCREADER_CALL_TIMEOUT (default 30m). Without this, a hung docreader pinned a worker for the full DocumentProcessTimeout window. - ImageMultimodal: finalize-on-last-attempt semantics. A permanently failing single image no longer strands the parent — the asynq retry is allowed to run, but on the final attempt we count the image regardless of outcome. Redis DECR errors fall back to enqueuing the post-process task instead of returning silently. - Dead-letter callback: when DocumentProcess / KnowledgePostProcess / ManualProcess exhausts retries, immediately mark the corresponding Knowledge as failed with the last error. This surfaces the failure in the UI without waiting for the housekeeping sweep. - HousekeepingService: 5-minute cron that flips knowledge rows stuck in "processing" past DocumentProcessTimeout + 10m to failed, plus summary rows stuck > 1h. Catches anything the other safety nets miss (worker SIGKILL mid-handler, etc.). Disable with WEKNORA_HOUSEKEEPING_ENABLED=false. - Distributed startup recovery: previously the post-restart sweep was skipped whenever REDIS_ADDR was set, even though Asynq does not reschedule the task that was actively running on the dead instance. Now the sweep runs in distributed mode too, but only against rows older than 30 minutes to avoid racing peer instances. |
||
|
|
ae4ec0cf06 |
fix(im): make presigned URL flow diagnosable end-to-end
When IM image rendering breaks, operators previously had no log line to correlate against the IM platform's fetch attempt. Add observability hooks on both ends and unblock HEAD probes so common IM previews work at all. - log rewriteStorageURLs success/failure/no-op with the full signed URL (operators can copy it from logs and verify public reachability) - log presigned handler 4xx with client_ip + UA + tenant_id + file_path so failures correlate against IM platform fetch logs; use the request context so trace IDs are preserved - accept HEAD on /api/v1/files/presigned: IM platforms (Feishu, Slack etc.) probe with HEAD before GET when rendering image previews, and a 401 there is enough to break the inline image even when the GET would have succeeded - add Admin-only GET /api/v1/files/presigned-preview that returns the exact URL an IM channel would embed for the calling tenant, for self-service verification without sending a real IM message - clarify APP_EXTERNAL_URL and MINIO_ENDPOINT public-reachability rules in .env.example; misconfigured endpoints are the most common cause of "image broken in IM" reports |
||
|
|
f5820c13ac |
feat(retriever): implement OpenSearch driver read/write paths (PR 2b of 3) (#1482)
Second half of the gated OpenSearch k-NN driver (#1440). Replaces the PR 2a stubs for Save / BatchSave / DeleteBy* / Retrieve with their real implementations, drops the stubs from `stubs.go`, and adds the 27 corresponding test cases that were intentionally held back. The driver remains gated dead code (no registry / factory / env path mentions OpenSearch — PR 3 ships the activation switch); the behaviour added here only fires once PR 3 lands. Depends on PR 2a of #1440 (driver skeleton). What lands in PR 2b ------------------- Three new production files (+762 LoC): - `query.go` (+154) — typed retrieveFilters struct as the only entry to query construction (closes the map[string]any injection surface; hostile chunk_id stays JSON-escaped). buildKNNQuery (pure k-NN, applies min_score when Threshold > 0) and buildKeywordQuery (BM25); native hybrid intentionally omitted (out of scope for Phase 3 — service-layer RRF stays in charge). Filter struct carries ExcludeKnowledgeIDs and an opt-in IncludeDisabled for admin-mode use cases. - `retrieve.go` (+204) — Retrieve resolves the per-dim alias from (1) params.AdditionalParams["dim"], (2) len(params.Embedding), or (3) the cross-dim wildcard <base>_* for BM25-only callers. Always returns exactly one *RetrieveResult (the Phase 2 fan-out contract). Hit parsing caps response body at 16 MB and warns if _id != _source.chunk_id. Score passes through raw in [0, 1] — the k-NN plugin's SpaceType.COSINESIMIL.scoreTranslation has already mapped (1 + cos) / 2 before the client sees it. - `crud.go` (+405) — Save / BatchSave with _id = chunk_id for idempotency, dim-aware pre-marshal NDJSON size cap (rejects with ErrBatchTooLarge before allocating a 50 MB body for a too-big batch), and per-item error inspection that does NOT leak cluster- side Reason strings into wrapped public errors. additionalParams contract verified against elasticsearch/structs.go ToDBVectorEmbedding + keywords_vector_hybrid_indexer.go: embedding is map[string][]float32 keyed by SourceID, chunk_enabled is map[string]bool keyed by ChunkID. Missing embedding routes Save / BatchSave to the dim-less <base>_keywords index. Sync DeleteBy* paths capped at 1000 IDs per call with ErrBatchTooLarge. toDoc writes the is_recommended boolean from IndexInfo. The matching mapping entry was added in PR 2a alongside is_enabled — the pair stays in sync between the vector and the keyword-only index so the field round-trips through CopyIndices (PR 3) and any future FAQ-priority filtering. Stubs removed (`stubs.go`, -86 LoC): Save / BatchSave / Retrieve / DeleteByChunkIDList / DeleteBySourceIDList / DeleteByKnowledgeIDList no longer return ErrFeatureNotEnabled — they delegate to the production implementations above. The remaining stubs (CopyIndices, BatchUpdateChunkEnabledStatus, BatchUpdateChunkTagID, swapToVersion, EstimateStorageSize lower-bound) stay until PR 3. Tests (+490 net to repository_test.go, 28 new cases): - retrieveFilters: JSON injection guard, ExcludeKnowledgeIDs, IncludeDisabled skip semantics, default is_enabled=true pin. - buildKNNQuery: min_score applied when Threshold > 0; omitted at threshold zero. - buildKeywordQuery: content match appended after typed filters. - toDoc: embedding field omitted when empty, present when set, source_type serialised as integer, **is_recommended round-trips both true and false**. - lookupEmbedding / lookupChunkEnabled: SourceID / ChunkID key contract; nil-params and wrong-type degrade paths. - extractBatchEmbeddings: mixed-dim rejection; all-empty embeddings yield dim=0. - estimateBulkBodyBytes: size estimator boundary. - effectiveTopK: clamping when caller leaves TopK at the magic-10 default. - resolveDim: AdditionalParams takes precedence; Embedding fallback; zero-zero triggers cross-dim multi-index search. - inspectBulkResponse: all-succeeded fast path; total-count + leak- guard for cluster-side Reason strings. - BatchSave cap: 1001 infos returns ErrBatchTooLarge (pre-marshal); empty list is a no-op. - DeleteByChunkIDList cap: 1001 IDs returns ErrBatchTooLarge; empty list is a no-op. SDK quirk addressed in PR 2b ---------------------------- DocumentDeleteByQueryParams.Refresh is *bool, so the wire-level "wait_for" value documented in the OpenSearch REST API is not directly expressible. deleteByTerms uses Refresh:&true which forces immediate segment flush (read-your-writes guaranteed but more expensive than wait_for). A follow-up PR can drop to *opensearchapi.Client.Transport.Perform for wait_for if telemetry shows the cost is material. Backward compatibility ---------------------- - Additive within the opensearch package only. No file outside the package is touched. - Driver still unreachable: PR 3 ships the activation switch. - The PR 1 normalizer case for OpenSearch remains unreachable here (no driver instance produces a result yet). Test plan --------- - [x] go build ./... clean - [x] go vet ./... clean - [x] go test -race -count=1 ./internal/application/repository/retriever/opensearch/... passes - [x] grep -r "case types.OpenSearchRetrieverEngineType" internal/ shows only PR 1's normalizer case + this driver's EngineType() and tests — no activation path. - [x] grep -r "case \"opensearch\"" internal/ shows no hits. |
||
|
|
d9ba7716b5 | fix(agent): harden tool parameter parsing against LLM type mismatches (#1505) | ||
|
|
3645477dba | refactor(qdrant): optimize batch save, map iteration, and error wrapping (#1508) | ||
|
|
3611a8607d |
feat(rbac): add multi-use share-link invitations for invite_only mode (#1504)
* feat(rbac): add multi-use share-link invitations for invite_only mode When the auth.registration_mode toggle flips to invite_only, the public /auth/register endpoint returns 403 — but until now there was no channel for unregistered users to actually join a tenant. The existing invitation table required a registered invitee, so new emails could not enter the system at all. This change introduces a share-link model: - Owner generates a multi-use registration link with a per-tenant role - Recipient opens the link, registers with their own email, and is added to the tenant on the same request - Token is stored in plaintext (not hashed) so the management UI can re-display the URL on demand without a "copy now or revoke" trap. Threat model is bounded by 7-day TTL, revocability, and the fact that all the link grants is membership in one tenant. - accepted_count tracks how many users joined through the link so the Owner can tell whether a link is fresh or has already been spread Per-user invitations (registered email -> in-app inbox accept) are unchanged. Share-link rows use empty invitee_user_id as discriminator; the partial unique index on (tenant_id, invitee_user_id) was relaxed to skip empty values so multiple share links can coexist per tenant. Frontend reuses Login.vue for /register?token=xxx instead of a parallel page, and adds icon-only revoke / remove actions plus a distinct "active" status tag for share-link rows. Migration 000054 adds the token column, the accepted_count column, and the relaxed pending unique index in a single step. * feat(auth): update invitation token handling and add rate limiting - Refactor the invitation token retrieval to use POST instead of GET, enhancing security by preventing the plaintext token from appearing in logs and browser history. - Update the API endpoint from `/auth/invitations/:token` to `/auth/invitations/lookup` to reflect the new method. - Introduce a rate limiter for unauthenticated share-link endpoints to mitigate brute-force attacks and abuse, ensuring a maximum of 30 requests per minute per IP. - Adjust the registration flow to maintain user experience while securing the invitation process. |
||
|
|
6220667517 |
fix(im): recover from deleted session when GetSession returns app sentinel (#1499)
The recovery branch added in
|
||
|
|
39efcf02d2 |
feat(settings): replace monogram badges with brand logos
Drop the per-engine first-letter monograms on Vector Store / Storage /
Web Search cards in favor of real brand SVGs. Logos live under
`assets/img/providers/{color,mono}/<category>/<id>.svg`:
- color/: full-color vendor SVGs (gilbarbara/logos, svgl, vendor github)
rendered directly via <img>; covers qdrant, milvus, elasticsearch,
postgres, opensearch, sqlite, s3, minio, bing, duckduckgo, google,
ollama, searxng.
- mono/: single-color marks (simple-icons / vendor) rendered via
mask-image so the existing per-engine `color: <brand>` rule tints
them; covers doris, cos, obs, oss, baidu.
- Anything without a logo falls back to the existing monogram badge.
A small `providerLogos.ts` helper auto-discovers SVGs with
`import.meta.glob` and returns `{ mode, url }`; the three settings
views read it to pick the render path. Badge styles stay backwards
compatible: a `--logo` modifier swaps the brand-tinted background for
a neutral white card with a thin stroke when a logo is present.
|
||
|
|
362c973aad |
refactor(settings): move vector store test into menu, toast result
The standalone "Test connection" button cluttered the title row and the inline result bar made cards visually unstable as users ran tests. Behavior change: - Move "测试连接" into the more-actions dropdown as the first item. env-source stores now also get a dropdown (test-only) instead of showing a hidden placeholder button. - Surface the result via MessagePlugin.success / .error toast and drop the per-card testResults state, getTestResult(), clearTestResult(), and the .store-card__result* SCSS rules. - Keep an inline spinner in the title row while a test is in flight, so the user still sees progress after the menu closes. |
||
|
|
272be9e8c5 |
feat(settings): redesign vector store cards
Bring vector stores in line with the rest of the settings list redesign. The previous list-row layout (lock icon + 2 tags + endpoint subtitle + buttons) carried the same "low" feel as the old model/provider cards: flat, tag-heavy, no engine identity. - Replace .store-list rows with a .store-grid card grid sharing the same skeleton used by Parser/Storage/Mcp: a 36x36 monogram badge tinted per engine_type (qdrant red, milvus blue, weaviate green, elasticsearch amber, postgres blue, opensearch / infinity purple, tencent_vectordb blue, doris orange, sqlite gray, others fall back to brand blue). - Title row carries the name + an env pill for read-only stores + a low-contrast text-style "Test" button (admin-only) + the hover-only ellipsis menu (edit / delete, user stores only). - Subtitle: engine_type · endpoint (monospaced, truncated). - Test result keeps its inline success/error bar but rebuilt as store-card__result so it lives inside the card instead of hanging off a sibling row. Drops the .store-list / .store-item / .item-* / .test-result-* selectors that were only ever used here. |
||
|
|
68d41ae7d9 |
feat(settings): redesign parser/storage engine cards
Bring the engine selectors in line with the rest of the redesigned settings cards (Model / WebSearch / Mcp). Same skeleton: leading 36x36 monogram badge tinted per engine, clean title row, dot+label status pill, two-line description, brand-color outline when the config drawer is open for that card. Parser engines: - Replace the bespoke .engine-card markup with a button-based card carrying a badge derived from the localized display name. - Keep available/unavailable as a status pill (green dot / red dot) instead of t-tags, retaining the unavailable-reason tooltip via a dedicated --help variant. Storage engines: - Collapse the eight hand-written cards into a single v-for over a STORAGE_PROVIDERS array; titles, monograms, and statuses funnel through providerTitle/providerInitial/providerStatus helpers so adding a future engine only needs an array entry plus translations. - Status pill mirrors Parser/Mcp idioms: green for available / configurable, gray for "needs config" (currently MinIO-only). Drops the now-unused .engine-card-header/.engine-card-desc/ .tag-with-tooltip rules. |
||
|
|
48e1886089 |
refactor(settings): tighten provider card chrome
- Web search: drop the "default" and "free" pills from the card header; the provider type subtitle and the existing form-side free badge already cover this, and three rows of pills made the cards feel busy. - MCP: collapse the inline switch and the on/off tag into a single on/off status pill (dot + label), and move the actual flip into the more-actions dropdown as the first item. Built-in services keep showing the "builtin" pill instead, since they're not togglable. Drops the now-unused isEntityFree helper and the .provider-card__pill SCSS variants. |
||
|
|
260b3eec26 |
feat(settings): redesign websearch/mcp provider cards
Apply the model-card pattern to the other two provider lists in the new "Data & Extensions" group so they share one visual idiom. Web search providers: - Drop the 3-tag row (provider type / default / free) for a 36x36 monogram badge tinted per provider (duckduckgo, bing, google, tavily, baidu, searxng, ollama). - Title row carries the entity name plus optional "default" / "free" pills; the more-actions button hides until hover/focus and opens on click. - Subtitle reads "Provider type · description"; proxy URL drops to a muted monospaced last line, no leading icon. - Grid is responsive: repeat(auto-fill, minmax(320px, 1fr)). MCP services: - Same skeleton with a transport-type badge (sse → cast / sse-green, http-streamable → link / brand-blue, stdio → code / amber). - Keep the inline enable switch, but slot it into the title row so the card stops repeating "on/off" as a tag. - Builtin services keep the muted background to signal read-only. - Three-dot menu hides until hover/focus and opens on click. Drops dead CSS (.provider-meta-item, .service-meta-item, the old 2-column grids) and the now-unused SettingCard imports. SettingCard itself is left in place for future consumers. |
||
|
|
f10b83d92f |
feat(settings): redesign model card with type badge
The two-column model list looked low-res because every card shared the same skeleton, with three saturated tags and a raw baseUrl meta line carrying most of the visual weight. Replace SettingCard usage on this page (other consumers — Mcp and WebSearch settings — keep using it untouched) with an inline ModelCard built around: - A 36x36 leading badge per model type (chat, embedding, rerank, vllm, asr) with low-saturation tinted backgrounds and a matching tdesign icon, so users can scan by type at a glance instead of decoding tag colors. - Title row with the model name and an optional muted "builtin" pill; the more-actions button only appears on hover/focus and opens the dropdown on click (not hover) to match the rest of the app. - A muted subtitle (type · source · embedding dimension) and a monospaced, single-line truncated baseUrl as the last line — so secondary info reads as secondary. - Responsive grid: repeat(auto-fill, minmax(320px, 1fr)) so wider modals get 3 columns and narrow viewports collapse to 1. Drops dead CSS (.model-type-tag-* color rules, the page-wide :deep(.t-tag) override, .model-meta-item) and the now-unused SettingCard import. |
||
|
|
ad82692785 |
refactor(settings): regroup sidebar nav and pin header on scroll
Sidebar UX polish for the Settings modal: - Regroup nav into 5 cleaner buckets: Account / Workspace / Models / Data & Extensions / Platform. Personal preferences (general) and credentials (api) move to Account; chathistory follows its scope into Workspace; integrations and engines merge to avoid two narrow groups. - Pin the sidebar header (title) so it stays visible while the nav list scrolls; move overflow from the sidebar wrapper to the nav list itself. - Tighten paddings/font sizes a notch for higher information density without sacrificing hit targets. - Sync the four locale files (zh-CN as source of truth, plus en-US, ko-KR, ru-RU): drop workspaceAccount / integrations / knowledgeInfra; introduce account / workspace / dataExtensions. |
||
|
|
b395db55de |
chore(cli): adapt ListKnowledgeChunks interface to variadic chunk_type filter
Upstream commit
|
||
|
|
2ee9741fa1 |
refactor(cli): finish context→profile cascade + post-review hardening (BREAKING)
Post-review polish on the v0.7 wire / surface contract. Bundles five
follow-ups that landed after the main BREAKING feat commit:
1. Complete context→profile cascade (internal API + YAML schema)
The prior commit renamed only the user-visible surface (commands /
flags / env / project link / envelope field). The internal Go API
and on-disk config schema were still half-renamed — an L-25
self-consistency violation flagged by post-merge review. Closed here:
Internal Go API:
- config.Context → config.Profile
- config.Config.CurrentContext → CurrentProfile
- config.Config.Contexts → Profiles
- LoginOptions.Context → LoginOptions.Profile
- clearContextSecrets() → clearProfileSecrets()
- saveContextRef() → saveProfileRef()
- secrets.Store: param name `context` → `profile` (interface +
FileStore + KeyringStore + MemStore)
- cmdutil.LoadSecret(store, context, key) → LoadSecret(store, profile, key)
- cmdutil.RefreshAndPersist's ctxName → profileName
- Local var `ctx := &config.Profile{...}` → `prof := &config.Profile{...}`
in auth/login.go to eliminate the visual collision with Go stdlib
context.Context that motivated the whole rename in the first place.
On-disk config.yaml schema:
- current_context: → current_profile:
- contexts: → profiles:
- Pre-1.0 break, no compat alias. Users on v0.6 dogfooded configs
must delete ~/.config/weknora/config.yaml or hand-rename the two
keys (CHANGELOG migration note added).
Tests / fixtures / golden files:
- factory_test.go YAML fixture + assertion updated.
- acceptance/e2e/e2e_test.go writeContextYAML → writeProfileYAML,
fixture YAML keys updated.
- acceptance/testdata/wire/doctor.error_network.json golden updated
("active context" → "active profile" in hint string).
User-visible prose sweep:
- cmd/mcp/serve.go --help Long: "active context (or --context)" →
"active profile (or --profile)" — most-visible miss.
- cmd/{kb/list, search/kb, session/list, api/api} Short/Long help.
- cmd/auth/login.go stdout: `(context=%s)` → `(profile=%s)`.
- cmd/auth/logout.go error: `"no current context"` → `"no current profile"`.
- cmd/doctor/doctor.go hint string (also the wire golden above).
- cmd/auth/refresh.go error: `"refresh token missing for context"` →
`"refresh token missing for profile"`.
- README.md: `## Multi-context` H2 → `## Multi-profile`; code-block
comment `# current context` → `# current profile`.
Code-comment / docstring sweep across cli/cmd/auth/ and
cli/internal/cmdutil/. Comments referencing Go stdlib context.Context,
the RAG / LLM "context window" concept, and historical CHANGELOG
entries for v0.4 / v0.5 were left alone.
CHANGELOG v0.7 BREAKING entry gains the on-disk-schema bullet under
the existing "context → profile" item.
2. Profile name validation (shell-injection guard)
`envelope.error.retry_command` is a single shell-string field. An
AI agent that exec()s it via `sh -c <retry_command>` was injectable
through a maliciously-named profile:
weknora auth logout --name 'x; rm -rf ~'
# would produce: retry_command = "weknora auth logout --name x; rm -rf ~ -y"
`cmd/profile/add.go` already enforced an alphanumeric + `-_.`
allowlist via `validateName`. The `auth login` and `auth logout`
paths bypassed it.
- Moved validation from `cmd/profile/add.go` to
`cli/internal/cmdutil/profilename.go` as exported
`ValidateProfileName` (cmdutil is the import-cycle-safe home;
internal/config can't depend on cmdutil).
- `auth login` runs the validator before any persist call.
- `auth logout` runs the validator on `opts.Name` before
constructing `retry_command`.
- Unit tests (`profilename_test.go`) cover the allowlist, empty
rejection, path-traversal, shell metacharacters (`;`, `&`, `|`,
`$()`, backticks, quotes, whitespace, glob, redirects), and the
user-facing hint text. The shell-metachar test exists as a
regression guard.
Wire shape (`retry_command` string → `retry_command_argv []string`)
remains a v0.8 additive change per ROADMAP — this fix removes the
practical exploit path without touching the wire contract.
3. AI-agent terminology disambiguation
"agent" has three referents in this codebase: (a) WeKnora's
server-side Custom Agent resource, (b) the removed `agent invoke`
verb, (c) external LLM/automation consumers. Per project memory
feedback_no_meta_disambiguation_in_docs, the fix is full-term
naming, not "X has N meanings" prose. Surgical changes at section
headers + ambiguous prose:
- AGENTS.md: "Agent decision shortcuts" → "AI agent decision
shortcuts"; "agent-callable surface" → "AI-agent-callable
surface".
- README.md: "Designed to be agent-first" → "AI-agent-first";
"Other agent ergonomics" → "Other AI-agent ergonomics"; "in
agent contexts" → "in AI-agent contexts"; "for CI / agents" →
"for CI / AI agents".
Anaphoric "agents" inside paragraphs that already established
"AI agents" was left alone — full substitution everywhere would
have been prose noise without clarity gain.
4. Wire-contract review follow-ups
Real findings from a second-pass review of the v0.7 envelope /
streaming / surface design. Per project memory
feedback_check_in_domain_anchor_first, candidate findings were
first verified against the in-domain peer CLI explicitly cited as
the envelope anchor; two earlier-flagged issues turned out to be
in-pattern and were withdrawn.
Surviving fixes:
- AGENTS.md success-envelope example rewritten. The prior example
showed `has_more: false` / `_notice: {}` as if they were always
present, but both fields are `omitempty` and never serialize
when zero / nil. Replaced with three realistic shapes (list /
single resource / mutation with no payload) and added a note
that optional fields are omitted when empty.
- cmd/chat/chat.go Args: MinimumNArgs(1) → ExactArgs(1).
v0.6 silently joined `weknora chat hello world` into
`"hello world"`. v0.7 now rejects multi-arg with exit 2,
matching `weknora session ask`. BREAKING; CHANGELOG entry
added under v0.7 BREAKING.
- internal/output/envelope.go extracts NewEnvelope(data, meta,
profile) constructor. The jq-filter path in
cmdutil.FormatOptions.Emit was manually rebuilding the
envelope literal alongside the canonical WriteEnvelope path —
drift risk when fields are added. Single construction point now.
- internal/cmdutil/factory.go adds AddKBFlag(cmd) helper.
Five files (chat, doc/list, doc/upload, doc/create, doc/fetch)
had verbatim-identical `cmd.Flags().String("kb", ...)`
declarations. Centralised so flag name + help text stay
in sync with Factory.ResolveKB. Docstring reordering + gofmt
fixup landed in the same edit to keep ResolveKB's own godoc
attached to its function.
5. OSS-readiness comment / doc sweep
Pre-publication scrub of code, comments, and shipped Markdown to
remove references that only make sense in the development repo:
- AGENTS.md "Deliberate deviations + mainstream alignments"
section: removed peer-project name-drops from the comparison
table; rewrote as five flagged design decisions with rationale
but no specific competitor named. The four rows that previously
contrasted against a named peer CLI now state WeKnora's choice
+ rationale directly. Section header renamed to "Design
decisions worth flagging" since it is no longer a
deviation/alignment matrix.
- CHANGELOG v0.7 BREAKING rationales: three references to a
named peer CLI removed; the context→profile rationale now
cites only mainstream multi-credential CLIs by category (AWS /
Stripe / OpenAI / Anthropic), and the `api -d/--data` removal
rationale cites only `gh api` / `curl`. `chat` BREAKING entry
rationale similarly simplified.
- 35 cross-references to design-spec section numbers (§4.1 /
§4.5 / §5.3 etc.) removed from Go doc comments and test
comments across 13 files. The referenced spec lives outside
the shipped tree; readers of the public repo cannot resolve
them. Each reference replaced with a self-contained semantic
description (e.g. "the batch envelope" / "AGENTS.md section
on the success path").
- Mixed-language strings translated to English:
- Four Go comments: internal/cmdutil/exit.go:213,215,
internal/cmdutil/errors.go:156,
internal/output/batch_test.go:90,
internal/output/envelope_test.go:27.
- One CHANGELOG section title:
`v0.7 — Agent-first wire contract + 命令面集中清理` →
`... + command-surface cleanup`.
- CJK test fixtures (internal/text/truncate_test.go CJK
truncation cases, cmd/session/list_test.go Chinese session
title, acceptance/e2e/e2e_test.go Chinese RAG corpus)
retained — they are intentional test inputs, not stray prose.
- Makefile help comment: `golangci-lint added in PR-9` →
`golangci-lint planned`. Internal PR numbering should not
surface in shipped Makefile prose.
Build green, 28/28 packages, +5 new ValidateProfileName tests.
go vet / gofmt / go mod verify / go mod tidy all clean.
Rationale for the cascade: pre-1.0 is the cheapest moment to close
L-25 self-consistency (L-26). The half-finished internal rename
would have perpetuated the very `context` vs `context.Context`
ambiguity that motivated v0.7's user-visible rename in the first
place.
|
||
|
|
2ce348d020 |
feat(cli): --format json default + NDJSON event stream + context→profile cascade + help calibration + docs (BREAKING)
D1 — --format default flipped to json regardless of TTY: - v0.6: smart default (text on TTY, json on pipe). - v0.7: always json; TTY only affects indent (compact in pipe). Enum values unchanged (text | json | ndjson). - Typed FormatMode enum replaces untyped string consts. - --format / --jq promoted to persistent root flags so unknown- subcommand paths still reach the typed-envelope guard (per-command registration in v0.6 would have rejected --format on unknown commands as cobra-prose exit 2). - WEKNORA_FORMAT env var added; precedence --format > env > default. Invalid env values silently ignored. D2 — chat / session ask default to NDJSON event-stream: - New cli/internal/output/ndjson_stream.go: InitEvent struct + EmitInit / EmitSDKEvent / WriteNDJSONLine helpers. EmitInit doc encodes the must-be-first-line invariant agents key on. - chat / session ask: --format json AND --format ndjson both emit one JSON event per line (no envelope wrapping). CLI injects exactly one `init` event at stream head carrying session_id + optional kb_id / agent_id / profile. Subsequent events pass through verbatim from the SDK (passthrough discipline per spec §5.1). - --format text keeps the SSE-style live renderer. context → profile full cascade: - Command group: cli/cmd/context/ → cli/cmd/profile/ (git mv; package contextcmd → profilecmd). - Global flag --context → --profile. Factory.ContextOverride → ProfileOverride. WEKNORA_PROFILE env var honored (--profile flag > env > config.CurrentContext). When --profile or WEKNORA_PROFILE references a missing profile, the error is input.invalid_argument with hint "weknora profile list" — not the destructive local.config_corrupt path (which would have told users to delete their config file). - Binding file .weknora/project.yaml field context: → profile: (no backwards-compat alias; re-run weknora link). - profile use JSON fields current_context / previous_context → current_profile / previous_profile. - weknora link JSON field context → profile. - CodeLocalContextNotFound → CodeLocalProfileNotFound (typed code rename). - Envelope top-level profile field populated via globalProfile (set by root PersistentPreRunE from Factory.ActiveProfile). chat / session ask NDJSON init event carries the same profile. - Rationale: "context" collided with LLM context window / RAG context / Go context.Context; mainstream multi-credential CLIs (AWS / Stripe / OpenAI / Anthropic / lark) all use "profile". H2/C1' help calibration: - AgentHelp gains Warnings []string; single SetAgentHelp helper routes on WEKNORA_AGENT_HELP=1 (emits JSON blob including warnings) vs human help (appends "AI agents:" block from same source). Warnings surface as both a structured JSON field and visible help-text addendum without drift. - 9 destructive commands carry warnings: kb / doc / agent / session / chunk delete; profile remove; kb / agent edit; auth logout. - weknora doc wait dedups ids at entry; SIGINT mid-wait returns silently (root signal handler maps to exit 130) instead of being miscategorised as operation.timeout / operation.failed. A4 — docs: - cli/AGENTS.md gains four agent-facing sections: Wire contract for AI agents (stdout / stderr / NDJSON / _notice evolution / SDK contract boundary); Deliberate deviations + mainstream alignments; Pre-1.0 breaking policy; Exit-10 anti-patterns. ERROR_REFERENCE table extended. - cli/README.md adds Agent quick start under Wire contract. - cli/CHANGELOG.md v0.7 section: BREAKING entries with migration notes, Added (WEKNORA_FORMAT / WEKNORA_PROFILE / retry_command / retry_after_seconds / risk / _notice reserved infra / meta.count / meta.has_more / doc fetch / doc create / session ask / doc delete --all / NDJSON init), Changed (docs additions), Deprecated (none — pre-release one-shot breaking). Spec: docs/superpowers/specs/2026-05-20-weknora-cli-v0.7-design.md §3 / §4 / §5 / §6 / §11 |
||
|
|
a2e368b1e8 |
refactor(cli): command-surface rename — session ask / doc fetch / doc create / doc delete --all
Three command renames consolidate the v0.7 verb table:
- `weknora agent invoke` → `weknora session ask --agent <id>`.
Server route POST /sessions/{session_id}/agent-qa is session-
anchored, so the verb moves with it. `weknora agent` subtree keeps
CRUD only (list / view / create / edit / delete / status / check).
SDK call (AgentQAStreamWithRequest) preserved verbatim; only the
command surface + flag layout move.
- `weknora doc upload` split into three commands:
- `weknora doc upload <file>` — local file (only).
- `weknora doc fetch <url>` — server-side remote fetch (was
`upload --from-url`). URL-only flags (--title / --file-type /
--tag-id) move with the verb.
- `weknora doc create --text` — direct text knowledge entry via
server CreateManualKnowledge.
- `weknora kb empty <id>` → `weknora doc delete --all --kb=<id>`.
Atomic server ClearKnowledgeBaseContents (no list-then-delete
race). Same exit-10 -y/--yes guard as other destructive verbs;
unified through the extended ConfirmDestructive helper.
Parent commands (agent, kb, doc, chunk, session, auth, profile,
search) lose their explicit Args:NoArgs + Run:cmd.Help so the
unknown-subcommand guard fires correctly — `weknora agent invoke
ag_x q` now emits the typed input.unknown_subcommand envelope with
detail.available[] instead of cobra's free-form exit-2 prose.
Spec: docs/superpowers/specs/2026-05-20-weknora-cli-v0.7-design.md §3.4 / §10.7
|
||
|
|
ad150c7126 |
refactor(cli): envelope sweep — Emit shape, batch ops, MCP StructuredContent, api flag cleanup
Sweep every CLI output path through the v0.7 envelope contract.
FormatOptions.Emit signature changes to (w, data, meta *output.Meta)
so list commands surface meta.count / has_more in the wire envelope.
TTY indent decision plumbed via FormatOptions.TTY (set by
ResolveDefault). --format and --jq promoted to persistent root flags
so unknown-subcommand paths still reach the typed-envelope guard
(per-command registration would have rejected --format on unknown
commands as cobra-prose exit 2). WEKNORA_FORMAT env-var fallback
honored before TTY-resolved default. AddFormatFlag retained for the
per-command --jq field-hint Long-text addendum.
~55 production Emit call sites swept across cli/cmd/**: list paths
populate meta.count; mutation paths pass nil meta; envelope.profile
threaded via globalProfile. Existing JSON-shape tests migrated to
decode the envelope wrapper.
Multi-target batch operations get a unified shape via new helpers:
- output.BatchItem + output.WriteBatchEnvelope (§4.5 per-item ok
pattern; data:[{id, ok, result?|error?}], meta:{count, successes*,
failures*}; Successes/Failures *int so explicit zero survives
omitempty in all-fail case).
- cmdutil.RunBatch + cmdutil.EmitBatch + cmdutil.DeletedAtNow +
cmdutil.ClassifyContextErr collapse the three previously-duplicated
multi-delete pipelines (doc / chunk / session) into one call site
pattern. doc upload --recursive uses the same helpers with a per-
file resultFn. Summary error sets Silent:true so the stderr
envelope path doesn't duplicate the stdout batch detail (exit 1
preserved via Code → ExitCode).
ConfirmDestructive / ConfirmDestructiveBatch extended with action +
retryCmd parameters. Every destructive command (kb delete, kb edit,
doc delete, doc delete --all, agent delete, agent edit, session
delete, chunk delete, auth logout, api -X DELETE) now attaches
error.risk.{level:"destructive", action:"<noun.verb>"} +
error.retry_command on the exit-10 envelope. kb edit / agent edit
also gain the destructive guard wiring (CHANGELOG listed them as
gated but they were silently executing without -y).
MCP tool handlers' 31 error paths return CallToolResult with
IsError + Content text fallback + StructuredContent (envelope.error
shape via cmdutil.ErrorToDetail). Handler Out type changed from
typed *sdk.X to any so the go-sdk auto-marshal doesn't overwrite
StructuredContent with a zero-struct on error returns. Success path
manually populates CallToolResult.StructuredContent via successResult
helper.
weknora api cleanup:
- -d/--data flag removed; body via --input <file> or --input -
(stdin) only. Aligns with gh / curl convention.
- HTTP method whitelist removed; any non-empty method accepted.
- HTTP Retry-After header → cmdutil.Error.RetryAfterSeconds →
envelope.error.retry_after_seconds.
- runAPIPaginated raw-passthrough fallback now wraps response in
envelope so --paginate never emits bare JSON to stdout.
Test sweep: ~60 cmd/** tests migrated to decode envelope shape.
9 wire-contract goldens updated under cli/acceptance/testdata/wire/.
Spec: docs/superpowers/specs/2026-05-20-weknora-cli-v0.7-design.md §4 / §5
|
||
|
|
733bb3aaa1 |
refactor(cli): symmetric envelope infrastructure (supersedes e623e820)
Re-introduce the agent-first symmetric envelope deleted in commit |
||
|
|
7d1fe6f11b |
chore(deps): add opensearch-go v4.6.0 to go.mod and update go.sum
- Added the opensearch-go v4.6.0 dependency to the go.mod file. - Removed the indirect requirement for opensearch-go from go.sum as it is now a direct dependency. |
||
|
|
2a117262bf |
test(system): cover system.* audit constants + emitAdminAudit contract
Two test surfaces, picked for cost/value:
internal/types/audit_log_test.go — extend the existing invariant suite
to include the system namespace:
- DotNamespaceConvention now covers system.setting_changed,
system.admin_promoted, system.admin_revoked.
- NoCollisionsAcrossNamespaces guards against duplicates across all
three new constants.
- New SystemNamespacePrefix test pins the shared "system." prefix —
this is the contract by which GET /system/admin/audit-log filters
out per-tenant rbac.* rows. Drift here would either leak per-tenant
events into the platform feed or hide platform events from
SystemAdmin.
- New SystemWireValues test pins the exact wire strings consumed by
the new frontend audit drawer, Langfuse exporters, and future SIEM
integrations; changes to these are a breaking change.
internal/handler/system_admin_audit_test.go — direct unit tests for
SystemHandler.emitAdminAudit, the helper that promote/revoke /
ApplyDefaultStorageQuotaToAllTenants all delegate to. Uses a
capturingAuditService stub (interface-embedded so any other method
call surfaces drift loudly) and a minimal SystemHandler with only
auditSvc wired — the helper deliberately doesn't touch other deps.
Coverage:
- NilServiceIsNoop: degraded-mode contract — a handler built without
an audit service must not panic on the audit hook.
- PopulatesCanonicalFields: every responsibility of the helper —
TenantID=0 (system scope), actor from ctx, role hard-pinned to
"system_admin", action passed through, outcome=success,
TargetType="user", TargetID/TargetUserID echoing user.ID, details
round-tripping through JSON.
- NilDetailsLeavesEmptyPayload: nil details map must NOT fabricate a
payload; the DB column defaults to '{}' and emitting an explicit
null would muddle "no extra context" filters.
- NilTargetStillEmitsRow: guards the nil-target defensive branch —
promote/revoke always supply one today, but the row still goes out
with empty target ids rather than crashing.
- IdempotentBranchSurvivesMarshal: pins the two boolean discriminator
flags (promote.idempotent, revoke.changed) so the audit reader can
distinguish a real grant from a probe and a real revoke from a noop.
Regression guard against accidentally swapping the payload to
stringly-typed shapes.
- LogErrorIsSwallowed: best-effort contract — a failing audit write
must NOT propagate, because the underlying privilege change has
already succeeded and bubbling the error would force the caller to
retry or roll back, both strictly worse than log-and-continue.
|
||
|
|
5b9542ebe6 |
test(system): cover ListSystemAuditLog handler contract
Mirrors the existing TestAuditLogHandler_* suite for the new GET /system/admin/audit-log endpoint: - AlwaysQueriesTenantZero: the defining contract — handler must call AuditLogService.List with tenant_id=0 unconditionally, regardless of any URL/header input. A regression here would leak per-tenant rbac.* rows into the platform feed (or hide system.* rows from SystemAdmin). - PassesQueryFiltersThrough: every advertised query key (after_id, limit, action, outcome, actor) propagates exactly. Catches typos in the param-key list. - EmptyResultProducesZeroCursor: an empty service response must collapse next_cursor to 0 so the drawer's infinite-scroll watcher stops paginating. - GarbageCursorAndLimitTolerated: malformed after_id / non-positive limit fall back to defaults (matches ListTenantAuditLog) instead of hard-failing, so stale URL params never blank-screen the drawer. - ServiceErrorReturns500: List() errors surface as 500 via errors.NewInternalServerError + ErrorHandler middleware, with a non-empty body so the drawer alert has something to render. |