- Updated the listFAQEntries function to support multiple tag IDs for filtering.
- Modified the frontend components to accommodate the new tag_ids parameter.
- Enhanced the KbTagManageDrawer and FAQEntryManager components to reflect changes in tag handling.
- Improved API documentation to clarify the new tag filtering capabilities.
This commit enhances the flexibility of FAQ entry retrieval, allowing users to filter by multiple tags for more precise results.
- Updated the exportFAQEntries function to support exporting FAQ entries in both CSV and JSON formats.
- Modified the frontend to include options for selecting the export format.
- Enhanced localization files to reflect the new export options in English, Russian, and Chinese.
- Implemented new tests to validate the export functionality for both formats.
This commit improves the flexibility of FAQ exports, allowing users to choose their preferred format for better integration with other tools.
- Added new fields for tracking added, merged, and partially failed entries during FAQ imports.
- Updated the import result statistics to include detailed counts for added and merged entries.
- Enhanced the UI to display progress hints and improved import result summaries.
- Implemented a new method for saving chunks in a single transaction to improve performance.
This commit improves the visibility and accuracy of FAQ import results, providing better user feedback and tracking capabilities.
- Added new actions for FAQ import lifecycle: `faq.import_started`, `faq.import_completed`, and `faq.import_failed`.
- Updated activity details to include success, failure, and skipped counts for FAQ imports.
- Enhanced UI to display additional activity information, including filtering options and clearer summaries.
- Implemented tests to validate the new FAQ import outcomes and activity details.
This commit improves the tracking and visibility of FAQ import activities within the knowledge base, ensuring better audit trails and user feedback.
Automatically remove wiki folders that become empty when knowledge is
retracted, using the durable finalize lane so pruning waits until ingest
drains. Repository DeleteFolder now atomically checks emptiness to avoid
races with concurrent page or child-folder writes.
Two backend fixes for the v0.7.0 regression where a deleted document keeps
showing in the list ('delete success' toast) and a follow-up reparse then
reports the file is missing:
1. List filtering: applyKnowledgeListFilter now hides rows in the transient
'deleting' state from the default document list. The async delete pipeline
marks a row 'deleting' before tearing down its resources, so without this
the entry lingers as a normal doc until it is physically gone. Rows whose
delete task exhausts retries are flipped to 'failed' by the dead-letter
callback and stay visible so the failure remains actionable; an explicit
parse_status=deleting filter can still surface in-flight rows.
2. Delete ordering: DeleteKnowledge / DeleteKnowledgeList now delete the DB
row FIRST and defer physical file (+ extracted image) cleanup until after
the row is gone. Previously file removal ran concurrently with the
index/chunk/graph cleanup; when one of those failed the row was kept but
the file could already be deleted, producing a 'file missing but row
present' zombie that could neither be reparsed nor cleanly re-deleted.
Orphaning a file after the row is gone is the tolerable failure mode.
Adds a repository regression test covering the default/explicit deleting
visibility rules.
Store authorization timestamps as timestamptz and normalize Unix expiry
inputs to UTC so tenant/platform API keys expire at the intended instant
even when the app process timezone differs from the PostgreSQL session.
Cloned images were unreachable via GET /knowledge-bases/:id/files:
copyOwnedObject used CopyFile, which writes to the knowledge-scoped upload
layout ({tenant}/{knowledgeID}/...), but the KB proxy only serves objects
under {tenant}/exports/ (ValidateKBScopedStoragePath). Read the source
bytes and re-save via SaveBytes instead, mirroring how images are
persisted during ingestion, so copies land in the servable exports
namespace and remain independent of the source object.
For document knowledge, extracted images live in independent image_ocr/
image_caption child chunks that carry image_info, while the parent text
chunk embeds the  reference with an empty image_info. Rewriting
content per-chunk therefore missed the parent text chunk, leaving it
pointing at soon-to-be-deleted source storage.
Move content URL rewriting to a final pass over all cloned chunks, run
once urlCache holds the complete old->new mapping. FAQ chunks are
self-contained, so they are rewritten inline from the same cache.
When cloning knowledge chunks, image_info was deep-copied to new storage
objects but Content still embedded the source markdown image URLs. After
the source knowledge is deleted, those in-content references break while
image_info points at the new copies. Rewrite Content alongside image_info
so both stay consistent.
SendReply now sets param.formatType=markdown by default since WeKnora
replies are authored in Markdown. Callers can override via
reply.Extra["yunzhijia_format_type"] (empty string disables the param).
Raise theme.css selector specificity so TDesign bundled styles cannot
override WeKnora tokens after minification, and add vite preview proxy
so production-like local testing can reach the API backend.
- Added AminusB method to the knowledgeRepository, which returns IDs of knowledge in A that have no counterpart in B, comparing by file_hash as a multiset.
- This implementation addresses issues with handling duplicate entries and NULL file_hash values, ensuring accurate synchronization between knowledge bases.
- Updated the KnowledgeRepository interface to reflect the new functionality and improved documentation for clarity.
- Modified tenant_api_keys table to allow nullable tenant_id and added scope_type with constraints.
- Removed legacy platform API keys migration files to streamline database management.
- Added new index for scope_type to improve query performance.
Enhance internationalization support by adding new localization entries for the audit log detail drawer in English, Korean, Russian, and Chinese. This includes sections for event summary, related IDs, request information, and method.
Introduce platform API keys with capability-based access to tenant management,
system settings, runtime queues, and audit logs, plus admin UI for key lifecycle
and system audit log viewing.
Large Feishu wikis stopped syncing after the first batch (Tencent/WeKnora#2136:
5000+ docs, only ~150 ingested; #1262: a child-list failure aborted the run).
Three compounding causes, fixed in layers:
1. Rate-limit / transient failures had no retry. Feishu's wiki/drive/export
APIs are aggressively rate limited; a thousand-doc sync issues tens of
thousands of calls, and a single 429/5xx burst failed whole swathes of docs.
Add exponential backoff honouring Retry-After to doRequest (JSON) and
downloadRawBytes (files): 429 up to 3x, 5xx once, transport errors back off.
2. The batch path fetched the entire tree into memory before ingesting, so a
sync that hit the 2h task timeout lost all progress and restarted from
scratch, and a transient per-document export failure advanced the cursor and
skipped that doc permanently. Add an optional StreamingConnector interface,
implemented for Feishu: FetchStream interleaves fetch -> ingest -> checkpoint,
bounding memory to one item and persisting a per-node edit-time cursor at
page boundaries. The service dispatches to it via type assertion; connectors
without it fall back to FetchAll / FetchIncremental unchanged (non-breaking).
A resumed/incremental run skips nodes already recorded at their current edit
time (converges), a failed fetch does not advance the cursor (retried next
run instead of dropped), and partial listing failures emit error items and
carry prior edit times forward instead of aborting the sync.
3. Failures and skips were invisible. Unsupported node types (mindnote/slides/…)
were skipped with no item, error or log ("13 docs synced only 3"). FetchAll
now tallies discovered/fetched/failed/skipped per resource with a per-type
skip breakdown and a progress line every 100 nodes. The backend already
persisted per-item errors in SyncLog.result.errors but the frontend type
omitted the field; declare `result` on SyncLog and render up to 50 failed
documents in the sync-log drawer (#1262). i18n for zh-CN/en-US/ko-KR/ru-RU.
result.errors is a jsonb column shipped in every list response, so it is
capped at 100 samples server-side (recordSyncError); the exact failure total
comes from the bounded items_failed counter, which the drawer uses for its
count and "N more" line so a sync that fails thousands of docs stays bounded.
Each sample is a structured, localisable SyncItemError instead of a raw
"title: ...api error: status=500 body={json}" dump that repeated the title
and leaked status/body/log_id. The connector (feishuFailure) classifies the
error into a stable i18n code (+ the feishu code as a param); the service
builds {title, code, params, message}; the drawer localises via vue-i18n to
the viewer's language (zh/en/ko/ru), falling back to message. A custom
UnmarshalJSON keeps old string-form logs readable. The banner is localised
from items_failed too. Raw errors stay in the server logs. Follows the
two-tier norm of Airbyte/Fivetran/Onyx: human, actionable, localised
summaries in the UI; raw detail in logs. Transient errors say "will retry"
(the cursor is retained); auth/permission errors point at the fix instead.
Tests: client_retry_test (backoff), tally_test (accounting), connector_stream /
datasource_stream (streaming dispatch + checkpoint), connector_convergence
(timeout + transient-failure resume converges to full coverage; proven to catch
#2136 via bug injection), datasource_result_cap (error-sample cap + structured
localisable SyncItemError, no raw body), connector_error_reason (error→i18n-code
classification), sync_item_error (legacy-string + structured back-compat) — all
in the normal suite. connector_realapi / connector_seed are an opt-in real-Feishu
harness gated by the feishu_integration build tag AND env vars, never run in CI.
Stop hardcoding DOCREADER_PDF_RENDER_PARALLELISM=4 in compose so low-core
containers use min(4,cpu). Clarify WEKNORA_LANGUAGE precedence and align
docreader JPEG quality docs with the 85 default.
* feat(compose): align env vars in docker-compose and .env.example with code
docker-compose.yml / docker-compose.dev.yml
- app.environment: remove deprecated vars Go app no longer reads
(CRYPTO_MASTER_KEY, CRYPTO_SALT, TENANT_AES_KEY, ENABLE_GRAPH_RAG);
add missing vars the code actually consumes (OSS/S3/TOS/COS completion,
OpenSearch, Tencent VectorDB, Milvus credentials, Redis TLS/namespace,
OIDC, audit/invitation/quota, LLM tuning, doc timeouts, etc.)
- docreader.environment: complete PDF tuning knobs, SSRF/LOG_LEVEL/gRPC
params, GRPC_MTLS_REQUIRE_CLIENT_CERT; fix DOCREADER_PDF_JPEG_QUALITY
default 90 -> 85 to match code (config.py)
- mcp.environment: add MCP_SERVER_AUTH_TOKEN (required by HTTP transport),
WEKNORA_CHAT_TIMEOUT, WEKNORA_VERIFY_SSL, MCP_ALLOWED_UPLOAD_DIRS
- dev compose docreader.environment kept symmetric with prod
.env.example
- reorganize into A-J sections (deploy / data / retrieval / models /
parsing / auth / agent / integrations / observability / security)
- remove the four deprecated vars, document why
- add missing vars, fix NEO4J_URI to bolt:// (matches compose + single-node),
fix DORIS_COMPAT_MODE typo, complete docreader PDF tuning defaults
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(helm): inject NEO4J_ENABLE and drop deprecated TENANT_AES_KEY
helm graph bug fix
- NEO4J_ENABLE is the sole graph switch the Go app reads (since v0.1.6,
ENABLE_GRAPH_RAG is no longer read). helm never injected NEO4J_ENABLE,
so GraphRAG was silently broken: setting neo4j.enabled=true gave the
app NEO4J_URI/USERNAME/PASSWORD but not the switch, and the UI rejected
Node Extractor config with "请正确配置环境变量NEO4J_ENABLE".
- app.yaml: inject NEO4J_ENABLE=true inside the neo4j.enabled block;
remove the dead ENABLE_GRAPH_RAG injection
- NOTES.txt: update GraphRAG instructions (just enable neo4j, no need
to set the deprecated ENABLE_GRAPH_RAG)
- values.yaml: drop ENABLE_GRAPH_RAG, fix neo4j section comment
TENANT_AES_KEY cleanup (deprecated since v0.4.0, encryption now uses
SYSTEM_AES_KEY exclusively; Go app no longer reads it)
- secrets.yaml: stop generating/looking-up/writing TENANT_AES_KEY,
fix stale comment about tenants.api_key rotation
- app.yaml: remove TENANT_AES_KEY secretKeyRef
- values.yaml: drop tenantAesKey, fix existingSecret key list + comments
- cloud-image/firstboot.sh: stop generating/writing/printing TENANT_AES
- cloud-image/README.md: drop TENANT_AES_KEY from credential list
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
The generic OpenAI-compatible reranker hardcoded `truncate_prompt_tokens:
511` on every /rerank request. Providers that honor this vLLM-specific
field (e.g. SiliconFlow serving Qwen3-Reranker) keep only the last 511
tokens of the templated rerank prompt, cutting the instruction and query
off any document longer than ~511 tokens. Every relevance score then
collapses to near zero (~0.0067 instead of ~0.998), falls below the score
threshold, and knowledge-base search returns empty results. A direct curl
without the field returns the correct scores.
WeKnora already removed this exact hardcoded parameter from the embedding
path in #960; the rerank path was never fixed.
- Add `omitempty` to `truncate_prompt_tokens` and `additional_data` so
empty optional fields are not sent.
- Make truncation opt-in via `extra_config.truncate_prompt_tokens`
(validated > 0 at construction), preserving an escape hatch for
self-hosted vLLM deployments with small-context rerankers.
- Add regression tests reproducing the collapsed-score symptom and the
opt-in path.
Fixes#2143
Multiple GORM tag options must be separated by ';', not ','. The comma was
parsed as part of the type value, producing malformed DDL that causes
AutoMigrate(&Message{}) to fail on SQLite with a syntax error.
Restrict channel-managed sessions (API keys, IM, embed) to tenant admin
in the Web console for list/read paths, hide channel sidebar buckets from
non-admins, and add CountSessionsBySource for aggregate stats without
exposing session rows to viewers.
Split oversized candidate sets into concurrent batches instead of failing when retrieval exceeds the VikingDB rerank per-request cap, and align ko/ru locale hints with zh-CN.
Let tenant-wide retrieve keys serve tenant-bounded /files paths while
denying KB-restricted keys, declare knowledge batch write routes for
ingest-capable keys, and short-circuit ownership checks for API-key
principals already authorized by the gate and KB allow-list.
The floating `nginx:stable-alpine` tag drifts with every stable release.
A newer Alpine base (3.24+) fails to start on old hosts such as CentOS 7
(kernel 3.10 + old libseccomp), which broke v0.7.0. Pin to nginx 1.30.3 /
Alpine 3.23.5 by digest, the same base as the working v0.6.3 image.
Add GetOwnedSession method to enforce strict ownership checks for session mutations, ensuring tenant admins can read but not modify API-key sessions. Update relevant handlers to utilize this method for operations that alter session data, maintaining security and integrity of API-key sessions.
Let tenant admins list and open API-key sessions from the Web console while
keeping per-key isolation for everyone else. Pin the session source filter so
it no longer jumps when switching buckets, and resolve session origin on read
so deep links stay in sync with the active folder after refresh.
Use bounded detached contexts for wiki ingest tail cleanup so pending row
settlement, span closure, finalize enqueueing, and follow-up scheduling can
finish even after the asynq task context is cancelled.
Return settlement errors from pending row trimming and failed-op requeueing,
and only mark claims settled after rows are deleted, released, or
dead-lettered successfully. Requeue documents whose reduce updates were not
applied after context cancellation to avoid silently dropping wiki work.
Add unit coverage for detached cleanup contexts and settlement error handling.
Add a new 'is-docked' class to the ChatHeader component for improved styling when the references panel is visible. Introduce a 'hasReferencesPanel' prop to manage this state, enhancing the layout and responsiveness of the chat header on wider screens.
Finish-time metadata was discarded and implicit root traces opened by
StartSpan/StartGeneration were never ended, leaving orphaned parents in
Langfuse. Also stop mutating the global OTel text-map propagator.
Migrate WeKnora's Langfuse tracing from the legacy /api/public/ingestion
batch API to the OpenTelemetry Go SDK emitting OTLP/HTTP to
/api/public/otel/v1/traces — the Langfuse v3+ standard ingestion protocol.
Why: Langfuse v3 deprecated the legacy ingestion API in favor of OTLP, and
OTel-only backends (e.g. the LiteFuse Doris fork) dropped it entirely, so the
legacy client was incompatible. Migrating to the OTel SDK also adds W3C
traceparent cross-service propagation: an upstream caller's trace id is
inherited so its run and the WeKnora call land under one trace in the UI.
What:
- Manager owns an OTel TracerProvider backed by an otlptracehttp exporter
(Basic auth + the x-langfuse-ingestion-version gate header). Spans are
buffered by BatchSpanProcessor and exported complete on End, so a long
trace no longer fragments across flush batches.
- *Trace/*Span/*Generation wrap OTel spans; the public API (StartTrace/
Span/Generation, handles, InjectTracing, asynq/gin middleware, Config,
TokenUsage) is unchanged, so callers (agent/chat/handler/scheduler/asynqdl)
need no edits.
- W3C traceparent: GinMiddleware extracts traceparent from incoming requests;
AsynqMiddleware + InjectTracing propagate it across the asynq payload.
types.TracingContext gains a LangfuseTraceparent field.
- logger.CloneContext preserves the active OTel span across context rebuilds
so child spans (agent engine, summary generation) stitch to the HTTP trace
instead of orphaning into separate traces.
- StartSpan/StartGeneration re-establish the parent from the *Trace handle
when the ctx lost the active OTel span, covering background/derived contexts.
Compatibility: works with Langfuse v3+ (cloud / self-hosted / the bundled
docker-compose profile) and LiteFuse. Drops Langfuse v2 (legacy-only) support.
Uses the OTel Go SDK already (indirectly) in go.mod, promoting
otlptracehttp to direct; no third-party langfuse-go library.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Give chat a left-aligned header for rename/pin/copy/clear/delete, share session mutations across sidebar and chat view, and support copying the full conversation as Markdown.
Publish scoped API keys, worker-pool governance, multi-instance storage,
and the rest of the v0.7.0 feature set across CHANGELOG, READMEs, QA,
Swagger, and version pins.
zhangzherui