- Updated the sanitization logic in `newStreamPacketDumper` to exclude the '.' character, enhancing security against path traversal vulnerabilities.
- Added checks to ensure the cleaned file path remains within the specified directory, preventing unauthorized access to the filesystem.
- Improved overall robustness of file handling by enforcing stricter path validation rules.
- Added a new thinking mode control field in the ModelEditorDialog, allowing users to specify how the agent's thinking mode is formatted for API requests.
- Introduced localization strings for thinking mode labels and descriptions in English, Korean, Russian, and Chinese, enhancing user experience across multiple languages.
- Updated the model form data structure to include thinking control settings, ensuring proper handling during model edits and saves.
- Implemented utility functions for resolving and defaulting thinking control values based on provider and model, improving backend integration.
- Added tests for the new thinking control functionality to ensure reliability and correctness in handling user configurations.
- Added support for Tencent Cloud LKEAP Rerank, including new configuration options for SecretId and SecretKey.
- Updated frontend components to handle LKEAP-specific fields and enhance user experience with appropriate labels and placeholders.
- Enhanced backend logic to manage LKEAP credentials and integrate with the new rerank API.
- Improved localization for LKEAP-related terms across multiple languages, ensuring clarity and consistency in user-facing messages.
- Introduced tests for LKEAP Reranker functionality to ensure reliability and correctness in handling requests.
- Eliminated all instances of the final_answer tool from the agent's configuration and logic, transitioning to a model where the agent concludes responses with plain text instead.
- Updated event handling and response analysis to reflect this change, ensuring that the agent's output is now directly delivered as text without relying on a dedicated final answer tool.
- Adjusted localization strings across multiple languages to remove references to final_answer, enhancing clarity and consistency in user-facing messages.
- Improved the overall structure of the agent's response handling to streamline the process and reduce potential confusion regarding tool usage.
refactor(agent): streamline answer handling and preamble logic
- Updated the agent's response handling to treat any text streamed before a tool call as a preamble, ensuring it is not included in the final answer.
- Removed the superseded marker logic from the event emission process, simplifying the flow of answer content and enhancing clarity in the UI.
- Adjusted the event handling to ensure that preamble segments are correctly retracted and relocated into the steps tree, improving the overall user experience.
- Enhanced the agent's internal logic to better manage answer segments and their states during streaming, leading to a more coherent response structure.
- Introduced a new test file for the Langfuse wrapper, implementing tests for the `buildLangfuseGenerationOutput` and `buildLangfuseMessages` functions.
- Validated the output structure for various scenarios, ensuring correct handling of content, tool calls, finish reasons, and reasoning content.
- Enhanced test coverage for the Langfuse functionality, improving reliability and maintainability of the codebase.
Large scanned PDFs (hundreds of pages) were slow and fragile end-to-end.
This change addresses the parse, transport, and task-scheduling layers:
docreader (parse + transport):
- Parallelize per-page scanned rendering across processes (forkserver/fork),
with serial fallback. ~4-7x faster on large scanned PDFs; pdfium is not
thread-safe so we fan out across processes. Configurable via
DOCREADER_PDF_RENDER_PARALLELISM.
- Add server-streaming ReadStream RPC: emit one meta frame then one frame per
image, so documents with many page images are no longer capped by the unary
gRPC message-size limit (a 874-page PDF produced ~193MiB of images, far over
the 50MB cap) and memory is bounded on both ends. Unary Read is kept for
backward compatibility; the Go production reader switches to ReadStream.
VLM:
- Make the VLM HTTP timeout configurable (VLM_HTTP_TIMEOUT_SECONDS) and raise
the default 90s -> 180s so dense scanned-page OCR does not time out with
"context deadline exceeded".
Async task queues:
- Isolate high-volume, model-heavy fan-out tasks into dedicated asynq queues so
a single large document cannot saturate the shared worker pool and block
user-facing document parsing:
image:multimodal -> "multimodal"
chunk:extract -> "graph"
question:generation -> "question"
- Register the new queues in the server weight map and the cancel inspector's
scanned-queue set (so cancelling a knowledge still purges its pending tasks).
- Add MiniMax-M3 to the MiniMax provider description across i18n locales
- Remove deprecated MiniMax-M2.5 reference
- Retain MiniMax-M2.7 and MiniMax-M2.7-highspeed as supported models
MiniMax-M3 is the latest flagship model from MiniMax with extended context
and stronger reasoning capabilities. Users can now see it in the provider
selection UI when configuring a new Minimax model.
- Added a LocalImageResolver function to resolve local storage URLs to their byte data using the tenant's storage configuration.
- Integrated the resolver into the container setup to support reading local images in multimodal chat scenarios.
- Enhanced the handling of local storage paths to correctly parse tenant IDs and retrieve files based on the configured storage settings.
`cached_tokens` is reported by every OpenAI-compatible provider that
supports prompt caching, but how it becomes non-zero differs by mode:
- Implicit caching (OpenAI, Azure OpenAI, DeepSeek, …) populates the
field automatically whenever a prompt prefix matches a previous
request within the provider's cache TTL. No client-side opt-in.
- Explicit caching (Qwen on Aliyun, Anthropic Claude, …) only
populates the field after the caller attaches `cache_control:
{"type": "ephemeral"}` to the relevant message / content block.
Until that opt-in is applied upstream of the request, the field
stays zero even when the prefix is otherwise byte-stable.
Without this distinction documented, the previous commit reads as if
`TokenUsage.CachedTokens` will show non-zero values for Qwen / Claude
once this PR lands — which is not the case. The plumbing here is a
prerequisite (stable prefix via sorted tools) and a meter (visibility
of the field), but the explicit-cache opt-in itself is out of scope
and lives elsewhere.
Document this on `TokenUsage.CachedTokens` and the `cachedTokens`
helper so callers do not mistake observability for activation.
OpenAI-compatible providers (Qwen, DeepSeek, OpenAI, Azure, etc.) report
prompt-cache hits in `usage.prompt_tokens_details.cached_tokens`. This
value was being read by go-openai but dropped at the WeKnora boundary,
so there was no way to tell whether prompt caching was actually working.
This change plumbs the field end-to-end:
- `types.TokenUsage.CachedTokens` (json:"cached_tokens,omitempty") —
zero-values are omitted so payloads stay quiet for providers that
never report cache hits.
- `cachedTokens` helper in remote_api.go guards against nil
`PromptTokensDetails` (Ollama and older OpenAI-compat backends omit
the details block entirely).
- All three response-parsing paths populate it:
* `parseCompletionResponse` (non-streaming)
* `processStream` (SDK streaming)
* `processRawHTTPStream` (raw-HTTP streaming, used when callers
need to inject custom fields like `cache_control`)
- The five `[LLM Usage]` log lines now print `cached_tokens=%d` so
cache hit rate is visible in `journalctl` / log tail without going
through metrics.
Together with the deterministic tool ordering from the previous commit,
this makes Qwen explicit caching observable: a warmed prefix should
show `cached_tokens` ≈ system + tools token count (typically several
thousand) on subsequent requests within the 5-minute TTL.
Tests:
- `TestCachedTokensHelper` — nil safety + round-trip
- `TestParseCompletionResponse_CachedTokens` — populated + missing
details paths through `parseCompletionResponse`
- `TestTokenUsage_CachedTokensJSONOmitempty` — zero is omitted,
non-zero is emitted
MiMo and DeepSeek V3.2/V4 reject multi-turn requests in thinking mode
when the prior assistant message lacks reasoning_content with HTTP 400:
"The reasoning_content in the thinking mode must be passed back to the API."
Agent's ReAct loop is the worst-case scenario — every round produces tool
calls, exactly the case DeepSeek's docs specify reasoning_content MUST
participate in subsequent context.
Plumb reasoning_content through the full assistant-message round-trip:
- chat.Message / types.ChatResponse / types.AgentStep gain a
reasoning_content field (AgentStep persists via the existing
Message.AgentSteps jsonb column, no migration needed).
- streamLLMToEventBus accumulates reasoning chunks into
result.ReasoningContent and surfaces it on the round's ChatResponse.
- engine.runReActIteration writes it onto AgentStep so cross-turn replay
preserves it.
- observe.appendToolResults attaches it to the same-turn assistant
message; agent_history.buildAssistantHistoryMessages does the same on
cross-turn replay.
- RemoteAPIChat.ConvertMessages forwards it on assistant turns to
openai.ChatCompletionMessage.ReasoningContent (already supported by
go-openai); providers that don't recognize the field ignore it.
Tests cover the three boundaries: ConvertMessages serializes it for
upstream, appendToolResults preserves it within the same turn, and
buildAssistantHistoryMessages replays it across turns.
Scope is intentionally limited to Agent mode — KnowledgeQA's chat
pipeline and Anthropic's signed thinking_blocks are separate fixes that
require schema changes (rendered_content / thinking_blocks columns).
Fixes#1302
OpenAI's GPT-5 series and o-series reasoning models (o1/o3/o4-mini) no
longer accept `max_tokens` and reject non-default sampling params
(`temperature`, `top_p`, `frequency_penalty`, `presence_penalty`).
Azure OpenAI propagates the same constraint, returning HTTP 400 with
"this model is not supported MaxTokens, please use MaxCompletionTokens".
Lower the compatibility shim into `RemoteAPIChat.BuildChatCompletionRequest`
(the single OpenAI-protocol egress) so every internal caller keeps using
`MaxTokens` uniformly:
- Add `provider.IsOpenAIReasoningOrGPT5Model` with precise prefix matching
(covers `gpt-5*`, `o1`/`o1-*`, `o3`/`o3-*`, `o4`/`o4-*`; rejects
`olympus-1`, `openai-*`, `o3xtra`, etc.).
- When the provider is `openai`/`azure_openai` and the model matches, map
`MaxTokens` to `MaxCompletionTokens` (explicit `MaxCompletionTokens`
wins) and skip the unsupported sampling fields; `omitempty` keeps them
off the wire.
- Behavior is unchanged for gpt-4o, gpt-4, and all other providers.
Add unit tests for the matcher and for the request-building path
(Azure gpt-5.2, OpenAI gpt-5/o1-mini/o3/o4-mini, plus negative cases).
Fixes#1283
The ollama Go SDK changed ToolCallFunctionArguments from
map[string]any to a struct backed by an ordered map. Use ToMap() for
read access and NewToolCallFunctionArguments()+UnmarshalJSON for
construction so the build matches the upgraded SDK.
Add per-stream once-only logs at the OpenAI-protocol layer so we can
triage streaming behavior (natural-stop vs tool-call, TTFC, ordering of
reasoning/content/tool_calls) without grepping through every delta.
streamState now tracks:
- streamStartedAt : baseline for elapsed_ms on each fire-once log
- firstContentSeen : delta.Content first appearance
- firstReasoningSeen : reasoning_content first appearance
- firstToolCallSeen / noToolCallStopLogged (existing flags retained)
Logs emitted at most once per stream:
[LLM Stream] First reasoning_content at OpenAI layer (len, preview, elapsed_ms)
[LLM Stream] First delta.Content at OpenAI layer (len, preview, tool_call_seen, thinking_seen, elapsed_ms)
[LLM Stream] First tool_calls delta at OpenAI layer (count, first_id, first_name, first_content_seen, thinking_seen, elapsed_ms)
[LLM Stream] Natural-stop at OpenAI layer (finish=stop, tool_calls field never observed, thinking_seen, first_content_seen, elapsed_ms)
Together with the existing agent-layer "Natural-stop candidate detected"
log this lets a single grep reconstruct the temporal layout of one stream
(reasoning -> content -> tool_call OR natural-stop) and tell apart:
(A) tool_calls field truly absent (real natural-stop)
(B) tool_calls field arrived but high-level marker not yet emitted
Pure logging change; no behavior change. Previews use the existing
truncateForDebug helper (rune-safe, capped at 80 chars).
Reproduce:
1. Run WeKnora pointing OPENAI_BASE_URL at any OpenAI-compatible
embedding endpoint.
2. Stop that endpoint.
3. Issue any RAG-mode chat / knowledge_search query — the pipeline
reaches chunk_search_parallel → knowledgeBaseService.GetQueryEmbedding
→ OpenAIEmbedder.Embed → OpenAIEmbedder.BatchEmbed.
4. Observe the entire WeKnora process SIGSEGV instead of returning a
graceful 5xx.
Stack (real, observed in production):
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x40]
internal/models/embedding/openai.go:195 +0x458 BatchEmbed
internal/models/embedding/openai.go:93 +0xb0 Embed
internal/application/service/knowledgebase_search.go:35
GetQueryEmbedding
internal/application/service/chat_pipeline/search.go:371
searchByTargets.func1
Root cause — variable shadowing in doRequestWithRetry:
func (e *OpenAIEmbedder) doRequestWithRetry(...) (*http.Response, error) {
var resp *http.Response
var err error // outer err
...
for i := 0; i <= e.maxRetries; i++ {
...
req, err := http.NewRequestWithContext(...) // ← `:=` shadows
// outer `err`
// because `req`
// is new in
// the loop scope
if err != nil { ... continue }
...
resp, err = e.httpClient.Do(req) // writes to the
// shadowed
// loop-local err
if err == nil { return resp, nil }
logger.Errorf("attempt %d failed: %v", i+1, err) // logs the inner err
}
return nil, err // returns OUTER
// err — never set
}
When httpClient.Do returns connection errors on every attempt, the loop
logs them faithfully but the outer err stays nil. The function returns
(nil, nil). Caller in BatchEmbed reads:
resp, err := e.doRequestWithRetry(ctx, jsonData)
if err != nil { return nil, ... } // skipped (err is nil)
if resp.Body != nil { // ← line 195 · resp itself is nil
defer resp.Body.Close() // nil pointer dereference
} // → SIGSEGV → process dies
Fix — declare `req` separately so the shared `err` variable is not shadowed:
var req *http.Request
req, err = http.NewRequestWithContext(...) // outer err
After the fix, when all retries fail, doRequestWithRetry returns the real
error (e.g. `dial tcp 127.0.0.1:11434: connect: connection refused`),
the caller propagates it as a regular error, the chat pipeline returns a
5xx to the client, and the process keeps running.
Verified by stopping the embedding upstream while WeKnora was serving
queries · process stayed up · returned graceful errors · resumed normal
operation when the upstream came back. Without the patch the same
sequence killed the process within one query.
Closes#620#497. Add opt-in Langfuse observability covering all five
model types (chat, embedding, rerank, VLM, ASR) with HTTP-request-scoped
traces and Docker Compose support (both cloud and self-hosted).
Core package internal/tracing/langfuse:
- HTTP client with batched async ingestion (non-blocking in request path)
- Sampling, environment / release tagging, and graceful fallback when
LANGFUSE_* env vars are absent (wrappers become no-ops)
- Gin middleware opens one trace per traced request and finishes it after
the handler chain returns, attaching method / path / user / session
- Trace context is stored under a typed key exported from internal/types
so logger.CloneContext can preserve it across handler / goroutine
boundaries (otherwise each LLM call auto-created an orphan trace,
fragmenting one request into many)
Per-model generation wrappers (opt-in via NewChat/NewEmbedder/...):
- chat: captures prompt, streaming output, token usage + TTFT
- embedding: approximates tokens when the provider omits usage
- rerank: previews query/docs, summarizes results to keep payload small
- vlm: records image count and total bytes, never uploads raw pixels
- asr: records file size and audio duration, never uploads audio bytes
Async title generation (GenerateTitleAsync) now forwards the trace key
into the goroutine so title calls appear under the parent chat trace.
Docker Compose:
- LANGFUSE_* env passthrough on the `app` service for cloud deployments
- Optional `langfuse` profile spins up a self-hosted Langfuse stack that
reuses WeKnora's existing PostgreSQL (separate database via an idempotent
init container that fixes ICU collation drift) and Redis (separate DB
number), adding only ClickHouse, MinIO, web and worker containers
- web/worker entrypoints URL-encode DB_PASSWORD / REDIS_PASSWORD at start
to avoid Prisma P1013 when passwords contain @ / # / etc.
Docs: docs/Langfuse集成.md covers cloud vs self-hosted, per-model usage
strategy, code map, and resource footprint.
Introduce per-model `custom_headers` config (similar to OpenAI Python SDK's
`extra_headers`) so users can inject gateway auth tokens, trace IDs, etc.
into every outbound API request for chat / embedding / rerank / VLLM / ASR
models. Reserved headers like Authorization / Content-Type are always
preserved to avoid breaking auth or signing flows.
Along the way, unify production and "test connection" paths onto a single
`ConfigFromModel(*types.Model, appID, appSecret)` constructor in each
model package. Both `service.modelService.GetXxxModel` and the four
`handler.initialization.Check*Model` / `TestEmbeddingModel` endpoints now
go through the same field mapping, so new parameters only need to be
added in one place.
Backend:
- Add `CustomHeaders map[string]string` to `types.ModelParameters` with
JSON/YAML `omitempty` tags for forward compatibility.
- New `internal/utils/extraheaders.go` providing `ApplyCustomHeaders`
(for hand-rolled HTTP paths) and `WrapHTTPClientWithHeaders` /
`CustomHeadersRoundTripper` (for SDK paths like go-openai). Reserved
headers (Authorization, api-key, Content-Type, Accept, Host,
Content-Length, User-Agent) are filtered out.
- Inject headers in every remote model path: chat (SDK + raw HTTP),
embedding (openai/aliyun/jina/volcengine/nvidia/azure_openai), rerank
(remote_api/aliyun/jina/nvidia/zhipu — via shared `customHeaderSetter`
interface), VLM and ASR (via http.Client wrapping).
- Add `ConfigFromModel` to each of chat/embedding/rerank/vlm/asr,
consolidating field mapping (ExtraConfig, CustomHeaders, InterfaceType
defaulting, WeKnoraCloud credentials, etc.) and cover it with
dedicated unit tests per package.
- Refactor `service/model.go`: replace ~100 lines of hand-written Config
literals with one-line `ConfigFromModel` calls; drop the now-unused
`stringMapToAnyMap` helper.
- Refactor `handler/initialization.go` test-connection endpoints: merge
four ad-hoc request structs into one `ModelTestRequest`, extract
`buildTestModel` and `resolveTenantWeKnoraCloudCreds` helpers, and
route all four endpoints through `ConfigFromModel` + `NewXxx` so the
test path is now behaviorally identical to production.
Frontend:
- Add `custom_headers` to the `ModelConfig` API type.
- `ModelEditorDialog.vue`: new Key-Value header editor (add/remove rows),
auto-converts map <-> array when loading/saving, and passes the
current header set to the `Test Connection` button so the preview
reflects exactly what production will send.
- `ModelSettings.vue`: serialize the header array back into a map for
the backend (drops empty rows).
- i18n: add custom-header labels / descriptions / placeholders to
zh-CN, en-US, ru-RU, ko-KR.
Tests:
- `internal/utils/extraheaders_test.go` covers reserved-header filtering
and round-tripper wrapping (including nil-client fallback).
- New `config_from_model_test.go` in chat / embedding / rerank / vlm /
asr verifies all fields (CustomHeaders, ExtraConfig, InterfaceType
defaulting, AppID/AppSecret) are propagated end-to-end.
- Removed redundant struct definitions for WeKnoraCloud requests and messages, streamlining the customization process.
- Updated the weKnoraCloudRequestCustomizer function to directly modify the request object, enhancing clarity and reducing complexity.
- Improved the convertToWeKnoraCloudMessagesFromOpenAI function to maintain the original message structure while ensuring compatibility with WeKnoraCloud requirements.
- Removed header printing from the chatWithRawHTTP method to streamline logging.
- Updated log message to focus on endpoint and model name, enhancing clarity while reducing verbosity.
- Enhanced the logging of the tenant access token in the Feishu client to handle variable lengths for prefix and suffix, ensuring accurate display even for shorter tokens.
- Updated the test cases in `connector_test.go` to reflect changes in the endpoint path for file downloads, improving test reliability.
- Adjusted example outputs in `example_test.go` for clarity and consistency.
- Modified image resolution tests to correct expected outcomes based on updated criteria for icon images.
- Added environment variable settings in image resolver tests to ensure localhost accessibility during testing.
These changes enhance the robustness of logging and testing mechanisms across the application.
- Added a new logger for LLM calls, enabling detailed logging of model interactions, including request and response data.
- Introduced configuration options for enabling and specifying the log directory for LLM debug logs.
- Implemented cleanup for old log files to manage disk space effectively.
- Wrapped existing chat implementations to log calls when debug logging is enabled, enhancing traceability for model interactions.
- Added WeKnoraCloudVLM struct to handle interactions with the WeKnoraCloud API, including image and text processing capabilities.
- Enhanced model configuration to include AppID and AppSecret for secure API access.
- Updated modelService to resolve WeKnoraCloud credentials and pass them to the VLM configuration.
- Implemented Predict method for sending requests to the WeKnoraCloud API and processing responses.
This update introduces a new integration with WeKnoraCloud, expanding the application's capabilities in handling multimedia content.
- Removed support for video file uploads across the application, including updates to the knowledge base and document processing services.
- Updated internationalization files to reflect the removal of video-related terminology, changing labels and descriptions to focus solely on audio processing.
- Adjusted file type handling to exclude video formats, ensuring users receive accurate feedback regarding supported file types.
- Enhanced error handling to provide clear messages when users attempt to upload unsupported video files.
This update streamlines the application by focusing on audio processing capabilities and improving user experience through clearer messaging.
- Changed 'Audio Processing' to 'Audio & Video' in English, Korean, Russian, and Chinese translations to reflect expanded functionality.
- Updated ASR configuration titles and descriptions to include video processing capabilities, enhancing user understanding of the feature.
- Renamed file type labels from 'Audio Files' to 'Audio & Video' across multiple languages for consistency and clarity.
- Enhanced document upload handling in the KnowledgeBase to filter unsupported file types, providing user feedback for filtered images, videos, and audio files.
This update improves the internationalization of the application and clarifies the capabilities of the ASR feature.
- Introduced a new WeKnoraCloudSettings component for managing APPID and APPSECRET.
- Added functionality to save WeKnoraCloud credentials without automatically creating models.
- Implemented credential status checks across various components to provide user feedback on configuration status.
- Updated API endpoints to handle credential saving and status retrieval.
- Enhanced UI elements to guide users in configuring WeKnoraCloud settings.
This update improves the integration of WeKnoraCloud by allowing users to manage their credentials more effectively and receive real-time feedback on their configuration status.
Remove the hardcoded TruncatePromptTokens: 512 limit from WeKnoraCloud
BatchEmbed to allow the API to use its default truncation behavior,
avoiding unexpected truncation of longer input texts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove infrastructure/crypto package (CryptoService, config, PBKDF2-based encryption)
- Replace with lightweight utils.GetAESKey/EncryptAESGCM/DecryptAESGCM helpers
- Remove cryptoSvc dependency from modelService, tenantService, weKnoraCloudService
- Remove crypto state persistence logic from container initialization
- Add TruncatePromptTokens field to WeKnoraCloud embed request
- Update frontend hint text for WeKnoraCloud credential errors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
wizardchen