Commit Graph

338 Commits

Author SHA1 Message Date
ochan.kwon 5df0ea2de2 feat(knowledge): resolve remote images in file upload path
The file upload pipeline (ProcessDocument) only called ResolveAndStore(),
which handles inline bytes and base64 data URIs but explicitly skips
http/https URLs. This meant markdown files with external image URLs
(e.g. GitHub issue screenshots) were never downloaded, stored, or
processed by VLM multimodal.

Add ResolveRemoteImages() call after ResolveAndStore() in the file
upload path, matching the existing manual processing path behavior.
2026-03-30 20:30:43 +08:00
ochan.kwon 5c0cda3630 fix(storage): add S3 provider scheme support to frontend and backend allowlists
The S3 storage adapter (s3.go) was added in e309e0b but the provider
scheme integration was incomplete — ParseProviderScheme() and frontend
security.ts were not updated, causing s3:// URLs to fail in the /files
proxy, VLM image processing, and frontend rendering.

Additionally, S3-compatible services (MinIO, Ceph, etc.) require
path-style addressing (endpoint/bucket/key) but the SDK client only
used virtual-hosted style. The deprecated WithEndpointResolver API
is replaced with BaseEndpoint, and path-style is auto-enabled for
non-AWS endpoints (detected by amazonaws.com in the URL).

Changes:
- types/knowledgebase.go: add "s3" to ParseProviderScheme()
- file/s3.go: auto path-style for S3-compatible endpoints + replace
  deprecated WithEndpointResolver with BaseEndpoint
- frontend/security.ts: add s3 to 5 provider scheme allowlists
- knowledgebase_test.go: add ParseProviderScheme + InferStorage tests
- s3_test.go: add path-style detection + parseS3FilePath tests
2026-03-30 16:57:27 +08:00
wizardchen 1a954099f0 feat(knowledge): add SQL LIKE keyword escaping and metadata search functionality
- Introduced `escapeLikeKeyword` function to escape SQL LIKE wildcards in search queries, ensuring accurate keyword matching.
- Updated search queries to utilize the new escaping function for filtering by file name and title.
- Added `FindByMetadataKey` method to retrieve knowledge items based on key-value pairs in the metadata JSON column, enhancing search capabilities.
2026-03-27 16:30:07 +08:00
wizardchen 72dfb9ce75 feat(prompt): enhance context and intent classification in templates
- Updated context template to include runtime metadata such as current time and week, improving contextual awareness in user queries.
- Enhanced rewrite template with critical instructions for intent classification, ensuring that rewritten questions preserve essential entities and keywords.
- Refined intent classification logic to prioritize user intents more effectively, improving the accuracy of responses based on user queries.
- Added examples to clarify expected input and output formats for intent classification, enhancing usability for developers.
2026-03-26 18:57:12 +08:00
ChenRussell 27d4525ba9 fix(app): prevent UTF-8 truncation in summary fallback
When LLM summary generation fails, the fallback logic truncates
the first chunk content to 500 characters. Previously used byte-based
slicing (summary[:500]) which could cut multi-byte UTF-8 characters
in half, causing PostgreSQL to reject invalid byte sequences.

Changed to rune-based truncation to ensure valid UTF-8 after truncation.

Fixes: Documents stuck in "Generating summary" status when containing
Chinese/emoji characters and summary generation fails.
2026-03-26 18:52:08 +08:00
wizardchen 23290efcec feat(image-multimodal): add task enqueuing for question generation based on image knowledge
- Introduced a new task enqueuer in the ImageMultimodalService to handle question generation for image knowledge.
- Implemented logic to enqueue question generation tasks when valid captions or OCR content are available, enhancing the knowledge base's interactivity.
- Updated image reference handling to ensure markdown URLs are correctly formatted, accommodating filenames with spaces.
- Enhanced regex pattern in image resolver to support filenames with spaces, improving image processing accuracy.
2026-03-25 22:08:29 +08:00
wizardchen 1960fc2d90 feat(suggested-questions): implement suggested questions feature for chat interface
- Added a new API endpoint to fetch suggested questions based on the agent's associated knowledge bases.
- Implemented frontend components to display suggested questions in both chat and create chat views, enhancing user interaction.
- Updated localization files to include translations for suggested questions in English, Korean, Russian, and Chinese.
- Introduced debounce logic for fetching suggestions to optimize performance during knowledge base changes.
- Enhanced backend services to support fetching and returning suggested questions, improving the overall chat experience.
2026-03-25 22:08:29 +08:00
wizardchen ae89871ac5 feat(knowledge): enhance search functionality to include title filtering and URL handling
- Updated search queries to filter by both file name and title when a keyword is provided, improving search accuracy.
- Added support for filtering by URL and HTML file types in search queries, enhancing the flexibility of the search functionality.
- Refactored query construction for file types to streamline the inclusion of additional conditions, ensuring robust handling of various file types.
2026-03-25 22:08:29 +08:00
wizardchen 3daaabfc4a refactor(elasticsearch): remove ".keyword" suffix from ID fields in Elasticsearch queries
- Updated Elasticsearch repository methods to use ID fields without the ".keyword" suffix, aligning with the new index mapping requirements.
- Enhanced documentation for the VectorEmbedding struct to clarify expected index mapping and usage guidelines for ID fields.
- Ensured consistency across all repository methods that interact with ID fields, improving query performance and maintainability.
2026-03-25 22:08:29 +08:00
wizardchen 5245475e0f feat(knowledge): add channel support to knowledge entries and related components
- Introduced a new `channel` field in the Knowledge struct and associated request types to track the source channel (e.g., "web", "api", "browser_extension").
- Updated various frontend components to display channel information and enhance user experience with channel labels.
- Enhanced localization files to support channel labels in English and Chinese.
- Modified backend services and database migrations to accommodate the new channel feature, ensuring consistent tracking across knowledge entries.
- Refactored related functions to integrate channel handling, improving overall knowledge management and context.
2026-03-25 22:08:29 +08:00
wizardchen a8924e301e feat(agent): enhance message consolidation and context management
- Improved the message consolidation logic to preserve the current turn's user query and all subsequent assistant/tool messages, ensuring better context retention.
- Updated the CompressContext function to reflect the new consolidation strategy, maintaining the system prompt and relevant recent messages.
- Refactored the context manager to support optional message repository for improved context rebuilding from persistent storage.
- Added comprehensive tests to validate the new consolidation behavior and ensure correct message handling across various scenarios.
2026-03-25 22:08:29 +08:00
wizardchen 2b4e614dc2 refactor(agent): enhanced the checkFAQQuestionDuplicate function in knowledge.go by adding structured comments to outline the validation steps for FAQ questions, improving readability and maintainability. 2026-03-25 22:08:29 +08:00
wizardchen 3e3ffa4b0b feat(i18n): add JSON file type support across multiple locales
- Introduced support for JSON file types in English, Korean, Russian, and Chinese locale files.
- Updated the file type acceptance criteria in the document upload component to include JSON.
- Enhanced the knowledge base parser settings to recognize JSON as a valid file type.
- Implemented JSON conversion functionality in the document parser, allowing for JSON content to be processed and converted to markdown.
- Added comprehensive tests for JSON to markdown conversion to ensure functionality and reliability.
2026-03-25 22:08:29 +08:00
wizardchen 649c9d3dc9 refactor(chat_pipeline): improve regex patterns for table parsing
- Updated regex patterns for table separator and row matching to use [ \t] instead of \s, preventing newline consumption across rows.
- Enhanced comments for clarity on the changes made to the regex expressions.
2026-03-25 22:08:29 +08:00
wizardchen 47397464ef refactor(chat_pipeline): enhance merging logic and add partial overlap removal
- Updated the merging process in the chat pipeline to include a new step for re-merging overlapping ranges introduced by context expansion.
- Implemented a new function, removePartialOverlaps, to eliminate chunks that are largely contained within higher-scored chunks, improving deduplication accuracy.
- Enhanced normalization and overlap checking utilities to support the new deduplication logic.
2026-03-25 22:08:29 +08:00
wizardchen 68dd6aa778 refactor(knowledge): increase page size for tag retrieval in buildTagMap function
- Updated the page size from 100 to 1000 in the buildTagMap function to enhance performance and efficiency when retrieving tags from the knowledge base.
2026-03-25 22:08:29 +08:00
wizardchen a6c38123b6 refactor(knowledge): optimize tag retrieval in buildTagMap function
- Changed the tag retrieval process to support pagination, allowing for more efficient handling of large sets of tags.
- Adjusted the page size to 100 for better performance and reduced memory usage.
- Ensured that the function continues fetching tags until all are retrieved, improving the completeness of the tag map.
2026-03-25 22:08:29 +08:00
wizardchen f368b987bb feat(agent): refactor agent engine creation and enhance MCP tool registration
- Improved the CreateAgentEngine method by breaking down the process into distinct steps for better readability and maintainability.
- Added a new registerMCPTools method to streamline the registration of MCP tools based on the agent configuration.
- Enhanced error handling and logging for MCP service registration, ensuring clearer feedback during the process.
- Introduced resolveKBAndDocInfos to load knowledge base metadata and selected document information for prompt generation.
2026-03-25 22:08:29 +08:00
wizardchen b02e3f1dbb feat(knowledge): implement knowledge item rebuild confirmation dialog and enhance chunk processing options
- Added a confirmation dialog for rebuilding knowledge items in the KnowledgeBase component.
- Introduced new state variables to manage the rebuild process and item selection.
- Enhanced the processChunks function to support question generation options based on configuration.
- Updated the knowledge service to handle question generation settings during document processing.
2026-03-25 22:08:29 +08:00
wizardchen 0e01a8ca64 feat(server): enhance server startup and shutdown process
- Added support for SO_REUSEPORT in the listenWithRetry function to improve port binding during hot-reloads.
- Implemented graceful shutdown by closing the listener immediately upon receiving a shutdown signal, allowing for quicker port release.
- Updated logging to provide clearer feedback during server shutdown and error handling.
2026-03-25 22:08:29 +08:00
wizardchen 052be53c42 feat(prompts): update assistant descriptions and add intent-specific templates
- Updated the assistant descriptions in various prompt templates to specify that they are developed by Tencent.
- Added new intent-specific system prompt templates for greeting, chitchat, follow-up, image analysis, summarization, and web search unavailability scenarios.
- Enhanced citation rules in the system prompts to ensure proper inline citation formatting.
- Refactored the rewrite prompt to classify user intent more accurately and output structured JSON responses.
2026-03-25 22:08:29 +08:00
wizardchen 0d4ceb6c96 feat(image): enhance image resolution handling for data URIs and remote images
- Added functionality to resolve embedded data:image;base64 images in markdown, storing them and replacing references with stable URLs.
- Updated the knowledge service to process data URIs before handling remote images, ensuring all images are resolved consistently.
- Improved logging for resolved images, providing better insights during manual processing.
- Added unit tests to validate the new data URI resolution functionality and ensure reliability.
2026-03-25 22:08:29 +08:00
wizardchen 9650188a35 feat(chunk): add FindFAQChunkWithDuplicateQuestion method for enhanced duplicate question checking
- Implemented a new method in the chunk repository to find FAQ chunks with overlapping standard or similar questions.
- Updated the knowledge service to utilize this method for checking duplicates during FAQ question creation.
- Improved error handling for duplicate questions, providing clearer feedback on conflicts.
2026-03-25 22:08:29 +08:00
wizardchen 59d2f6b54c feat(graph): enhance graph extraction prompt rendering and language context handling
- Added a new method to render graph extraction prompts with shared placeholders for language.
- Updated entity and relationship extraction methods to utilize the new rendering function, improving prompt customization.
- Enhanced question generation task to include language context, ensuring prompts are generated in the correct language.
- Improved error handling for empty prompt configurations in question generation, enhancing robustness.
2026-03-25 22:08:29 +08:00
wizardchen e138e0c81a refactor(agent): remove query display from DatabaseQuery component and update tool result structure
- Removed the query display section from the DatabaseQuery component to streamline the UI.
- Updated the DatabaseQueryData interface to eliminate the query field, reflecting changes in tool result handling.
- Enhanced the DatabaseQuery tool execution to focus on returning structured results without exposing raw SQL queries.
- Added a new utility to strip <think> blocks from LLM outputs, improving content clarity and user experience.
2026-03-25 22:08:29 +08:00
wizardchen 0851d6d656 feat(agent): implement memory consolidation and context management
- Introduced a memory consolidator to summarize older messages when the context window exceeds a specified limit, preserving key facts and tool results.
- Enhanced the AgentEngine to utilize the new token estimator and memory consolidator for improved context window management.
- Implemented logic to compress message history when approaching token limits, ensuring efficient use of context.
- Added unit tests for the memory consolidator and token estimator to validate functionality and performance.
- Updated tool execution to validate parameters against JSON schema before execution, improving error handling and robustness.
2026-03-25 22:08:29 +08:00
wizardchen e936e0b347 feat(agent): implement LLM call timeout and transient error handling
- Introduced a configurable LLM call timeout with a default value, allowing for better control over LLM call durations.
- Added logic to retry transient errors (e.g., timeouts, rate limits) up to a specified maximum number of retries, improving robustness in error handling.
- Implemented parameter casting for tool arguments to ensure correct types are used, addressing common LLM quirks.
- Enhanced tool execution error messages with guidance for retrying with different approaches.
- Added validation for configuration values to prevent runtime errors.
2026-03-25 22:08:29 +08:00
lyingbug 00ac14aabb fix: add LOAD_HISTORY to rag_stream pipeline for independent history
Previously, rag_stream relied entirely on the rewrite plugin's
internal loadHistory() to load conversation history. This meant that
when rewrite was disabled (enable_rewrite=false), RAG mode had NO
conversation history at all — multi-turn conversations silently
degraded to single-turn.

Changes:
1. Add LOAD_HISTORY as the first step in rag_stream pipeline,
   ensuring history is always available regardless of rewrite config.

2. Make rewrite plugin reuse chatManage.History if already populated
   by the upstream LOAD_HISTORY step, avoiding duplicate DB queries.

This brings rag_stream in line with chat_history_stream which already
has LOAD_HISTORY as an independent step.
2026-03-25 22:08:29 +08:00
lyingbug e5c46642e9 fix: handle thinking stream edge cases (unclosed tags + missing done)
Two related fixes for thinking mode streaming:

1. chat_completion_stream.go: When the response channel closes
   unexpectedly (context cancelled, upstream error) while thinking
   is in progress, the <think> tag was left unclosed. This caused
   the stored message to have an incomplete tag, making the frontend
   show a perpetual "thinking..." state on history reload.
   Now we emit a closing </think> tag when the channel ends with
   thinkingStarted=true but thinkingEnded=false.

2. remote_api.go: The thinking Done=true event was only sent when
   the first answer content chunk arrived. If the model produced
   only reasoning with no answer (e.g., finish_reason set with
   empty content), the thinking-done signal was never emitted.
   Now we also check isDone && state.hasThinking at the end of
   processStreamDelta to ensure the done event is always sent.
2026-03-25 22:08:29 +08:00
lyingbug 1c5013e559 fix: strip thinking content before indexing to chat history KB
The assistant message content may contain <think>...</think> tags from
extended thinking mode. Previously this raw thinking content was
indexed into the chat history knowledge base, polluting embeddings
and degrading RAG retrieval quality with intermediate reasoning text.

Now we strip thinking blocks before building the passage for
IndexMessageToKB, consistent with how load_history and rewrite
plugins already strip thinking from conversation history.
2026-03-25 22:08:29 +08:00
lyingbug cbee01f5e9 fix: improve markdown table cleaning and filter empty passages in rerank (#752)
Enhanced cleanPassageForRerank() to convert markdown table data rows
(| col1 | col2 |) into plain text ("col1, col2") instead of leaving
them intact. Added filtering to skip empty/whitespace-only passages
before sending to the rerank API, preventing Baidu Qianfan 400 errors
caused by empty document fragments after markdown cleaning.
2026-03-25 22:08:29 +08:00
ochan.kwon b291381cc2 feat: auto-describe MCP tool result images via VLM
When an MCP tool returns image content items, the agent engine now
automatically generates text descriptions using the configured VLM
model and appends them to the tool result message. This makes image
content accessible to LLMs that receive tool results as text.

Images are not passed directly to the LLM because Chat Completions API
does not reliably support images in tool role messages across providers
(tested: gpt-5.4 silently ignores them, Qwen3.5 processes them).

Changes:
- Add Images field to ToolResult for preserving base64 data URIs
- Add extractContentAndImages() with MIME whitelist, size/count limits
- Add ImageDescriberFunc injection to AgentEngine (no vlm package dep)
- Add describeImages() with graceful error handling and ctx cancellation
- Add VLM model injection in CreateAgentEngine (same pattern as
  SetAppConfig/SetSkillsManager)
- Redact image base64 from Data map to prevent log/SSE exposure
2026-03-25 21:38:15 +08:00
ochan.kwon 8d8cc051be fix(tenant): prevent api_key encryption loss on tenant settings update
When a tenant setting is updated via PUT /tenants/kv/{key} (e.g.,
retrieval-config, parser-engine-config), the GORM AfterFind-decrypted
plaintext api_key was silently written back to the database because
db.Updates() does not trigger the BeforeSave hook.

Guard the repository's UpdateTenant to skip writing plaintext api_key
back to the database. Pre-encrypted values (enc:v1:… from CreateTenant
/ UpdateAPIKey) are written as-is; AfterFind-decrypted plaintext is
blanked so GORM skips the column entirely.

Closes #798
2026-03-25 21:25:21 +08:00
Dounx 8df12aeee2 fix: make Milvus vector metric type configurable via MILVUS_METRIC_TYPE 2026-03-25 21:20:49 +08:00
ChenRussell bfa78cc933 fix(rerank): filter empty passages to prevent Aliyun API error
Aliyun rerank API rejects empty documents with error "document index:X
should not be empty". This fix validates passage content before building
the documents array, skipping empty chunks and logging them for data
quality monitoring.
2026-03-25 21:19:40 +08:00
Dounx e11bd8c57d fix: use milvus result set length for empty search results 2026-03-20 18:22:40 +08:00
Andy Yang 5c31a9123f fix: Adjust temperature validation to allow zero as a valid setting and default only for negative values. 2026-03-20 18:21:39 +08:00
wizardchen c25741673c feat(chunk): implement batch deletion for chunks to avoid MySQL placeholder limit
- Updated DeleteChunks method to split IDs into batches of 5000 to prevent MySQL Error 1390.
- Added unit tests for CalculateFAQContentHash to ensure consistent hashing behavior across various scenarios, including normalization and case sensitivity.
2026-03-19 18:20:09 +08:00
wizardchen 8db4b6cdea fix(chunk): update timestamp function in UpdateChunks method to use NOW() for consistency 2026-03-19 18:20:09 +08:00
wizardchen f3a3c72a4f fix(docparser): handle nested linked images and url parens 2026-03-18 21:56:34 +08:00
wizardchen 3450ed5e41 feat(knowledge): offload manual processing to async tasks 2026-03-18 21:56:34 +08:00
wizardchen 4e2583e58d feat: add language support to summary generation payload
- Enhanced the SummaryGenerationPayload struct to include a Language field for better localization.
- Updated enqueueSummaryGenerationTask and ProcessSummaryGeneration methods to handle language context in summary generation tasks.
2026-03-17 22:23:24 +08:00
wizardchen 17da967f92 refactor: update session title generation and language context handling
- Modified the session title generation template to focus on extracting only the intent of the user's question.
- Enhanced the session service to include language context when generating titles.
- Updated middleware to inject language into the request context.
- Added a new test suite for validating SSRF safety of URLs.
2026-03-17 22:23:24 +08:00
wizardchen 139a9c40ff feat: enhance localization and configuration support
- Added timezone and language settings to the environment configuration.
- Introduced built-in agent configurations with multilingual support for various agents.
- Updated Docker Compose to utilize new environment variables for timezone and language.
- Created new prompt templates for question generation, summary generation, and keywords extraction.
2026-03-17 22:23:24 +08:00
wizardchen 09d9fe5ddb feat(im): add file support and knowledge base integration 2026-03-17 22:23:24 +08:00
wizardchen 2fd6e226a5 fix: sanitize tab chars and avoid double .md extension in manual download filename
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 18:18:23 +08:00
wizardchen 66c384823b test: add unit tests for manual knowledge download filename logic
Table-driven tests for sanitizeManualDownloadFilename covering normal titles,
each dangerous character replacement (/  \  "  \n  \r), blank/whitespace-only
titles falling back to "untitled.md", and safe passthrough of = ; characters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 18:01:16 +08:00
wizardchen e0c1b77b70 fix: quote Content-Disposition filename and add empty-title fallback
- Use mime.FormatMediaType for both attachment and inline Content-Disposition
  headers in DownloadKnowledgeFile and the preview handler, preventing header
  injection via a user-controlled knowledge title (e.g. containing ';' or '=')
- Extract inline sanitize logic into sanitizeManualDownloadFilename helper and
  add a blank/whitespace-only title fallback to "untitled" so the download
  never produces a degenerate ".md" filename

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 18:01:07 +08:00
wizardchen 63caf564ee feat: support manual knowledge download in GetKnowledgeFile
Add an early-return branch that streams manual knowledge content
(stored as Markdown in Metadata.content) directly as a .md file,
avoiding the runtime error that occurred when FilePath is empty.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 17:49:45 +08:00
wizardchen 6a7eb420a9 refactor: update conversation handling and image description requirements
- Removed outdated instructions regarding the `skip_kb_search` field in the conversation configuration.
- Clarified requirements for `image_description` to ensure it is non-empty when images are present.
- Adjusted the handling of message parts in the chat pipeline to improve image processing.
- Deleted unused migration files related to image handling in the database schema.

These changes enhance the clarity and functionality of the conversation and image processing logic.
2026-03-16 15:15:24 +08:00