Commit Graph

1090 Commits

Author SHA1 Message Date
DaWesen b80bdb1079 feat: add video multimodal functionality 2026-04-09 19:19:42 +08:00
liweixin 2b10cc54de test: remove unnecessary OpenAI dimensions regression test 2026-04-09 19:19:25 +08:00
liweixin 5233394d01 fix: gate Azure OpenAI dimensions support 2026-04-09 19:19:25 +08:00
liweixin bf514ff68a test: harden remote api chat fixture for SSRF validation 2026-04-09 19:19:25 +08:00
liweixin c663182a00 feat: add support for dimensions parameter in Azure OpenAI and OpenAI embedding requests 2026-04-09 19:19:25 +08:00
liweixin 188f317248 fix: treat 400 errors as successful connection in model check (parameter mismatch is not a connectivity issue) 2026-04-09 19:19:25 +08:00
liweixin c6215405a5 fix: override AzureModelMapperFunc to preserve deployment name as-is 2026-04-09 19:19:25 +08:00
liweixin 4e4118d406 fix: pass provider to connection test so Azure OpenAI uses correct endpoint 2026-04-09 19:19:25 +08:00
liweixin 2eb2dea634 feat: add Azure OpenAI support for embedding models 2026-04-09 19:19:25 +08:00
liweixin f72b032c1d feat: add Azure OpenAI to frontend provider list and i18n 2026-04-09 19:19:25 +08:00
liweixin 3cfd034eec feat: add Azure OpenAI support for VLM models 2026-04-09 19:19:25 +08:00
liweixin 22ec575727 feat: add Azure OpenAI support for chat models 2026-04-09 19:19:25 +08:00
liweixin c9001ef9d9 feat: add Azure OpenAI provider registration with metadata 2026-04-09 19:19:25 +08:00
liweixin 8ceb0b194b feat: add Azure OpenAI provider constants and URL detection 2026-04-09 19:19:25 +08:00
ochan.kwon 69785ad0a3 feat: add VectorStore entity, repository, and migrations
Introduce the VectorStore domain model as a foundation for multi-store
vector DB architecture (Phase 1, PR 1/4).

- VectorStore entity with ConnectionConfig (AES-GCM encrypted) and IndexConfig
- VectorStoreRepository interface and GORM implementation (CRUD + duplicate check)
- PostgreSQL migration (000031) and SQLite migration (000001)
- Unit tests for types, encryption, validation, and helpers
2026-04-09 01:39:51 +08:00
nullkey 11b30fdd61 feat(frontend): tree view resource selector with cascading selection
- Render resources as expandable tree with parent-child indentation
- Cascade check/uncheck to all descendants
- Add has_children to Resource API type
- Add 'untitled' i18n key for en-US, zh-CN, ko-KR, ru-RU
2026-04-08 16:31:25 +08:00
nullkey 2cd979e1ea feat(notion): implement Connector interface, markdown renderer, and DI registration
- Implement Validate, ListResources, FetchAll, FetchIncremental
- Per-record database sync with individual knowledge items
- Incremental sync with record-level change detection for databases
- Tree-structured resource listing with parent-child relationships
- Markdown renderer for all Notion block types including nested content
- Register NotionConnector in DI container
- Add unit tests for connector and markdown
2026-04-08 16:31:25 +08:00
nullkey 9f35abafff feat(notion): add Notion API client and type definitions
- Define notionPage, notionBlock, notionParent, notionFile and cursor types
- Implement rate-limited HTTP client with context-aware retry and backoff
- Support search, page/block retrieval, database queries, and file downloads
- Add HasChildren field to Resource type, NotionChannel to knowledge types
- Add unit tests for types and client
2026-04-08 16:31:25 +08:00
AndyYang d5ecc150e0 feat(agent): support customizable LLM call timeout and add docker-compose mapping 2026-04-07 11:26:52 +08:00
AndyYang f63c9f6dbe fix: resolve tool name duplication in streaming tool calls 2026-04-05 18:25:45 +08:00
nullkey 29de7dfbbd fix: allow MINIO_ENDPOINT to be configured via environment variable
Previously MINIO_ENDPOINT was hardcoded to minio:9000 in docker-compose.yml,
preventing users from connecting to an external MinIO service. Now it supports
override via .env while keeping the same default for backward compatibility.
2026-04-03 18:41:26 +08:00
wizardchen 7604b793eb chore(release): update to version 0.3.6 2026-04-03 13:42:37 +08:00
wizardchen 155fc690a7 feat(audio): add audio file preview support and enhance UI components
- Integrated audio file preview functionality in document content and preview components.
- Updated localization files to include audio-related messages.
- Enhanced the UI to display an audio player for supported audio formats.
- Implemented audio loading state and error handling for unsupported audio playback.
2026-04-03 13:41:16 +08:00
nullkey 39831fb61c fix(wecom): strip @mention from group chat messages to fix slash commands
WeCom AI Bot does not strip @mention from text.content in group chats,
causing all slash commands (/stop, /clear, /help) to fail — the content
arrives as "@BotName /stop" instead of "/stop".

- Add stripAtMentionBasic (stateless) and stripAtMention (with bot name
  caching) to handle multi-word bot display names
- Support bot_name in channel credentials for immediate precision
- Apply stripping in both WebSocket and webhook adapters, text and mixed
- Add mention_test.go with 21 test cases
2026-04-03 11:34:22 +08:00
nullkey b66552749d fix(im): anti-hallucination for non-text quotes and unprocessable media messages
- Replace content placeholders with LLM instructions for non-text quotes (NonTextType field)
- Intercept image/file messages on channels without KB config
- Wrap text quotes in <quoted_message> XML tags with escape sanitization
- Add nil safety to isQuoteFromBot, move debug logging to callers with real ctx
- Improve reply copy to guide users toward channel KB configuration
2026-04-03 11:34:22 +08:00
nullkey dc5046ad2a feat(pipeline): inject QuotedContext at LLM prompt stage 2026-04-03 11:34:22 +08:00
nullkey 4e7922c8a1 feat(im): add QuotedContext to shared types and IM service layer 2026-04-03 11:34:22 +08:00
nullkey 3587ddb1c7 feat(wecom): implement quote extraction and populate in handleCallback 2026-04-03 11:34:22 +08:00
nullkey 9790cb4830 feat(im): add QuotedMessage type and Quote field to IncomingMessage 2026-04-03 11:34:22 +08:00
ChenRussell 2cbd98e36f fix(migrations): correct migration numbers and remove broken trigger
Migration 000029 and 000030 had off-by-one numbering in comments/notices.
Also removed update_updated_at_column() trigger from 000030 as the
function is never defined and no other table uses this pattern.
2026-04-03 11:31:21 +08:00
wizardchen 9bb776a91c feat(chat): refine parent-child chunk replacement logic in merge process
- Enhanced the logic for replacing child content with parent content to only apply to text chunks whose parent is a parent_text chunk.
- Added checks to skip non-text chunks and ensure that only appropriate parent chunks are used for replacement, improving content quality and integrity.
2026-04-02 21:42:47 +08:00
wizardchen 4ea58bc6f3 feat(assets): add ASR test audio file and embed it in the application
- Introduced a new audio file, asr_test.wav, for ASR testing purposes.
- Created embed.go to embed the ASR test audio file as a byte slice.
- Updated initialization.go to utilize the embedded ASR test audio for model transcription checks.
2026-04-02 21:27:27 +08:00
wizardchen c93f4e94ae feat(asr): add ASR model connection check and update related components
- Implemented a new function to check ASR model connectivity via the /v1/audio/transcriptions endpoint.
- Updated ModelEditorDialog.vue to integrate ASR model checks and adjust UI elements accordingly.
- Enhanced localization files to reflect changes in ASR terminology and descriptions.
- Added ASR configuration options to the knowledge base and updated related migration scripts.
- Refactored model handling to support ASR as a distinct model type in the application.
2026-04-02 21:27:27 +08:00
wizardchen d3c4992e01 docs(api): enhance agent and knowledge base documentation 2026-04-02 21:27:27 +08:00
wizardchen 624e24ba49 feat(asr): integrate Automatic Speech Recognition (ASR) support
- Added ASR configuration options to the knowledge base and model settings.
- Implemented ASR model selection and transcription capabilities in the knowledge processing pipeline.
- Enhanced file upload support to include audio formats for ASR processing.
- Updated localization files to include ASR-related labels and descriptions.
- Refactored model handling to accommodate ASR as a new model type.
2026-04-02 21:27:27 +08:00
nullkey d68c94a11a fix(stream): use io.EOF instead of errors.New("EOF") in SSEReader
errors.New("EOF") creates a distinct error instance that doesn't match
io.EOF, causing processRawHTTPStream to silently ignore normal stream
termination without sending a done response.

- SSEReader: return io.EOF instead of errors.New("EOF")
- processRawHTTPStream: handle io.EOF as normal completion, send done
  response with usage/toolCalls (aligned with processStream behavior)

Fixes #860
2026-04-02 18:35:53 +08:00
wizardchen cb36570cde feat(summary): enhance document summarization capabilities and UI
- Added max_input_chars configuration to limit input size for summary generation.
- Updated generate_summary.yaml to clarify summary generation steps and requirements.
- Introduced a new summary section in the document content view to display generated summaries or loading states.
- Refactored doc-content.vue to utilize computed properties for improved performance and readability.
- Enhanced knowledge base hooks to include description and summary status fields for better data handling.
- Updated internationalization files to include new summary-related labels in multiple languages.
2026-04-02 18:34:39 +08:00
chenrui7 8daecd346c fix(frontend): add null checks for markdown code block text parameter
Prevents TypeError when rendering empty code blocks in markdown documents.
Adds validation in doc-content.vue and document-preview.vue renderer.code.
2026-04-02 15:27:50 +08:00
wizardchen 9d021a29b7 feat(elasticsearch): enhance ID field handling with dynamic suffix detection
- Added logic to determine whether to append the ".keyword" suffix to ID fields based on index mapping.
- Introduced `detectFieldTypes` method to inspect index mappings and set the `useKeywordSuffix` flag accordingly.
- Updated methods to utilize the new `idField` function for consistent ID field querying across the repository.
- Improved logging for better visibility into mapping detection and suffix usage.
2026-04-02 15:26:34 +08:00
ochan.kwon 33e919cc80 fix(knowledge): delete extracted images from storage when knowledge is removed
Previously, only the original uploaded file was deleted via knowledge.FilePath.
Images extracted by the builtin parser (stored as individual S3/MinIO/local files
via ImageResolver) were never cleaned up, causing orphan files to accumulate.

- Add ListImageInfoByKnowledgeIDs repository method (no chunk_type filter)
- Add collectImageURLs utility with URL deduplication and JSON parse logging
- Add deleteExtractedImages helper (standalone, best-effort with error logging)
- Fix DeleteKnowledge: collect image URLs before errgroup, delete in file goroutine
- Fix DeleteKnowledgeList: bulk collect with KB-level fileSvc mapping
- Fix cleanupKnowledgeResources: collect before chunk deletion, delete after
- Fix ProcessKBDelete: collect before chunk deletion for KB-wide cleanup
2026-04-02 12:04:04 +08:00
ochan.kwon 8a375749f1 fix(i18n): use WEKNORA_LANGUAGE env for prompt language fallback instead of hardcoded zh-CN
- Add DefaultLanguage() that reads WEKNORA_LANGUAGE env with zh-CN fallback
- Replace hardcoded zh-CN in LanguageNameFromContext with DefaultLanguage()
- Unify middleware/language.go to use types.DefaultLanguage()
- Add Language field to DocumentProcessPayload (8 enqueue sites + worker restore)
- Add Language field to ImageMultimodalPayload (enqueue + worker restore)
- Propagate language from ImageMultimodalPayload to QuestionGenerationPayload
2026-04-02 12:01:25 +08:00
wizardchen 115bc826c2 docs: update README files to streamline section headings and remove redundant content across multiple languages 2026-04-02 11:55:14 +08:00
wizardchen c866cf5d44 docs: update project descriptions and features in multiple language READMEs 2026-04-02 11:32:41 +08:00
wizardchen 301d75eb52 docs: reorder Web UI table rows and remove outdated MCP server setup instructions 2026-04-02 10:35:05 +08:00
Windfarer 54da98fc24 feat: add docx max pages env config 2026-04-02 10:31:52 +08:00
wizardchen 3756c7c796 feat(session): restrict @mentions to agent's allowed knowledge base scope
- Implemented logic to filter user-provided KB and knowledge IDs based on the shared agent's allowed scope.
- Added a new method, restrictMentionsToAgentScope, to validate and restrict @mentions, preventing unauthorized access to knowledge bases.
- Enhanced logging to provide visibility into blocked mentions and the filtering process.
2026-04-02 10:31:20 +08:00
wizardchen b1fe7abbe1 feat(knowledge): enhance knowledge tag batch update with authorization checks
- Updated UpdateKnowledgeTagBatch method to include an authorizedKBID parameter for restricting updates to specific knowledge items.
- Implemented validation to ensure that all requested knowledge IDs belong to the authorized knowledge base.
- Enhanced GetKnowledgeBatch and UpdateKnowledgeTagBatch methods to incorporate agent-specific knowledge base access restrictions.
- Added utility functions to resolve allowed knowledge base IDs for agents based on their configuration.
2026-04-02 10:31:20 +08:00
wizardchen cb504e35b8 refactor(knowledge): remove unnecessary URL resolution for image paths 2026-04-01 21:59:22 +08:00
wizardchen 8e1cfaccb7 refactor(prompt_templates): improve question generation guidelines and context handling
- Updated the question generation template to clarify the role of surrounding context and main content.
- Enhanced quality rules for generated questions to better align with user search intent.
- Revised output format and added explicit instructions on what not to generate.
- Improved logging and output in the web parser for better visibility of parsed content and metadata.
2026-04-01 21:59:22 +08:00
Windfarer 1b7070c1a2 simplify oidc callback result payload 2026-04-01 20:47:18 +08:00