Commit Graph

629 Commits

Author SHA1 Message Date
begoniezhao d4edd374df refactor: replace CryptoService with utils AES helpers and simplify encryption
- Remove infrastructure/crypto package (CryptoService, config, PBKDF2-based encryption)
- Replace with lightweight utils.GetAESKey/EncryptAESGCM/DecryptAESGCM helpers
- Remove cryptoSvc dependency from modelService, tenantService, weKnoraCloudService
- Remove crypto state persistence logic from container initialization
- Add TruncatePromptTokens field to WeKnoraCloud embed request
- Update frontend hint text for WeKnoraCloud credential errors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 20:55:39 +08:00
begoniezhao 2c0c1b8bab feat: integrate ChatBot provider and docreader 2026-04-13 17:47:24 +08:00
nullkey f6b484c94a fix(im): strip <kb> and <web> citation tags before sending to IM platforms
IM platforms (WeCom, Feishu, Slack, etc.) cannot render the inline
citation tags produced by the agent pipeline, causing raw tags like
<kb doc="..." chunk_id="..." /> to appear in user-facing messages.

Strip these tags in the IM service layer before sending, while
preserving the original content in the database for web frontend
rendering.
2026-04-13 15:10:47 +08:00
wizardchen d1c7da97f9 feat(im): add WeChat integration with QR code login and long-polling support
- Extended the IMChannel interface to include WeChat as a platform, supporting long-polling mode and full output mode.
- Implemented WeChat QR code login functionality, allowing users to authenticate via a QR code.
- Added new API endpoints for generating and polling the status of WeChat QR codes.
- Updated the frontend to handle WeChat-specific UI elements and interactions, including dynamic form adjustments based on the selected platform.
- Enhanced localization files to include WeChat-related translations.
- Introduced a WeChat adapter for message handling and file downloading via the iLink Bot API.
2026-04-13 14:17:57 +08:00
Jhao 45ad36845e feat(web-search): add baidu web search provider support #907 2026-04-12 20:13:05 +08:00
wizardchen a655a25cf8 fix: revert to neo4j v6 and workaround Go 1.24 Windows compiler bug with -p=1
Go 1.24 has an internal compiler race condition on Windows that causes
'package without types' errors during parallel compilation. Setting
GOFLAGS="-p=1" serializes the build and avoids triggering the bug.
2026-04-11 22:08:02 +08:00
wizardchen 9525f08038 fix: downgrade neo4j driver from v6 to v5 to fix Go 1.24 Windows internal compiler bug 2026-04-11 21:59:03 +08:00
wizardchen 4d244f51d2 feat: integrate lumberjack for log file management, enhancing logging capabilities with rotation and compression 2026-04-11 18:51:16 +08:00
wizardchen 92bf52721c feat: implement auto-setup for Lite edition, enhancing user experience by automating user and tenant creation on first launch 2026-04-11 18:51:16 +08:00
wizardchen 9a2b54536d fix: update environment configuration for debugging and logging, enhance SQLite FTS5 table management, and improve API base URL handling 2026-04-11 18:51:16 +08:00
Jhao fd18264183 feat(web-search): add ollama web search provider support #907 2026-04-10 01:27:29 +08:00
DaWesen b80bdb1079 feat: add video multimodal functionality 2026-04-09 19:19:42 +08:00
liweixin 2b10cc54de test: remove unnecessary OpenAI dimensions regression test 2026-04-09 19:19:25 +08:00
liweixin 5233394d01 fix: gate Azure OpenAI dimensions support 2026-04-09 19:19:25 +08:00
liweixin bf514ff68a test: harden remote api chat fixture for SSRF validation 2026-04-09 19:19:25 +08:00
liweixin c663182a00 feat: add support for dimensions parameter in Azure OpenAI and OpenAI embedding requests 2026-04-09 19:19:25 +08:00
liweixin 188f317248 fix: treat 400 errors as successful connection in model check (parameter mismatch is not a connectivity issue) 2026-04-09 19:19:25 +08:00
liweixin c6215405a5 fix: override AzureModelMapperFunc to preserve deployment name as-is 2026-04-09 19:19:25 +08:00
liweixin 4e4118d406 fix: pass provider to connection test so Azure OpenAI uses correct endpoint 2026-04-09 19:19:25 +08:00
liweixin 2eb2dea634 feat: add Azure OpenAI support for embedding models 2026-04-09 19:19:25 +08:00
liweixin 3cfd034eec feat: add Azure OpenAI support for VLM models 2026-04-09 19:19:25 +08:00
liweixin 22ec575727 feat: add Azure OpenAI support for chat models 2026-04-09 19:19:25 +08:00
liweixin c9001ef9d9 feat: add Azure OpenAI provider registration with metadata 2026-04-09 19:19:25 +08:00
liweixin 8ceb0b194b feat: add Azure OpenAI provider constants and URL detection 2026-04-09 19:19:25 +08:00
ochan.kwon 69785ad0a3 feat: add VectorStore entity, repository, and migrations
Introduce the VectorStore domain model as a foundation for multi-store
vector DB architecture (Phase 1, PR 1/4).

- VectorStore entity with ConnectionConfig (AES-GCM encrypted) and IndexConfig
- VectorStoreRepository interface and GORM implementation (CRUD + duplicate check)
- PostgreSQL migration (000031) and SQLite migration (000001)
- Unit tests for types, encryption, validation, and helpers
2026-04-09 01:39:51 +08:00
nullkey 2cd979e1ea feat(notion): implement Connector interface, markdown renderer, and DI registration
- Implement Validate, ListResources, FetchAll, FetchIncremental
- Per-record database sync with individual knowledge items
- Incremental sync with record-level change detection for databases
- Tree-structured resource listing with parent-child relationships
- Markdown renderer for all Notion block types including nested content
- Register NotionConnector in DI container
- Add unit tests for connector and markdown
2026-04-08 16:31:25 +08:00
nullkey 9f35abafff feat(notion): add Notion API client and type definitions
- Define notionPage, notionBlock, notionParent, notionFile and cursor types
- Implement rate-limited HTTP client with context-aware retry and backoff
- Support search, page/block retrieval, database queries, and file downloads
- Add HasChildren field to Resource type, NotionChannel to knowledge types
- Add unit tests for types and client
2026-04-08 16:31:25 +08:00
AndyYang d5ecc150e0 feat(agent): support customizable LLM call timeout and add docker-compose mapping 2026-04-07 11:26:52 +08:00
AndyYang f63c9f6dbe fix: resolve tool name duplication in streaming tool calls 2026-04-05 18:25:45 +08:00
nullkey 39831fb61c fix(wecom): strip @mention from group chat messages to fix slash commands
WeCom AI Bot does not strip @mention from text.content in group chats,
causing all slash commands (/stop, /clear, /help) to fail — the content
arrives as "@BotName /stop" instead of "/stop".

- Add stripAtMentionBasic (stateless) and stripAtMention (with bot name
  caching) to handle multi-word bot display names
- Support bot_name in channel credentials for immediate precision
- Apply stripping in both WebSocket and webhook adapters, text and mixed
- Add mention_test.go with 21 test cases
2026-04-03 11:34:22 +08:00
nullkey b66552749d fix(im): anti-hallucination for non-text quotes and unprocessable media messages
- Replace content placeholders with LLM instructions for non-text quotes (NonTextType field)
- Intercept image/file messages on channels without KB config
- Wrap text quotes in <quoted_message> XML tags with escape sanitization
- Add nil safety to isQuoteFromBot, move debug logging to callers with real ctx
- Improve reply copy to guide users toward channel KB configuration
2026-04-03 11:34:22 +08:00
nullkey dc5046ad2a feat(pipeline): inject QuotedContext at LLM prompt stage 2026-04-03 11:34:22 +08:00
nullkey 4e7922c8a1 feat(im): add QuotedContext to shared types and IM service layer 2026-04-03 11:34:22 +08:00
nullkey 3587ddb1c7 feat(wecom): implement quote extraction and populate in handleCallback 2026-04-03 11:34:22 +08:00
nullkey 9790cb4830 feat(im): add QuotedMessage type and Quote field to IncomingMessage 2026-04-03 11:34:22 +08:00
wizardchen 9bb776a91c feat(chat): refine parent-child chunk replacement logic in merge process
- Enhanced the logic for replacing child content with parent content to only apply to text chunks whose parent is a parent_text chunk.
- Added checks to skip non-text chunks and ensure that only appropriate parent chunks are used for replacement, improving content quality and integrity.
2026-04-02 21:42:47 +08:00
wizardchen 4ea58bc6f3 feat(assets): add ASR test audio file and embed it in the application
- Introduced a new audio file, asr_test.wav, for ASR testing purposes.
- Created embed.go to embed the ASR test audio file as a byte slice.
- Updated initialization.go to utilize the embedded ASR test audio for model transcription checks.
2026-04-02 21:27:27 +08:00
wizardchen c93f4e94ae feat(asr): add ASR model connection check and update related components
- Implemented a new function to check ASR model connectivity via the /v1/audio/transcriptions endpoint.
- Updated ModelEditorDialog.vue to integrate ASR model checks and adjust UI elements accordingly.
- Enhanced localization files to reflect changes in ASR terminology and descriptions.
- Added ASR configuration options to the knowledge base and updated related migration scripts.
- Refactored model handling to support ASR as a distinct model type in the application.
2026-04-02 21:27:27 +08:00
wizardchen 624e24ba49 feat(asr): integrate Automatic Speech Recognition (ASR) support
- Added ASR configuration options to the knowledge base and model settings.
- Implemented ASR model selection and transcription capabilities in the knowledge processing pipeline.
- Enhanced file upload support to include audio formats for ASR processing.
- Updated localization files to include ASR-related labels and descriptions.
- Refactored model handling to accommodate ASR as a new model type.
2026-04-02 21:27:27 +08:00
nullkey d68c94a11a fix(stream): use io.EOF instead of errors.New("EOF") in SSEReader
errors.New("EOF") creates a distinct error instance that doesn't match
io.EOF, causing processRawHTTPStream to silently ignore normal stream
termination without sending a done response.

- SSEReader: return io.EOF instead of errors.New("EOF")
- processRawHTTPStream: handle io.EOF as normal completion, send done
  response with usage/toolCalls (aligned with processStream behavior)

Fixes #860
2026-04-02 18:35:53 +08:00
wizardchen cb36570cde feat(summary): enhance document summarization capabilities and UI
- Added max_input_chars configuration to limit input size for summary generation.
- Updated generate_summary.yaml to clarify summary generation steps and requirements.
- Introduced a new summary section in the document content view to display generated summaries or loading states.
- Refactored doc-content.vue to utilize computed properties for improved performance and readability.
- Enhanced knowledge base hooks to include description and summary status fields for better data handling.
- Updated internationalization files to include new summary-related labels in multiple languages.
2026-04-02 18:34:39 +08:00
wizardchen 9d021a29b7 feat(elasticsearch): enhance ID field handling with dynamic suffix detection
- Added logic to determine whether to append the ".keyword" suffix to ID fields based on index mapping.
- Introduced `detectFieldTypes` method to inspect index mappings and set the `useKeywordSuffix` flag accordingly.
- Updated methods to utilize the new `idField` function for consistent ID field querying across the repository.
- Improved logging for better visibility into mapping detection and suffix usage.
2026-04-02 15:26:34 +08:00
ochan.kwon 33e919cc80 fix(knowledge): delete extracted images from storage when knowledge is removed
Previously, only the original uploaded file was deleted via knowledge.FilePath.
Images extracted by the builtin parser (stored as individual S3/MinIO/local files
via ImageResolver) were never cleaned up, causing orphan files to accumulate.

- Add ListImageInfoByKnowledgeIDs repository method (no chunk_type filter)
- Add collectImageURLs utility with URL deduplication and JSON parse logging
- Add deleteExtractedImages helper (standalone, best-effort with error logging)
- Fix DeleteKnowledge: collect image URLs before errgroup, delete in file goroutine
- Fix DeleteKnowledgeList: bulk collect with KB-level fileSvc mapping
- Fix cleanupKnowledgeResources: collect before chunk deletion, delete after
- Fix ProcessKBDelete: collect before chunk deletion for KB-wide cleanup
2026-04-02 12:04:04 +08:00
ochan.kwon 8a375749f1 fix(i18n): use WEKNORA_LANGUAGE env for prompt language fallback instead of hardcoded zh-CN
- Add DefaultLanguage() that reads WEKNORA_LANGUAGE env with zh-CN fallback
- Replace hardcoded zh-CN in LanguageNameFromContext with DefaultLanguage()
- Unify middleware/language.go to use types.DefaultLanguage()
- Add Language field to DocumentProcessPayload (8 enqueue sites + worker restore)
- Add Language field to ImageMultimodalPayload (enqueue + worker restore)
- Propagate language from ImageMultimodalPayload to QuestionGenerationPayload
2026-04-02 12:01:25 +08:00
wizardchen 3756c7c796 feat(session): restrict @mentions to agent's allowed knowledge base scope
- Implemented logic to filter user-provided KB and knowledge IDs based on the shared agent's allowed scope.
- Added a new method, restrictMentionsToAgentScope, to validate and restrict @mentions, preventing unauthorized access to knowledge bases.
- Enhanced logging to provide visibility into blocked mentions and the filtering process.
2026-04-02 10:31:20 +08:00
wizardchen b1fe7abbe1 feat(knowledge): enhance knowledge tag batch update with authorization checks
- Updated UpdateKnowledgeTagBatch method to include an authorizedKBID parameter for restricting updates to specific knowledge items.
- Implemented validation to ensure that all requested knowledge IDs belong to the authorized knowledge base.
- Enhanced GetKnowledgeBatch and UpdateKnowledgeTagBatch methods to incorporate agent-specific knowledge base access restrictions.
- Added utility functions to resolve allowed knowledge base IDs for agents based on their configuration.
2026-04-02 10:31:20 +08:00
wizardchen cb504e35b8 refactor(knowledge): remove unnecessary URL resolution for image paths 2026-04-01 21:59:22 +08:00
wizardchen 8e1cfaccb7 refactor(prompt_templates): improve question generation guidelines and context handling
- Updated the question generation template to clarify the role of surrounding context and main content.
- Enhanced quality rules for generated questions to better align with user search intent.
- Revised output format and added explicit instructions on what not to generate.
- Improved logging and output in the web parser for better visibility of parsed content and metadata.
2026-04-01 21:59:22 +08:00
Windfarer 1b7070c1a2 simplify oidc callback result payload 2026-04-01 20:47:18 +08:00
ochan.kwon 258a786129 Revert "refactor(elasticsearch): remove ".keyword" suffix from ID fields in Elasticsearch queries"
This reverts commit 3daaabfc4a.
2026-04-01 20:41:24 +08:00