Commit Graph

10 Commits

Author SHA1 Message Date
wizardchen a81ef5fbb2 fix: improve tenant access token logging and update test cases
- Enhanced the logging of the tenant access token in the Feishu client to handle variable lengths for prefix and suffix, ensuring accurate display even for shorter tokens.
- Updated the test cases in `connector_test.go` to reflect changes in the endpoint path for file downloads, improving test reliability.
- Adjusted example outputs in `example_test.go` for clarity and consistency.
- Modified image resolution tests to correct expected outcomes based on updated criteria for icon images.
- Added environment variable settings in image resolver tests to ensure localhost accessibility during testing.

These changes enhance the robustness of logging and testing mechanisms across the application.
2026-04-22 21:17:30 +08:00
nullkey 4939011b0d fix(feishu): truncate sanitized filenames at UTF-8 rune boundary
Raw byte truncation at 200 bytes splits multi-byte codepoints (Chinese
characters are 3 bytes), producing invalid UTF-8 that downstream filename
validation (utf8.ValidString) rejects with "文件名包含非法字符". Peel
trailing bytes via DecodeLastRuneInString until the last rune is complete.

Same latent bug and fix as the Yuque connector's sanitizeFileName.
2026-04-22 16:39:55 +08:00
nullkey 656b07027a feat(datasource): add Yuque (语雀) connector
Syncs personal and team Yuque knowledge bases into WeKnora as Markdown,
with full and incremental sync + deletion detection. Mirrors the Feishu
connector pattern; fetches V2Doc.body directly (no async export step).

Config: api_token (required) + base_url (optional, defaults to public cloud).
401/403 wraps ErrInvalidCredentials; 429 honors Retry-After; filters to
status=1 Doc-type at list stage to conserve API quota. Body is trusted as
Markdown for format=markdown|lake — empirically verified against the v2
API: Lake XML lives separately in body_lake, body itself is the Markdown
view. Other formats (e.g. html) are skipped with a placeholder item so
they surface in sync logs instead of silently feeding HTML to the parser.

v2Doc.status accepts both int and string shapes — Yuque's OpenAPI spec
declares string but the live API returns int. sanitizeFileName truncates
at a UTF-8 rune boundary so multi-byte titles don't emit invalid UTF-8
that downstream filename validation rejects.

Also adds logger.SetOutput test hook, used by the token-redaction test.
2026-04-22 16:39:55 +08:00
wizardchen 9a2b54536d fix: update environment configuration for debugging and logging, enhance SQLite FTS5 table management, and improve API base URL handling 2026-04-11 18:51:16 +08:00
nullkey 2cd979e1ea feat(notion): implement Connector interface, markdown renderer, and DI registration
- Implement Validate, ListResources, FetchAll, FetchIncremental
- Per-record database sync with individual knowledge items
- Incremental sync with record-level change detection for databases
- Tree-structured resource listing with parent-child relationships
- Markdown renderer for all Notion block types including nested content
- Register NotionConnector in DI container
- Add unit tests for connector and markdown
2026-04-08 16:31:25 +08:00
nullkey 9f35abafff feat(notion): add Notion API client and type definitions
- Define notionPage, notionBlock, notionParent, notionFile and cursor types
- Implement rate-limited HTTP client with context-aware retry and backoff
- Support search, page/block retrieval, database queries, and file downloads
- Add HasChildren field to Resource type, NotionChannel to knowledge types
- Add unit tests for types and client
2026-04-08 16:31:25 +08:00
wizardchen eaf17e65f8 docs: remove WeCom document integration from data source import documentation
- Deleted references to WeCom document integration in the data source import module documentation.
- Updated the quick start guide and related sections to reflect the removal of WeCom, enhancing clarity and focus on supported platforms.
2026-03-30 20:46:07 +08:00
wizardchen 1a340a8374 refactor(datasource): remove WeCom document integration and related components
- Deleted WeCom document connector and callback handling code to streamline the datasource management.
- Removed associated localization entries and UI elements related to WeCom integration.
- Updated data source management logic to eliminate references to WeCom, enhancing maintainability and clarity.
- Adjusted error handling and logging to reflect the removal of WeCom-specific functionality.
2026-03-30 20:46:07 +08:00
wizardchen 11cabd5952 feat(datasource): add WeCom document integration and callback setup
- Implemented WeCom document connector to sync smart documents and WeDrive files into WeKnora.
- Added detailed documentation for WeCom integration, including setup instructions for callback URL and API access.
- Enhanced frontend localization to support WeCom-specific messages and instructions.
- Updated data source management UI to include WeCom configuration options and callback URL handling.
- Improved error handling and validation for WeCom API interactions.
2026-03-30 20:46:07 +08:00
wizardchen adabf8e089 feat(datasource): implement data source management functionality
- Added a new API for managing data sources, including CRUD operations and connection validation.
- Introduced UI components for data source settings, sync logs, and editor dialog, enhancing user experience in managing external data sources.
- Updated localization files to support new data source features in multiple languages.
- Implemented cron expression humanization for better scheduling visibility.
- Enhanced knowledge base editor to include data source management options.
2026-03-30 20:46:07 +08:00