Commit Graph

7 Commits

Author SHA1 Message Date
wizardchen 8325dac738 fix(security): harden web_fetch and datasource connector SSRF paths
Use SSRF-safe HTTP clients in chat pipeline web_fetch and Feishu/Yuque/Notion
connectors, validating tenant-supplied base_url values before outbound requests.
2026-07-02 12:27:48 +08:00
wizardchen c694fcf6cd test: address PR review follow-ups for OAuth and logger cleanup
Restore logger config via t.Cleanup after t.Setenv so Yuque token tests
do not leak global state, add unit tests for MCP OAuth non-interactive
context helpers and IM auth notice formatting, and fix EmbedChatCore
indentation.
2026-07-01 17:31:42 +08:00
wizardchen 8d40959f88 fix lint and test failures 2026-07-01 17:31:42 +08:00
wizardchen 5d0d317a36 feat(datasource): reveal pre-existing selections in lazily-loaded picker
When editing a data source whose tree loads one level at a time (Feishu
wiki), a saved deep-node selection was invisible: only top-level spaces
were listed, so the selected node could not be seen and its ancestor
space rendered as unchecked.

- Add Connector.ResolveResourceAncestors: given selected resource IDs,
  return the ancestor IDs that must be expanded to reveal them. Feishu
  walks up via GetWikiNode (O(depth) per selection, no full-tree walk);
  Notion/Yuque return empty.
- Fix wikiNode.ParentNodeID JSON tag: the Feishu API field is
  parent_node_token, not parent_node_id, so GetWikiNode can return the
  parent and the ancestor walk works in production.
- Expose POST /datasource/{id}/resource-ancestors (service + handler +
  route) and a matching frontend API client.
- Edit-mode picker reveals hidden selections by expanding and lazily
  loading each ancestor level, so selections are visible and checked.
- Gate child fetching on treeFullyLoaded instead of a partial-children
  heuristic, avoiding a redundant request for full-tree connectors.
2026-06-22 19:24:08 +08:00
wizardchen d8f13ada4a fix(datasource): lazy-load Feishu wiki resources to avoid listing timeout (#1672)
Listing resources for a Feishu data source eagerly walked the entire wiki
tree (one API call per node across every space), which exceeded the request
timeout for large knowledge bases. ListResources now takes a parentID and
returns a single level at a time: empty lists the wiki spaces, a space/node
ID lists only its direct children. The frontend fetches children on demand
when a node is expanded. Notion (full tree in one call) and Yuque (flat)
keep their existing behavior by ignoring non-empty parentID.

Refs: https://github.com/Tencent/WeKnora/issues/1672
2026-06-22 17:53:30 +08:00
liuwei435 e0c6599c54 fix(datasource): support Yuque team token in connector
- Add Type field to v2User struct to distinguish personal/team tokens
- Route team tokens directly to ListGroupRepos (skip ListUserGroups)
- Gracefully handle ListUserGroups 404 for personal tokens without teams
- Add rate-limiting delay between GetDocDetail calls to avoid API throttling
2026-05-26 20:46:22 +08:00
nullkey 656b07027a feat(datasource): add Yuque (语雀) connector
Syncs personal and team Yuque knowledge bases into WeKnora as Markdown,
with full and incremental sync + deletion detection. Mirrors the Feishu
connector pattern; fetches V2Doc.body directly (no async export step).

Config: api_token (required) + base_url (optional, defaults to public cloud).
401/403 wraps ErrInvalidCredentials; 429 honors Retry-After; filters to
status=1 Doc-type at list stage to conserve API quota. Body is trusted as
Markdown for format=markdown|lake — empirically verified against the v2
API: Lake XML lives separately in body_lake, body itself is the Markdown
view. Other formats (e.g. html) are skipped with a placeholder item so
they surface in sync logs instead of silently feeding HTML to the parser.

v2Doc.status accepts both int and string shapes — Yuque's OpenAPI spec
declares string but the live API returns int. sanitizeFileName truncates
at a UTF-8 rune boundary so multi-byte titles don't emit invalid UTF-8
that downstream filename validation rejects.

Also adds logger.SetOutput test hook, used by the token-redaction test.
2026-04-22 16:39:55 +08:00