Commit Graph

5 Commits

Author SHA1 Message Date
ochan.kwon b291381cc2 feat: auto-describe MCP tool result images via VLM
When an MCP tool returns image content items, the agent engine now
automatically generates text descriptions using the configured VLM
model and appends them to the tool result message. This makes image
content accessible to LLMs that receive tool results as text.

Images are not passed directly to the LLM because Chat Completions API
does not reliably support images in tool role messages across providers
(tested: gpt-5.4 silently ignores them, Qwen3.5 processes them).

Changes:
- Add Images field to ToolResult for preserving base64 data URIs
- Add extractContentAndImages() with MIME whitelist, size/count limits
- Add ImageDescriberFunc injection to AgentEngine (no vlm package dep)
- Add describeImages() with graceful error handling and ctx cancellation
- Add VLM model injection in CreateAgentEngine (same pattern as
  SetAppConfig/SetSkillsManager)
- Redact image base64 from Data map to prevent log/SSE exposure
2026-03-25 21:38:15 +08:00
wizardchen daa9ef500c feat: add agent management functionality
- Introduced a new package for managing custom agents, including CRUD operations for agent creation, retrieval, updating, and deletion.
- Implemented API endpoints for listing agents and retrieving agent placeholders.
- Added data structures for agent configuration and requests, enhancing the overall agent management capabilities.
- Enhanced the client with methods to interact with the new agent management features, improving user experience in managing agents.

These changes significantly expand the application's functionality for handling custom agents, providing users with a comprehensive toolset for agent management.
2026-03-09 17:34:13 +08:00
wizardchen 12497a2fea feat: 重构客户端结构体,增强块、消息、会话和租户模型,并更新API文档 2025-12-02 22:47:13 +08:00
wizardchen eb8bfddfd1 refactor: Update function signatures across multiple files to improve readability by adding context parameters and enhancing code structure 2025-11-28 16:17:52 +08:00
wizardchen 56eb2bce33 init commit 2025-08-05 15:08:07 +08:00