- guard against empty content completion in analyzeResponse/executeLoop
with retry (max 2), nudge message, and fallback response
- propagate actual finish_reason through streaming pipeline instead of
hardcoding "stop" in streamThinkingToEventBus
- add FinishReason field to StreamResponse and streamLLMResult
- add finish marker chunk in processStreamDelta for empty content scenarios
- propagate lastFinishReason through EOF handler in processStream
- strengthen PureAgent system prompt with MCP tool guidelines and
graceful fallback instructions for unanswerable questions
- add unit tests for empty response bug and finish_reason propagation
- Improved the message consolidation logic to preserve the current turn's user query and all subsequent assistant/tool messages, ensuring better context retention.
- Updated the CompressContext function to reflect the new consolidation strategy, maintaining the system prompt and relevant recent messages.
- Refactored the context manager to support optional message repository for improved context rebuilding from persistent storage.
- Added comprehensive tests to validate the new consolidation behavior and ensure correct message handling across various scenarios.
- Added token usage tracking to the AgentEngine, allowing for better estimation of current context token counts.
- Implemented a new method to estimate current tokens based on previous usage and newly appended messages, improving context management.
- Updated context window management to utilize estimated token counts for more efficient message consolidation.
- Enhanced streaming methods to include token usage information, providing better insights into LLM interactions.
- Refactored the token estimator to support BPE tokenization, ensuring accurate token counts for messages.
- Added unit tests to validate new token estimation and context management functionalities.
- Removed the query display section from the DatabaseQuery component to streamline the UI.
- Updated the DatabaseQueryData interface to eliminate the query field, reflecting changes in tool result handling.
- Enhanced the DatabaseQuery tool execution to focus on returning structured results without exposing raw SQL queries.
- Added a new utility to strip <think> blocks from LLM outputs, improving content clarity and user experience.
- Introduced the AgentEngine with methods for executing tool calls, managing context windows, and handling final answer generation.
- Added functionality to stream LLM responses and tool results through the EventBus, enhancing real-time interaction.
- Implemented context management to consolidate messages and prevent exceeding token limits, ensuring efficient use of context.
- Enhanced error handling and logging for tool execution and response analysis, improving robustness and traceability.
- Added unit tests to validate the new functionalities and ensure reliability in agent operations.
- Introduced a default timeout for tool execution to prevent indefinite hangs during long-running tasks.
- Implemented context cancellation handling in the execution loop to gracefully manage timeouts and user cancellations.
- Added a JSON repair utility to fix common malformations in tool arguments, improving robustness in argument parsing.
- Enhanced message sanitization to merge consecutive user messages and handle orphaned tool results, ensuring compatibility with LLM providers.
- Added unit tests for the new JSON repair and message sanitization functionalities to validate their effectiveness.
- Introduced a memory consolidator to summarize older messages when the context window exceeds a specified limit, preserving key facts and tool results.
- Enhanced the AgentEngine to utilize the new token estimator and memory consolidator for improved context window management.
- Implemented logic to compress message history when approaching token limits, ensuring efficient use of context.
- Added unit tests for the memory consolidator and token estimator to validate functionality and performance.
- Updated tool execution to validate parameters against JSON schema before execution, improving error handling and robustness.
- Introduced a configurable LLM call timeout with a default value, allowing for better control over LLM call durations.
- Added logic to retry transient errors (e.g., timeouts, rate limits) up to a specified maximum number of retries, improving robustness in error handling.
- Implemented parameter casting for tool arguments to ensure correct types are used, addressing common LLM quirks.
- Enhanced tool execution error messages with guidance for retrying with different approaches.
- Added validation for configuration values to prevent runtime errors.
When an MCP tool returns image content items, the agent engine now
automatically generates text descriptions using the configured VLM
model and appends them to the tool result message. This makes image
content accessible to LLMs that receive tool results as text.
Images are not passed directly to the LLM because Chat Completions API
does not reliably support images in tool role messages across providers
(tested: gpt-5.4 silently ignores them, Qwen3.5 processes them).
Changes:
- Add Images field to ToolResult for preserving base64 data URIs
- Add extractContentAndImages() with MIME whitelist, size/count limits
- Add ImageDescriberFunc injection to AgentEngine (no vlm package dep)
- Add describeImages() with graceful error handling and ctx cancellation
- Add VLM model injection in CreateAgentEngine (same pattern as
SetAppConfig/SetSkillsManager)
- Redact image base64 from Data map to prevent log/SSE exposure
- Updated the AgentEngine to emit final answer events even when the LLM stops without calling the final_answer tool, ensuring the frontend can render responses correctly.
- Added fallback logic in the AgentStreamHandler to emit final answers as fallback events when no answer events are streamed, improving robustness in edge cases.
- Added timezone and language settings to the environment configuration.
- Introduced built-in agent configurations with multilingual support for various agents.
- Updated Docker Compose to utilize new environment variables for timezone and language.
- Created new prompt templates for question generation, summary generation, and keywords extraction.
- Added functionality for image uploads in chat, allowing users to attach images for multimodal Q&A.
- Enhanced the input field to handle image selection via drag-and-drop and paste, with validation for file types and sizes.
- Updated the backend to process images alongside text queries, including support for image analysis.
- Introduced new UI components for image previews and management, improving user interaction with uploaded content.
- Added localization strings for image upload features and error messages, enhancing accessibility for users in multiple languages.
These changes significantly improve the chat experience by enabling users to incorporate images into their queries, facilitating richer interactions and responses.
- Introduced the final_answer tool to ensure agents submit their complete responses as the last action, improving response accuracy and consistency.
- Updated the agent engine to log detailed information about tool calls and responses, including the total duration of agent execution.
- Enhanced the chat message model to include agent_duration_ms, tracking the total execution time from query start to answer delivery.
- Implemented JSON field extraction for streaming responses, allowing for more efficient handling of tool call outputs.
These changes significantly improve the agent's response handling and ensure that final answers are consistently delivered, enhancing overall user experience.
Convert all LLM-facing prompts, templates, and tool output labels to
language-neutral English so the model responds in the user's input
language. Update frontend relevance/match-type mappings to match.
Output language strategy:
- User-facing: reply in user's input language
- Vector-indexed: match source document language
- LLM-internal: fixed English
- Added support for query expansion in conversation settings, allowing the system to generate additional queries when recall is low.
- Updated localization files in English, Russian, and Chinese to include new strings for the query expansion feature.
- Enhanced UI in AgentSettings.vue to allow users to toggle query expansion settings.
- Refactored related API and service logic to accommodate the new query expansion functionality in session management and chat processing.
- Updated agent configuration to support separate system prompts for web search enabled and disabled states.
- Removed deprecated agent configuration parameters to streamline settings management.
- Enhanced UI components in AgentSettings.vue to allow configuration of custom prompts based on web search status.
- Improved localization in English, Russian, and Chinese for new prompt settings and UI elements.
- Refactored related API and service logic to accommodate changes in agent configuration structure.
- Added functionality for manual knowledge entry, allowing users to create and edit Markdown content directly in the browser.
- Introduced new API endpoints for creating and updating manual knowledge entries.
- Enhanced UI components to support the new manual knowledge editor, including integration with existing knowledge base features.
- Updated knowledge retrieval logic to accommodate manual entries, ensuring seamless user experience across knowledge types.
- Improved agent configuration to support custom system prompts, enhancing flexibility in agent responses.
- Eliminated duplicate assignment of step.Duration in the executeLoop method to streamline duration tracking.
- Ensured accurate logging of round duration without affecting overall functionality.
- Added duration tracking for agent steps and tool calls, improving event data accuracy.
- Updated AgentStreamDisplay and ToolResultRenderer components to include fallback headers and improved output styling.
- Refactored duration handling in the agent engine and session management to ensure consistent duration reporting.
- Enhanced UI elements for better user experience, including scrollable output areas and improved tooltip visibility.
- Added logic to load and manage web search configuration in the InputField component, ensuring proper handling of web search states.
- Implemented user notifications for unconfigured web search settings, guiding users to the appropriate settings page.
- Enhanced the AgentStreamDisplay component to improve tooltip content rendering for knowledge base citations.
- Updated event data structures to include total duration in milliseconds for better precision in agent completion events.
- Refactored citation handling to ensure inline citations are displayed correctly, improving user experience in content rendering.
- Introduced a new endpoint to fetch chunks by ID, improving access to specific knowledge base entries.
- Updated tool result types to include web search results, enhancing the display capabilities in the chat component.
- Enhanced the AgentStreamDisplay component to support rendering web search results with improved HTML formatting.
- Implemented logic for managing web search result summaries and citations, ensuring a cohesive user experience.
- Refactored session management to accommodate new web search functionalities, improving overall system integration.
- Updated AgentConfig to include web search settings, allowing for dynamic configuration of web search features.
- Refactored AgentService to accept SessionService, enabling better management of session-specific web search states.
- Implemented methods in SessionService for handling temporary knowledge base states related to web search.
- Enhanced PluginSearch to utilize SessionService for managing web search state, improving integration with the overall system.
- Updated container setup to ensure proper dependency injection for the new web search service and session management functionalities.
- Introduced a new PlaceholderDefinition interface to manage available placeholders for system prompts.
- Enhanced AgentSettings.vue to allow users to configure system prompts with dynamic placeholders, improving user experience.
- Updated the AgentConfig structure to include a system prompt field, enabling custom prompt templates.
- Modified backend components to support the new system prompt template, ensuring compatibility with existing configurations.
- Implemented logic for rendering placeholders in the system prompt, allowing for dynamic content generation based on user input.
- Updated agentChat API to accept optional knowledge_base_ids and agent_enabled parameters.
- Introduced getSession API to retrieve session details.
- Modified useStream to validate knowledge_base_ids for agent-chat requests and prepare POST body accordingly.
- Enhanced createKnowledgeBase API to include additional configuration options.
- Refactored chat routing to remove knowledge base ID from the chat path, allowing for dynamic knowledge base selection.
- Improved session management by preventing duplicate session entries in the menu store.
- Updated UI components for better session handling and display.
- Added context management for agent conversations to improve state persistence across sessions.
ochan.kwon