wizardchen 0bb1a581de feat(agent): enhance token management and context handling in AgentEngine ...

- Added token usage tracking to the AgentEngine, allowing for better estimation of current context token counts.
- Implemented a new method to estimate current tokens based on previous usage and newly appended messages, improving context management.
- Updated context window management to utilize estimated token counts for more efficient message consolidation.
- Enhanced streaming methods to include token usage information, providing better insights into LLM interactions.
- Refactored the token estimator to support BPE tokenization, ensuring accurate token counts for messages.
- Added unit tests to validate new token estimation and context management functionalities.

2026-03-25 22:08:29 +08:00

..

compress.go

feat(agent): enhance token management and context handling in AgentEngine

2026-03-25 22:08:29 +08:00

estimator_test.go

feat(agent): enhance token management and context handling in AgentEngine

2026-03-25 22:08:29 +08:00

estimator.go

feat(agent): enhance token management and context handling in AgentEngine

2026-03-25 22:08:29 +08:00