Missing Context Window Management and Token Compression

Problem Description

The trae-agent framework currently lacks context window management mechanisms, which can lead to:

1. Unbounded Message History Growth: \BaseAgent\ accumulates messages in the conversation loop without any limit, causing memory overflow and token limit exceeded errors during long-running tasks.

2. No Token Counting: There is no mechanism to count tokens or detect when approaching model context limits.

3. LakeView Steps Accumulation: \LakeView.steps\ list grows indefinitely without cleanup, leading to memory leaks.

4. Frequent Disk I/O: \TrajectoryRecorder\ writes to disk on every LLM interaction and agent step, causing performance issues.

Impact

• Memory Issues: Long-running agents can consume excessive memory
• Token Limit Errors: Agents may exceed model context windows, causing failures
• Performance Degradation: Frequent disk I/O slows down execution
• Scalability Problems: Cannot handle complex, multi-step tasks effectively

Suggested Solution

This issue will be addressed through multiple focused PRs:

1. PR-1: Token Counter Utility + Context Compression

   • Implement token counting using \ iktoken\ (already installed)
   • Add automatic message compression when approaching context limits

2. PR-2: Message History Limits

   • Add configurable \max_history_length\ parameter to \BaseAgent\

3. PR-3: LakeView Step Limits

   • Add \max_steps\ parameter to \LakeView\ with automatic cleanup

4. PR-4: Batched Trajectory Writing

   • Implement buffered writing in \TrajectoryRecorder\ to reduce I/O

Additional Context

This is a critical issue for production use cases where agents need to run for extended periods or handle complex tasks with many steps. The current implementation works for short tasks but fails for longer workflows.

Note: Each PR will be minimal and focused to ensure easy review and integration.