ScoreBasedContextCreator token-count cache goes stale under a sliding memory window
Bug description
ScoreBasedContextCreator.create_context() caches the total token count keyed only on message count, not message identity/content:
if current_count == self._cached_message_count:
# Same message count, use cached value directly
return messages, self._cached_token_countWhen the agent's memory has a window_size set (ChatHistoryMemory(window_size=N)), the retrieved record count saturates at window_size once the history fills: each new turn drops the oldest message and appends a new one, so current_count stays constant indefinitely even though the actual message content — and its real token count — keeps changing. The cache then returns an arbitrarily stale total forever.
This isn't cosmetic: create_context()'s returned token count is what ChatAgent compares against token_limit to decide whether to trigger auto-summarization / context truncation. A stale, undercounted total means the agent can believe it has budget when it doesn't, and skip the exact protection meant to keep requests under the model's context window.
Reproduction
Confirmed against camel-ai==0.2.90, no network/LLM calls (pure local token counting):
from camel.memories.context_creators.score_based import ScoreBasedContextCreator
from camel.memories.records import ContextRecord, MemoryRecord
from camel.messages import BaseMessage
from camel.types import ModelType, OpenAIBackendRole
from camel.utils import OpenAITokenCounter
def make_record(content, ts):
msg = BaseMessage.make_user_message(role_name="user", content=content)
mr = MemoryRecord(message=msg, role_at_backend=OpenAIBackendRole.USER, timestamp=ts)
return ContextRecord(memory_record=mr, score=1.0, timestamp=ts)
creator = ScoreBasedContextCreator(
token_counter=OpenAITokenCounter(model=ModelType.GPT_4O_MINI),
token_limit=100_000,
)
turn1 = [make_record("hi", 1.0), make_record("hello", 2.0), make_record("ok", 3.0), make_record("yes", 4.0)]
messages1, real_tokens1 = creator.create_context(turn1)
creator.set_cached_token_count(real_tokens1, len(messages1)) # what ChatAgent does after a real LLM response
# Window slides: oldest dropped, one huge message appended -- count is unchanged, content is not.
huge = "TOKEN " * 5000
turn2 = [make_record("hello", 2.0), make_record("ok", 3.0), make_record("yes", 4.0), make_record(huge, 5.0)]
messages2, cached_tokens2 = creator.create_context(turn2)
real_tokens2 = creator.token_counter.count_tokens_from_messages(messages2)
print(cached_tokens2, real_tokens2)
# -> 23 5023create_context() returns the stale turn-1 count (23) for a completely different message set whose real token count is 5023.
Expected behavior
The cache should be invalidated whenever the underlying message set actually changes, not just when its length changes — e.g. by keying on message identity/hash rather than count, or by detecting when older records have been evicted from the window.
Environment
camel-ai==0.2.90, verified against a fresh master clone (byte-identical file).
Happy to send a PR for this once a direction is agreed — my inclination is to key the cache on the id/hash of the oldest and newest record rather than just len(messages), but open to whatever approach you'd prefer.
Source: camel-ai/camel