#4328·camel

ScoreBasedContextCreator token-count cache goes stale under a sliding memory window

Author: mittalpkCreated Sep 11, 2026Updated Sep 12, 2026

Bug description

ScoreBasedContextCreator.create_context() caches the total token count keyed only on message count, not message identity/content:

https://github.com/camel-ai/camel/blob/master/camel/memories/context_creators/score_based.py#L148-L154

python
if current_count == self._cached_message_count:
    # Same message count, use cached value directly
    return messages, self._cached_token_count

When the agent's memory has a window_size set (ChatHistoryMemory(window_size=N)), the retrieved record count saturates at window_size once the history fills: each new turn drops the oldest message and appends a new one, so current_count stays constant indefinitely even though the actual message content — and its real token count — keeps changing. The cache then returns an arbitrarily stale total forever.

This isn't cosmetic: create_context()'s returned token count is what ChatAgent compares against token_limit to decide whether to trigger auto-summarization / context truncation. A stale, undercounted total means the agent can believe it has budget when it doesn't, and skip the exact protection meant to keep requests under the model's context window.

Reproduction

Confirmed against camel-ai==0.2.90, no network/LLM calls (pure local token counting):

python
from camel.memories.context_creators.score_based import ScoreBasedContextCreator
from camel.memories.records import ContextRecord, MemoryRecord
from camel.messages import BaseMessage
from camel.types import ModelType, OpenAIBackendRole
from camel.utils import OpenAITokenCounter

def make_record(content, ts):
    msg = BaseMessage.make_user_message(role_name="user", content=content)
    mr = MemoryRecord(message=msg, role_at_backend=OpenAIBackendRole.USER, timestamp=ts)
    return ContextRecord(memory_record=mr, score=1.0, timestamp=ts)

creator = ScoreBasedContextCreator(
    token_counter=OpenAITokenCounter(model=ModelType.GPT_4O_MINI),
    token_limit=100_000,
)

turn1 = [make_record("hi", 1.0), make_record("hello", 2.0), make_record("ok", 3.0), make_record("yes", 4.0)]
messages1, real_tokens1 = creator.create_context(turn1)
creator.set_cached_token_count(real_tokens1, len(messages1))  # what ChatAgent does after a real LLM response

# Window slides: oldest dropped, one huge message appended -- count is unchanged, content is not.
huge = "TOKEN " * 5000
turn2 = [make_record("hello", 2.0), make_record("ok", 3.0), make_record("yes", 4.0), make_record(huge, 5.0)]
messages2, cached_tokens2 = creator.create_context(turn2)
real_tokens2 = creator.token_counter.count_tokens_from_messages(messages2)

print(cached_tokens2, real_tokens2)
# -> 23 5023

create_context() returns the stale turn-1 count (23) for a completely different message set whose real token count is 5023.

Expected behavior

The cache should be invalidated whenever the underlying message set actually changes, not just when its length changes — e.g. by keying on message identity/hash rather than count, or by detecting when older records have been evicted from the window.

Environment

camel-ai==0.2.90, verified against a fresh master clone (byte-identical file).

Happy to send a PR for this once a direction is agreed — my inclination is to key the cache on the id/hash of the oldest and newest record rather than just len(messages), but open to whatever approach you'd prefer.