[Bug][Memory] persistent_memory injects the whole MEMORY.md on every start, so startup context grows by ~5k characters per session, forever

persistent_memory=True injects the entire MEMORY.md as a system preamble on every construction. MEMORY.md is an append-only log of every message the agent has ever written, so the preamble grows linearly with the agent's whole history — and it is paid on the very first request of every run, before the agent has done anything.

Root cause

swarms/structs/conversation.py:290 (read) and :310 (inject):

    def _preload_memory_md(self) -> None:
        """Preload prior MEMORY.md content as a System preamble message."""
        try:
            with open(self.memory_md_path, "r", encoding="utf-8") as f:
                content = f.read()
        ...
        self.conversation_history.append(
            {
                "role": "System",
                "content": (
                    "[Persistent Memory — MEMORY.md]\n"
                    ...
                    f"{content}"
                ),
            }
        )

content is the whole file. Every message goes into that file, because add_in_memory mirrors unconditionally (conversation.py:494):

        # Persist to MEMORY.md if enabled
        if self.memory_md_path:
            self._append_to_memory_md(role, content)

Nothing caps either side. compact() does wipe and re-seed MEMORY.md (conversation.py:334), but only when a compaction fires mid-run — it is a reaction to the in-run conversation crossing 90% of context_length, not to the preamble. An agent doing many short runs never compacts and never stops growing.

grep -rn "memory_md" swarms/ | grep -i "limit\|max\|cap\|truncate" returns nothing.

Reproducer

Five sessions, ten exchanges each, against the same MEMORY.md:

import os
from swarms.structs.conversation import Conversation

path = os.path.join(os.environ["WORKSPACE_DIR"], "agents", "Probe", "MEMORY.md")

def session(n_messages):
    c = Conversation(name="Probe", system_prompt="You are helpful.", memory_md_path=path)
    preamble = [m for m in c.conversation_history if m["role"] == "System"
                and "[Persistent Memory" in str(m.get("content", ""))]
    size = len(preamble[0]["content"]) if preamble else 0
    for i in range(n_messages):
        c.add("User", f"question {i} " + "x" * 200)
        c.add("Assistant", f"answer {i} " + "y" * 200)
    return size

print("run  preamble_chars  memory_md_bytes")
for run in range(1, 6):
    size = session(10)
    print(f"{run:>3}  {size:>14,}  {os.path.getsize(path):>15,}")

Output on master @ 31f93639 (v15.0.1):

run  preamble_chars  memory_md_bytes
  1               0            5,354
  2           5,462           10,564
  3          10,632           15,774
  4          15,802           20,984
  5          20,972           26,194

Straight line, ~5,170 characters per session, forever. At 20 exchanges a day an agent is carrying roughly 100k characters — order 25k tokens — of its own back-catalogue into the first request of every run, before the task. At context_length=32000 that is most of the window spent on history nobody asked for, and it is the system preamble, so it is not what dynamic_context_window trims.

The content is also raw: every tool result, every [SUBTASK DONE] line, every previous run's completion summary, verbatim.

Suggested change

Bound what gets injected, not what gets written — the log has value on disk.

# conversation.py
MEMORY_MD_PRELOAD_CHARS = 8000

def _preload_memory_md(self) -> None:
    ...
    if len(content) > MEMORY_MD_PRELOAD_CHARS:
        # Keep the head (headings and any curated sections) and the most
        # recent entries; drop the middle, and say so rather than
        # silently truncating.
        head, tail = self._split_for_preload(content, MEMORY_MD_PRELOAD_CHARS)
        content = f"{head}\n\n[… older entries omitted …]\n\n{tail}"

Two decisions worth making explicitly:

• Head plus tail, not just the tail. A plain tail slice would cut the file header and any curated section off the front. The recent entries are the useful part of the log, but a section like ## Lessons Learned (PR #2197) is the part that should survive a bound by design.
• A character cap, not a token cap. count_tokens on the whole file at construction time costs a tokenizer pass on every agent start, for a bound that does not need to be exact. If a token bound is wanted it should be derived from context_length and computed once.

The alternative I would reject: running ContextCompressor over MEMORY.md at startup. It turns every agent construction into an LLM call, which is a much larger behavior change than the problem warrants, and it is billed.

Acceptance criteria

• The preamble injected at construction is bounded regardless of MEMORY.md's size.
• The bound is visible to the model — omitted history is marked, not silently dropped.
• A file smaller than the bound is injected unchanged, so nothing changes for ordinary use.
• MEMORY.md itself is still appended to in full.

Environment

swarms master @ 31f93639 (v15.0.1), Python 3.12, macOS.

Related

• #1998 — lessons in MEMORY.md; PR #2197 gives them a capped section and explicitly leaves this bullet open.
• #1962 — ContextCompressor, which handles the in-run conversation but not this preamble.