Gateway does not shed live sessions under memory pressure: the cgroup-aware eviction planner is used only for the bot warm-agent cache, not for WebSocketGateway sessions

Summary

The gateway server observes memory pressure but never acts on it for its own live sessions. WebSocketGateway._sessions is bounded only by an idle/TTL resume window and count-based reaping — there is no RSS/cgroup-budgeted shedding of live gateway session objects. A burst of concurrent, long-lived sessions (each pinning a multi-MB transcript and agent state) can therefore grow the resident set until the kernel/cgroup OOM-kills the process — taking every connected user down at once — even though the SDK already ships a ready-made, persistence-safe pressure-eviction planner that the bot warm-agent cache uses. The planner is simply not wired into the gateway.

For a world-class, robust gateway, staying alive under load is table stakes: it should shed the coldest sessions (whose transcripts are already durable) before it hits the memory ceiling, not fall over.

Current behaviour

The core planner exists and is production-shaped — it derives a budget from the cgroup limit and picks LRU victims:

# src/praisonai-agents/praisonaiagents/gateway/protocols.py:4078
def plan_pressure_evictions(...):
    """Select least-recently-used sessions to evict under a memory budget..."""

The bot warm-agent cache uses it (persistence-gated: only sheds sessions whose transcript is already flushed):

# src/praisonai-bot/praisonai_bot/bots/_session.py:1958-1976
# ... uses plan_pressure_evictions which caches to shed, and drops their ...
from praisonaiagents.gateway import plan_pressure_evictions  # lazy
plan = plan_pressure_evictions(...)

The gateway server does not. WebSocketGateway only folds a cheap /proc+cgroup memory sample into a heartbeat for observability (Issue #4603) and surfaces pressure on /health — it never calls plan_pressure_evictions:

$ grep -n 'plan_pressure_evictions' src/praisonai-bot/praisonai_bot/gateway/server.py
# (no matches)

Live gateway sessions are bounded only by idle TTL and count-based reaping, never by a memory budget:

# src/praisonai-bot/praisonai_bot/gateway/server.py
resume_window = self.config.session_config.resume_window   # :5259  (idle TTL only)
async def _reap_session(self, client_id: str): ...          # :4380  (idle/slow-consumer)

_evict_slow_consumer (server.py:4246) evicts a connection whose outbound buffer overflows, and admission backpressure (_compute_pressure, server.py:5498) meters inbox/outbox/event-loop lag — none of these shed live sessions under RSS/cgroup memory pressure.

Desired behaviour

When the gateway's resident set approaches its cgroup budget, it soft-evicts the least-recently-used sessions whose transcripts are already fully persisted (so they transparently rebuild from the durable session on the next turn), shedding before the throttling/OOM point rather than being killed. Pressure eviction should be:

• Budgeted from the actual cgroup limit (v2 memory.high/memory.max), with a fraction of headroom so eviction runs before throttling makes a graceful flush miss the stop deadline.
• Persistence-gated — never evict a session whose transcript is not yet on disk (avoid data loss), exactly as the warm-agent cache already does.
• Bounded per pass, protecting the hottest N sessions.

Layer placement

• Primary layer: wrapper (praisonai-bot gateway server — it owns WebSocketGateway._sessions and the sweep loop that must call the planner).
• Why not core: the pure decision function (plan_pressure_evictions) and cgroup readers already live in core as protocol-level policy; adding the actuation loop there would pull runtime session ownership into core, which must stay protocol-only.
• Why not tools: this is runtime lifecycle self-protection, not an agent-callable capability.
• Why not plugins: OOM-avoidance of the framework's own session store is a core runtime guarantee, not an optional user-installed lifecycle hook (a plugin also cannot safely mutate WebSocketGateway._sessions).
• Secondary touch: core — reuse plan_pressure_evictions as-is; optionally expose a transcript_persistence_caught_up-style predicate on GatewaySessionProtocol so the planner can gate on durability uniformly.
• 3-way surface (CLI + YAML + Python): partial — a gateway.memory_pressure YAML block (budget fraction, protected count, sweep interval) and a start flag; not a per-agent Python API.

Proposed approach

• Extension point: the existing periodic gateway housekeeping/heartbeat sweep calls plan_pressure_evictions over self._sessions, gated on per-session transcript-persisted state, then evicts victims (and malloc_trim), mirroring run_agent_cache.py's _sweep_agent_cache_under_pressure.

Minimal API sketch:

# in WebSocketGateway housekeeping loop
budget = cgroup_budget_mb(fraction=0.65)
if anon_rss_mb() >= budget:
    victims = plan_pressure_evictions(
        sessions=self._sessions.values(),
        budget_mb=budget,
        protect_hottest=cfg.protect_count,
        is_evictable=lambda s: s.transcript_persistence_caught_up(),  # never drop unsaved
    )
    for s in victims:
        await self._soft_evict_session(s)   # rebuilt from durable store next turn

Resolution sketch

# Before (today): gateway only *samples* memory for /health; sessions grow until OOM
self._record_memory_sample()             # server.py ~:2315 (observability only)
# no eviction of live self._sessions under RSS/cgroup pressure -> OOM kill

# After (proposed): shed cold, already-persisted sessions before the ceiling
if anon_rss_mb() >= cgroup_budget_mb(0.65):
    for s in plan_pressure_evictions(self._sessions.values(), ...):
        if s.transcript_persistence_caught_up():
            await self._soft_evict_session(s)   # transparent rebuild on next message

Severity

High — under concurrent long-lived load the gateway can OOM-kill itself and drop all sessions, despite a ready-made, persistence-safe eviction planner already shipping in the SDK and being used elsewhere.

Validation

• Core planner present: src/praisonai-agents/praisonaiagents/gateway/protocols.py:4078 (plan_pressure_evictions).
• Used by the bot warm-agent cache: src/praisonai-bot/praisonai_bot/bots/_session.py:1958-1976.
• Not used by the gateway server: grep -n plan_pressure_evictions src/praisonai-bot/praisonai_bot/gateway/server.py → no matches; the server only samples memory into a heartbeat (server.py:2315, Issue #4603) and surfaces it on /health.
• Gateway sessions bounded only by idle TTL / count: server.py:5259 (resume_window), server.py:4380 (_reap_session); connection-level eviction only in _evict_slow_consumer (server.py:4246) and admission metering in _compute_pressure (server.py:5498) — neither sheds sessions on RSS/cgroup pressure.

Generated with Claude Code

https://claude.ai/code/session_01YXMLKVbg32jxwo829ZEApT