#398·EverOS

OME: SkillClusterUpdated persists a 1024-dim vector, inflating run_record ~18x

Author: Kendrick-SongCreated Aug 6, 2026Updated Aug 6, 2026

What

SkillClusterUpdated.case_vector (added in #393) carries the triggering case's 1024-dim embedding so extract_agent_skill avoids a second embedding call. The event payload is persisted verbatim, so the vector lands in two stores:

  • run_record.event_payload (_stores/run_record.py, column is TEXT NOT NULL)
  • the APScheduler jobstore, while a job is queued (engine.py:686 serializes the event into args)

Measured with a realistic payload:

with case_vector : 14,074 bytes
without          :    765 bytes   -> 18.4x, +13.3 KB/record

At the default max_records_per_strategy = 1000, the skill_cluster_updated ring buffer is ~14 MB instead of ~0.8 MB. It has no audit value — nobody reads 1024 floats out of a run record.

The vector is only consumed when a cluster holds more skills than MAX_SKILLS_IN_PROMPT (extract_agent_skill.py:265-273), which for a highly-aggregated artifact like a skill should be the minority of runs. Most of the time the 13 KB rides along unused.

Why it wasn't fixed in #393

The obvious fix — strip large fields when persisting to run_record — is not local, and this is the part worth carrying forward:

Crash recovery replays the persisted payload to rebuild the event (_background/crash_recovery.py:63 -> engine.py:749 model_validate_json). A trimmed persisted copy deserializes with case_vector is None, which is a legal value that silently routes the recovered run down the md-ordering fallback branch while the original run took the LanceDB-ranked branch. Same event id, different selection, no error. So trimming needs either a payload/audit split with recovery reading the untrimmed copy, or an explicit "vector unavailable after recovery" signal.

Options

  1. Audit projection. Give BaseEvent a hook (or pass model_dump_json(exclude=...)) used only for the run_record write, keeping the recovery path on the full payload. Fixes the audit bloat, leaves the jobstore copy.
  2. Don't carry the vector; re-embed in the > MAX_SKILLS_IN_PROMPT branch. Costs one embedding call on the minority path. Note this does not reintroduce the eventual-consistency dependency #393 removed: that branch already reads LanceDB (find_topk_relevant_in_cluster) and already degrades to md ordering when the index is stale.
  3. Keep as-is and document the sizing (done in CHANGELOG 1.2.3 + #393 body).

Option 2 is the smaller diff and removes the field entirely; option 1 preserves the saved call. Either needs the recovery-divergence question answered first.

Not urgent

Bounded by the ring buffer, so it does not grow without limit. Filed so ome.db sizing is traceable to a decision rather than an accident.

Found during post-merge-review of #393.