Request for review: open, matched memory benchmark protocol

Hello Mem0 maintainers,

I maintain Agent Memory, an OSS local memory SDK. We are preparing a public benchmark suite and explicitly do not want to publish a "better than Mem0" claim without a configuration you consider fair.

The proposed protocol keeps these tracks separate:

• retrieval: one LongMemEval release and one retrieval unit for every system;
• end-to-end QA: same reader model, prompt, top-k, temperature, context budget, and official judge;
• decision safety: repeats, stale facts, contradictory updates, TTL, abstention, and wrong-replay rate;
• local resources: same machine/OS/corpus with CPU, RSS, ingest time, and p50/p90/p95/p99 latency.

We will publish raw per-case outputs, dataset hashes, command lines, dependency versions, failures, and unsupported capabilities. Hosted/API results would be reported separately from self-hosted OSS results.

Could you recommend the canonical self-hosted Mem0 configuration for this comparison, including retrieval unit, embedding/LLM configuration, and any capability or failure mode our schema should capture? We are happy to accept a maintainer-provided adapter or configuration correction before publishing comparative results.

Thanks for helping make this useful rather than marketing-driven.