Swarm.ResourceMgr.MaxMemory vs GOMEMLIMIT

Checklist

• This is a bug report, not a question. Ask questions on discuss.ipfs.tech.
• I have searched on the issue tracker for my bug.
• I am running the latest kubo version or have an issue updating.

Installation method

dist.ipfs.tech or ipfs-update

Version

v0.32.1

Config

https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmresourcemgrmaxmemory

Description

Problem

Swarm.ResourceMgr.MaxMemory is often interpreted and used as "memory limit for Kubo", while it is only passed to go-libp2p and used for limited set of things (transports, but not bitswap).

Ref. https://github.com/libp2p/go-libp2p/tree/master/p2p/host/resource-manager#readme

The default Swarm.ResourceMgr.MaxMemory is dynamic and conservative:

Default: [TOTAL_SYSTEM_MEMORY]/2 Type: optionalBytes

The actual memory limit people want to set is for entire Kubo, and that is likely GOMEMLIMIT but we do not set it by default.

This means:

• some OOM could be avoided, but are not
• some leaks could be mitigated, but are not
• users who set Swarm.ResourceMgr.MaxMemory are still getting OOM, because it does not cover all the bases
• bad ux, and perception of "Kubo being memory hog"

Solution brainstorm

• Have a sensible (dynamic) limit by default
  • Implicit default should be something conservative like we have in Swarm.ResourceMgr.MaxMemory.
    • [TOTAL_SYSTEM_MEMORY]*0.75 ?
  • Linux users / Docker uses cgroups (control groups) limits extensively to manage and constrain resource usage. We could leverage something like https://github.com/KimMachineGun/automemlimit / https://kupczynski.info/posts/go-container-aware/ to seamlessly respect mem limits set
• Allow user to override default
  • Respect GOMEMLIMIT if is present in env, it should always take precedence, and implicit default for Swarm.ResourceMgr.MaxMemory should be calculated from that ceiling.
  • Refuse to start if user manually set Swarm.ResourceMgr.MaxMemory higher than implicit (or explicit) GOMEMLIMIT
  • TBD: to solve UX problem of user searching for memory in config and only finding
  • Detect when users are running with less memory, or limits than https://github.com/ipfs/kubo?tab=readme-ov-file#minimal-system-requirements ResourceMgr.MaxMemory
    • Perhaps have clear Limits.MaxMemory ? Or generic Env.* and support setting Env.GOMEMLIMIT via config?

cc @sukunrt @ipfs/kubo-maintainers for feedback / ideas.