Proton Drive: concurrent operations each construct a new Fs, running Argon2 per construction and exhausting memory

What is the problem you are having with rclone?

When several requests reach a freshly started rclone rcd at the same time for the same Proton Drive remote, each one constructs its own Fs. Constructing a Proton Drive Fs unlocks the account's OpenPGP keys, which uses Argon2 — memory-hard by design — so N concurrent first-requests cost N times that allocation.

Measured here at 64–128 MiB per construction (it varies between runs). On a memory-limited host the result is fatal: rclone is OOM-killed mid-upload, every in-flight HTTP request is reset (ECONNRESET / socket hang up), and the process restarts.

Note this is not limited to full logins. The cached-credential path pays the same cost — see the DEBUG output below.

A heap profile taken during the failure (/debug/pprof/heap?debug=1, aggregated by allocation site):

939524096  golang.org/x/crypto/argon2.initBlocks <- argon2.deriveKey <- argon2.IDKey
 67108864  golang.org/x/crypto/argon2.initBlocks <- argon2.deriveKey <- argon2.IDKey
 67108864  golang.org/x/crypto/argon2.initBlocks <- argon2.deriveKey <- argon2.IDKey
 22020096  github.com/rclone/rclone/lib/pool.New.func1 <- pool.(*Pool).GetN

Sys = 1316 MiB, HeapAlloc = 1192 MiB — roughly 1 GiB of Argon2 across about 16 allocations. Everything else is negligible.

Growth is fast; the process starts small and is killed within ~15–30 seconds:

sys=22MB   heap=7MB      <- fresh process
sys=174MB  heap=158MB
sys=619MB  heap=547MB
sys=1003MB heap=878MB    <- OOM shortly after

For contrast, when a single construction is allowed to complete before any other traffic arrives, the same daemon doing the same uploads into the same folder sits at:

HeapAlloc   8.7 MB
Sys        26.4 MB

So one construction is entirely affordable. The problem is only that several run concurrently.

The cached-credential path pays the same cost

With --log-level=DEBUG, every construction in the failing run took the reusable path — there is no Using username and password to log in anywhere:

DEBUG : proton drive root link ID '': Has cached credentials
DEBUG : proton drive root link ID '': Has cached credentials
DEBUG : proton drive root link ID '': Has cached credentials
...
DEBUG : proton drive root link ID '': Used cached credential to initialize the ProtonDrive API

Caching the session avoids the SRP handshake, not the OpenPGP key unlock, and it is the key unlock that allocates.

The concurrency window

Thirteen operations/uploadfile calls arrived within three seconds, and each construction takes roughly three seconds, so they all miss the cache:

13:20:52 rc: "operations/uploadfile"   x4
13:20:53 rc: "operations/uploadfile"   x3
13:20:54 rc: "operations/uploadfile"   x3
13:20:55 ... Used cached credential to initialize the ProtonDrive API   <- first completes
13:20:55 fs cache: renaming cache item "protondrive:" to be canonical "protondrive{...}:"
13:20:55 fs cache: switching user supplied name "protondrive:" for canonical name   <- later calls hit

Once the first construction finishes, the entry is registered and subsequent calls hit the cache normally. Everything that arrived before then had already started its own.

Root cause

lib/cache.Cache.Get releases its mutex before calling create, so concurrent callers for the same key all miss and all construct:

https://github.com/rclone/rclone/blob/master/lib/cache/cache.go

func (c *Cache) Get(key string, create CreateFunc) (value any, err error) {
	c.mu.Lock()
	entry, ok := c.cache[key]
	if !ok {
		c.mu.Unlock() // Unlock in case Get is called recursively
		value, ok, err = create(key)
		...

There is no de-duplication of in-flight creations. For most backends create is cheap, so this is harmless. For protondrive it is an Argon2 derivation, so the cost is multiplied by the number of concurrent callers.

One thing worth flagging for anyone looking at this: the unlock appears deliberate, to allow Get to be called recursively, so holding the mutex across create would presumably deadlock. I have not attempted a patch.

I am reporting the behaviour rather than proposing where it should be addressed, since that depends on trade-offs I am not close to. The reason it seemed worth filing is that the multiplication is what turns an affordable one-off cost into an out-of-memory kill.

For completeness: the remote here is declared through RCLONE_CONFIG_<NAME>_<KEY> environment variables, so rclone logs detected overridden config - adding "{...}" suffix to name and caches under the canonical suffixed name. I do not think that is the cause — the same window would exist for a remote defined in rclone.conf — but it is visible above and may be worth ruling out.

Run the command 'rclone version' and share the full output of the command.

rclone v1.75.0
- os/version: alpine 3.24.1 (64 bit)
- os/kernel: 6.18.33-v8+ (aarch64)
- os/type: linux
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.26.5
- go/linking: static
- go/tags: none

Running the official rclone/rclone:1.75.0 image on a Raspberry Pi (64-bit Raspberry Pi OS). Nothing about the failure looks architecture-specific — the allocation is the same on any platform; a small-memory host simply reaches the limit sooner.

Which cloud storage system are you using?

Proton Drive

The command you were trying to run

rclone rcd with a protondrive remote, driven over the rc API:

rclone rcd --rc-addr=:5572 \
  --protondrive-replace-existing-draft=true \
  --temp-dir=/data/temp \
  --low-level-retries=1 \
  --fs-cache-expire-duration=24h \
  --log-level=INFO

Clients then call operations/uploadfile (multipart) for the same fs=protondrive:. Any situation where more than one request arrives before the first construction completes reproduces it.

How to reproduce

1. Configure a protondrive remote.
2. Start rclone rcd in a container limited to 1 GiB.
3. Issue several rc calls for the same remote simultaneously, e.g. a handful of operations/list fs=protondrive: remote= in parallel, before any single call has completed.
4. Observe memory climb by roughly 64–128 MiB per concurrent call and the process being OOM-killed. A single call instead completes and leaves the daemon at under 30 MiB.

Workaround

Ensure the Fs exists before any concurrent traffic. A single cheap call such as operations/fsinfo forces construction and makes no backend API request of its own; after that a burst costs almost nothing, because the uploads themselves are small (the attachments here are 11–87 KB, and one upload keeps the heap under 10 MB).

Also useful, though neither avoids the allocation on its own:

• Raise --fs-cache-expire-duration above the polling interval; the 5m default means a remote polled every 15m reconstructs on every run.
• Stagger independent clients so they do not poll simultaneously.

Raising the memory limit does not fix it — it only allows more concurrent constructions to survive before the kill.