`encore test` / `encore exec` never call ResourceManager.StopAll() — daemon leaks nsqd + miniredis fds per invocation (still at v1.58.1)
Symptom
On a busy multi-agent development machine running a shared encore daemon across many short-lived sessions, the daemon accumulates roughly 4 file descriptors per encore test / encore exec invocation for the rest of its lifetime:
- 1 flocked
DIRfd on an undeleted/tmp/encore-nsqd*directory - 2 nsqd
LISTENports (TCP + HTTP) - 1 miniredis
LISTENport
Observed rate: ~48-54 fds/hour, with 98 orphaned /tmp/encore-nsqd* dirs and ~100 corresponding port triples still open against zero live sessions, ENCORE_DAEMON_WATCH=0, on encore v1.58.1. The only mitigation is restarting the daemon — no config/env knob disables per-run nsqd/redis.
Root cause (line refs at tag v1.58.1, commit 0579d5ce)
encore test and encore exec each build a per-run infra.ResourceManager, which starts an in-process nsqd + miniredis, but the manager is never torn down.
There are exactly four NewResourceManager call sites; only two of them call StopAll():
| call site | starts services | StopAll() called? |
|---|---|---|
cli/daemon/run/run.go:182 (encore run / check) |
yes | ✅ run.go:240 via Run.Close() |
cli/daemon/run/exec_script.go:68 |
yes | ✅ exec_script.go:69 defer rm.StopAll() |
cli/daemon/run/tests.go:175 (encore test) |
tests.go:178 rm.StartRequiredServices(...) |
❌ never |
cli/daemon/run/exec_command.go:70 (encore exec) |
exec_command.go:126 rm.StartRequiredServices(...) |
❌ never |
In tests.go, rm is only kept as configGen.infraManager to render env vars and then falls out of scope when testSpec returns (tests.go:248) — there's no defer rm.StopAll() anywhere in Manager.Test. Same shape in exec_command.go for ExecSpec.
This isn't a case of an unreferenced-but-open handle waiting for a GC finalizer: nsqd's dirlock (internal/dirlock/dirlock.go) opens the data dir and keeps the *os.File in l.f, released only via NSQDaemon.Stop() → nsqd.Exit(). nsqd.Main() runs in a goroutine with a live accept loop and waitGroup-tracked subsystems, and miniredis likewise keeps an accept goroutine running — both keep everything reachable, so the fds are held for the daemon's full lifetime, not just until the next GC cycle.
Separately, nothing in cli/daemon/pubsub/nsq.go or cli/daemon/run/infra/infra.go ever RemoveAlls the /tmp/encore-nsqd* directory — so even the correctly wired teardown paths (encore run, exec-script) leave the directory behind on disk after StopAll() runs.
Not a regression from #2520
PR #2520 (merged as f368c34b, in v1.58.1) fixed a different, now-closed issue (#2526): eager fsnotify watching and retained *Run entries in mgr.runs. Its diff touches only cli/daemon/apps/apps.go, cli/daemon/run/manager.go, cli/daemon/run/run.go, cli/daemon/run/watch.go, cli/daemon/run_spec.go, cli/daemon/watch.go, and pkg/watcher/*. It never touches cli/daemon/run/infra/infra.go, cli/daemon/pubsub/nsq.go, cli/daemon/redis/redis.go, cli/daemon/run/tests.go, or cli/daemon/run/exec_command.go. This leak is independent of, and predates, that fix.
Suggested fix (3 parts)
cli/daemon/run/tests.go— inManager.Test(aroundtests.go:40), adddefer rm.StopAll()around thebld.RunTestscall, mirroring the existingexec_script.go:69pattern. Covers the dominant case.tests.go'sTestSpecRPC path andexec_command.go'sExecSpecRPC path run the spec on the client side after the RPC returns, sormneeds to outlive the handler call — bind its teardown to the caller's context, e.g.go func() { <-ctx.Done(); rm.StopAll() }()using the gRPC request/stream context (seecli/daemon/test.go:98), or alternatively an idle-timeout sweep inManager.cli/daemon/pubsub/nsq.go—NSQDaemon.Stop()shouldos.RemoveAll(n.Opts.DataPath)aftern.nsqd.Exit(), so the/tmp/encore-nsqd*dir doesn't linger even on the already-correct teardown paths.
As optional hardening: caching a single nsqd/miniredis pair per (app, namespace) in Manager instead of instantiating fresh ones per run would bound the leak even if a future teardown call site is missed.
Happy to send a PR for part 1 and 3 if useful.
Source: encoredev/encore