PostgreSQL backend reads SQL from disk at runtime, breaking every single-file bundler
Summary
The PostgreSQL backend loads its SQL with readFileSync at runtime, from paths derived from __dirname. That works when bullmq is on disk in node_modules, and breaks under every single-file bundler — the JS is bundled, the .sql files are not, and the first queue operation throws ENOENT.
Reproduction
// app.ts
import { Queue, createPostgresBackend } from 'bullmq';
const q = new Queue('repro', { connection: { connectionString: process.env.U, schema: 'bullmq' } }, createPostgresBackend);
await q.add('t', { n: 1 });bun build --compile app.ts --outfile server
rm -rf node_modules
./serverENOENT: no such file or directory,
open '/path/to/project/node_modules/bullmq/dist/esm/postgres/commands/add_job.sql'
at readFileSync (unknown)
at addJob (/$bunfs/root/server)The binary keeps the absolute build-time path baked in, so this fails on any machine or image where that path is not populated — which for a container image is the normal case.
Restoring only dist/esm/postgres/{commands,migrations} at that exact path (488K, 84 files) makes it work again, which confirms the loader is the only thing involved.
Why this is not Bun-specific
src/postgres/sql-loader.ts computes currentDir from __dirname — falling back to parsing a stack trace for a file:// URL — and reads commands/*.sql and migrations/*.sql beneath it. Any tool that produces a single artefact hits the same wall:
bun build --compileesbuild --bundle(without a copy plugin)pkgdeno compile- Node's Single Executable Applications
The Redis backend has no such problem, because its .lua scripts are inlined into the published JS. The asymmetry between the two backends is the surprising part: switching backend factory changes the packaging requirements of the application, which is not something a BackendFactory argument suggests.
Impact
This makes the PostgreSQL backend unusable in a compiled/bundled deployment without vendoring the library's private directory layout into the build. We are doing exactly that as a workaround — copying dist/esm/postgres into our runtime image behind a build-time assertion that the files still exist — but it encodes an internal path we have no right to depend on, and a patch release relocating those files would break it silently.
Suggested fix
Inline the SQL into the published JS, as the Lua scripts already are — a .sql → exported-string codegen step at build time. That removes the filesystem dependency entirely and needs no API change.
A smaller alternative, if inlining is unwanted: export the loader or accept a directory override on the connection options, so an application can point it at a path it controls.
Related
- #4601 —
maximumBlockTimeoutis hardcoded to 10s whenever a delayed job exists, so the PostgreSQL backend polls at least every 10s regardless ofdrainDelay. Same package, same backend, also from us.
Versions: [email protected], [email protected], PostgreSQL 16.
Source: taskforcesh/bullmq