Unhandled rejection kills the process when the internal array-types query is cancelled (57014) - repro for #279
This is #279 - closed in 2022 for lack of a reproduction ("Let me know if you can repro this") - with a deterministic repro attached. The stack in that report already names the culprit frame (fetchArrayTypes -> ReadyForQuery).
Summary
When the server sends an ErrorResponse for the driver's internal array-types query - for example when that query is cancelled by statement_timeout or pg_cancel_backend (SQLSTATE 57014) - postgres.js emits an unhandledRejection that no application code can catch. Under Node's --unhandled-rejections=throw (the default on Vercel and in Node >= 15 for unhandled rejections) this kills the process, taking every in-flight request with it.
We hit this in production as repeated "whole site hangs" incidents: a statement gets cancelled, and the serverless instance dies with exit 128 instead of the one request failing.
Where it happens
src/connection.js, ReadyForQuery calls fetchArrayTypes() without awaiting it or attaching a handler:
if (needsTypes) {
initial.reserve && (initial = null)
return fetchArrayTypes() // connection.js:564 - promise dropped
} async function fetchArrayTypes() { // connection.js:768
needsTypes = false
const types = await new Query([`
select b.oid, b.typarray
...
`], [], execute)
types.forEach(({ oid, typarray }) => addArrayType(oid, typarray))
}If that Query rejects, the await throws and the plain promise returned by the async function rejects with no handler attached. Node reports exactly one unhandled rejection, top frame ErrorResponse (src/connection.js:815):
PostgresError: canceling statement due to statement timeout
at ErrorResponse (postgres/src/connection.js:815:30)
at handle (postgres/src/connection.js:489:6)
at Socket.data (postgres/src/connection.js:324:9)Because the rejected promise belongs to no caller query, adding .catch() to every application query does not help (verified by promise-identity checks - it is an internal promise, not one of ours).
fetch_types: false avoids it entirely (verified with the repro below: 1 unhandled rejection with the default fetch_types: true, 0 with it disabled), since fetchArrayTypes is the only consumer of needsTypes - but that gives up array parsing.
The same shape exists in fetchState() (query.execute() at connection.js:810 is also dropped, reachable with target_session_attrs), and in the unawaited async auth handlers called from handle().
Reproduction
Deterministic, no pooler or special server needed. The only trick is making the array-types catalog query slower than a 1 ms statement_timeout, which the script does by creating throwaway composite types:
docker run -d --name pgrepro -e POSTGRES_PASSWORD=pg -e POSTGRES_DB=repro -p 55433:5432 postgres:16
node repro.mjs 'postgres://postgres:[email protected]:55433/repro'import postgres from 'postgres'
const DBURL = process.argv[2]
const SCHEMA = 'pgjs_dangling_repro'
const ARRAY_TYPES_SQL = `
select b.oid, b.typarray
from pg_catalog.pg_type a
left join pg_catalog.pg_type b on b.oid = a.typelem
where a.typcategory = 'A'
group by b.oid, b.typarray
order by b.oid`
const setup = postgres(DBURL, { max: 1, fetch_types: false, onnotice: () => {} })
const time = async () => {
const t = performance.now()
await setup.unsafe(ARRAY_TYPES_SQL)
return performance.now() - t
}
// make the driver's internal array-types query take > 1ms
await setup.unsafe(`CREATE SCHEMA IF NOT EXISTS ${SCHEMA}`)
let created = 0, elapsed = await time()
while (elapsed < 5 && created < 30000) {
let ddl = ''
for (let i = created; i < created + 2000; i++) ddl += `CREATE TYPE ${SCHEMA}.t${i} AS (a int);`
await setup.unsafe(ddl)
created += 2000
elapsed = await time()
}
console.log(`array-types query: ${elapsed.toFixed(1)}ms (${created} helper types)`)
const unhandled = []
process.on('unhandledRejection', (r) => unhandled.push(r))
// fetch_types defaults to true, so connection setup runs fetchArrayTypes();
// statement_timeout makes the server cancel it mid-flight
const victim = postgres(DBURL, { max: 1, connection: { statement_timeout: '1' } })
try { await victim`SELECT 1 AS n` } catch (e) { console.log('caller query rejected with', e.code) }
await new Promise((r) => setTimeout(r, 500))
console.log('unhandled rejections:', unhandled.length)
unhandled.forEach((r) => console.log(String(r.stack).split('\n').slice(0, 3).join('\n')))
await victim.end({ timeout: 3 }).catch(() => {})
// cleanup in batches - DROP SCHEMA CASCADE over thousands of types exceeds max_locks_per_transaction
for (;;) {
const names = await setup`
SELECT t.typname FROM pg_catalog.pg_type t
JOIN pg_catalog.pg_namespace n ON n.oid = t.typnamespace
WHERE n.nspname = ${SCHEMA} AND t.typtype = 'c' LIMIT 500`
if (!names.length) break
await setup.unsafe(names.map(({ typname }) => `DROP TYPE ${SCHEMA}."${typname}" CASCADE;`).join(''))
}
await setup.unsafe(`DROP SCHEMA IF EXISTS ${SCHEMA} CASCADE`)
await setup.end({ timeout: 5 })Output on 3.4.9 (postgres 16, Node 26):
array-types query: 5.0ms (6000 helper types)
caller query rejected with 57014
unhandled rejections: 1
PostgresError: canceling statement due to statement timeout
at ErrorResponse (.../postgres/src/connection.js:815:30)
at handle (.../postgres/src/connection.js:489:6)Run the same script with node --unhandled-rejections=throw and the process dies.
Suggested fix
Give the dropped promise a handler and route the failure through the connection's normal error path:
async function fetchArrayTypes() {
needsTypes = false
- const types = await new Query([`
- ...
- `], [], execute)
- types.forEach(({ oid, typarray }) => addArrayType(oid, typarray))
+ try {
+ const types = await new Query([`
+ ...
+ `], [], execute)
+ types.forEach(({ oid, typarray }) => addArrayType(oid, typarray))
+ } catch (err) {
+ errored(err)
+ }
}Verified against the repro above, same machine and database, only the driver differing:
| unhandled rejections | caller query | array parsing | |
|---|---|---|---|
| 3.4.9 as published | 1 (fatal under --unhandled-rejections=throw) |
rejects with 57014 |
fine |
| 3.4.9 + diff above | 0 | rejects with 57014 |
fine (int[], text[] round-trip unchanged) |
We are running this as a local pnpm patch with the repro wired up as a regression test, and will drop it as soon as a release handles the cancellation path.
Happy to send this as a PR if the approach looks right - and to cover fetchState() and the unawaited auth handlers in the same pass.
Environment
- postgres 3.4.9
- Node 26.7.0 locally; Vercel's Node serverless runtime in production
- PostgreSQL 16 locally; Supabase/Supavisor in production
Same class of unhandled rejection as #252 and #162.
Source: porsager/postgres