NodeFS ordered persistence and synchronization extension contract
We are evaluating PGlite 0.5.8 for a Node application and trying to establish the supported route for acknowledging persistent writes across an OS crash or power loss. We found gaps in the released NodeFS path and would appreciate guidance before building a custom VFS. We have not demonstrated naturally occurring database corruption or a real power-loss failure.
Version: @electric-sql/[email protected], upstream
ae182ff8bd5ba4acb887d6c925d607a1498aa0b5.
Source inspected from the published package and its embedded TypeScript maps:
- Startup defaults include
-F. NodeFS inherits an emptysyncToFs(). - Emitted
_fd_syncreturns success if a stream has no fsync callback; NODEFS has no such callback. Emitted fdatasync validates the stream and returns success without a host synchronization call. - NODEFS acquires native descriptors only for regular-file streams. Its replacement path attempts a virtual unlink before the host rename.
- The exported BaseFilesystem helper also lacks fsync/fdatasync operations.
Its generated open handler does not pass stream flags to
baseFS.open, its unlink handler suppresses errors, and its syncfs implementation is empty.
Public source references: startup,
NodeFS,
BaseFilesystem.
The inspected emitted dist/pglite.js has SHA256
d7db324430326d0a2189a4aa10a11214fd0c92fdbf0df5781bb57f947b491707.
Our existing synthetic traces used Node 24.20.0 on Windows/x64, Linux/x64 and macOS/arm64. They traced actual engine and host operations through bootstrap, migration, mutation, receipt reads, checkpoint and close. Positive host controls verified that synchronization hooks were observable. No complete persistence path was established. These were capability/fault probes, not a cache-loss model.
| Control | Observed result |
|---|---|
| Default NodeFS; explicit fsync settings | No complete host synchronization path established. An awaited hook alone does not establish a barrier. |
wal_sync_method=open_datasync |
Linux/Mac WAL opens carried Node's O_SYNC bit; Windows observed O_RDWR and exposed no Node O_SYNC/O_DSYNC constant. This only characterizes that tested path. |
wal_sync_method=open_sync |
Startup rejected in stock and traced controls on all three tuples. |
| Direct host directory synchronization | Succeeded on Linux/Mac; EPERM on Windows. This is not a claim that all Windows persistence mechanisms are unavailable. |
| Checkpoint replacement and injected rename EIO | Replacement was observed; crash atomicity was not established. The injected error also exposed a separately diagnosed protocol-health problem described below. |
After a deliberately injected checkpoint rename EIO, our raw vendor controls observed subsequent receipt SELECTs without Bind/command completion being represented as empty query results. Fresh-owner reopening after observed prior termination could read the earlier receipt. We now reject incomplete protocol observations and stop ordinary use of an unhealthy instance in our application. This is failure containment, not a repair of the persistent storage path or evidence that a rejected operation rolled back. We can separate this topic from the VFS question if that is easier to track.
We reviewed #1066, #1063 and #1089. They appear to improve synchronization scheduling/failure handling and close ordering without supplying the missing NodeFS file/namespace barriers.
Questions:
- Is there a supported Node filesystem implementation or planned upstream route that supplies complete ordered persistence, including bootstrap, WAL/data and namespace operations, on these platforms?
- Which extension seam should own flags, descriptor management, both fsync and fdatasync, error propagation and replacement semantics? Which compiled routes require upstream changes rather than a BaseFilesystem subclass?
- What persistence and maintenance contract does upstream intend for the Node backend, especially Windows namespace operations and close with pending I/O?
The source references and trace summary above are the available report. We have not attached our application's fixtures or a standalone reproduction. If useful, we can prepare a reduced vendor-only synthetic example for a specific path and verify it before sharing it.
Source: electric-sql/pglite