Parcel MemoryFS significantly slower than default config (importing & large files)
bug report
When implementing an in-memory file system for Parcel (as per https://parceljs.org/features/parcel-api/#file-system), Parcel builds become significantly slower than the default configuration, even when caching is disabled to make for a fair comparison. There is a worked example in a repo below; I've attempted to follow the documentation as much as possible (noting the documentation needs fixing as per https://github.com/parcel-bundler/parcel/issues/9050#issuecomment-1700904511).
It appears that the issue relates to the use of import, which may in turn be related to the module loader or else file access under the hood. If an empty or basic file (e.g. console.log('foobar')) is built without any imports, then the in-memory implementation is vastly quicker (177ms vs 345ms). However, with each import, the in-memory implementation gets slower and slower. In the example below, just 1 import from a plugin from a library called amCharts takes 4x longer (7.3s vs 1.8s) in memory compared to the default.
As I mention below though, the issue isn't isolated to this one library. Mocking out a single JavaScript file full of noop lines of 1;, the default implementation is quicker than using MemoryFS.
Configuration (.babelrc, package.json, cli command)
See https://gist.github.com/davidjb/1f3c090b7bae52fecc0a6eac57e32db2 for a worked example. To run:
git clone https://gist.github.com/1f3c090b7bae52fecc0a6eac57e32db2.git example
cd example
npm i
npm tIt boils down to:
const bundler = new Parcel({
entries: 'a.js',
defaultConfig: '@parcel/config-default',
shouldDisableCache: true,
});
await bundler.run();running a lot quicker than:
const workerFarm = createWorkerFarm();
const outputFS = new MemoryFS(workerFarm);
const bundler = new Parcel({
entries: 'a.js',
defaultConfig: '@parcel/config-default',
shouldDisableCache: true,
workerFarm,
outputFS,
});
await bundler.run();
await workerFarm.end();Expected Behavior
Ideally, the in-memory implementation would be faster, otherwise it negates its reason for being. Otherwise, if there are situations where the MemoryFS implementation is known to be slower, these should be documented so it's clear when MemoryFS is a better choice.
Current Behavior
$ npm t
> test
> NODE_OPTIONS=--experimental-vm-modules npx jest
(node:52151) ExperimentalWarning: VM Modules is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
(node:52151) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
PASS ./build.test.mjs (9.304 s)
✓ run() - default options (1806 ms)
✓ run() - in memory (7342 ms)
Test Suites: 1 passed, 1 total
Tests: 2 passed, 2 total
Snapshots: 0 total
Time: 9.318 s, estimated 10 s
Ran all test suites.That said, this is time to first build in a cold environment. Once having built initially and bundler/workers/FS are already in existence, a re-build is faster on the in-memory implementation by a measurable amount compared to the defaults (e.g. 650ms vs 750ms).
Possible Solution
Context
I'm attempting to speed up builds by using memory compared to using the file system.
Code Sample
See https://gist.github.com/davidjb/1f3c090b7bae52fecc0a6eac57e32db2
As a more contrived test, by replacing a.js with just a lot of lines like so: 1;, the default configuration's build time increases somewhat linearly, but the in-memory implementation is consistently worse after a.js reaches a few 1000 lines of code, and gets progressively worse. For a 10MB .js file full of lines of 1;, the results look this:
./build.test.mjs (7.162 s)
✓ run() - default options (2742 ms)
✓ run() - in memory (4233 ms)Your Environment
| Software | Version(s) |
|---|---|
| Parcel | 2.10.3 |
| Node | 21.1.0 |
| npm/Yarn | 10.2.0 |
| Operating System | macOS but also observed on AL2023 |
Source: parcel-bundler/parcel