Pre-bake cmakeBaseEnv and buildenv arch detection into discovery JSON
Background
loadPrediscovered() claims to be a fast-path: feed in a JSON, instantiate compilers without re-probing. In practice every compiler still runs through BaseCompiler.initialise(isPrediscovered=true), which keeps doing two things that hit the network/NFS:
getCmakeBaseEnv()(base-compiler.ts:365) — callsutils.fileExists()forld,ar, andasundertoolchainPath. For a compiler withtoolchainPathset this is 3 NFS round-trips per compiler. With ~6,000 prediscovered compilers and many having toolchain paths, that's a lot of stat traffic during boot — exactly the workload NFS is slowest at.buildenvsetup.initialise()(buildenvsetup/base.ts:68) — runshasSupportForArch()which callsexecCompilerCached(...)with--target-help/--help. That's an S3 GET againstcompiler-info-cacheper compiler withbuildenvsetup, plus a real exec on cache miss.compilerSupportsX86/compilerSupportsAMD64/compilerArchare the only side effects.
Neither of these depends on anything we can't compute during --discoveryonly. They're being recomputed every boot purely because the results aren't being persisted.
Proposal
Capture both during discovery and restore them in loadPrediscovered, gated on the field being present (so older discovery JSON keeps working).
Step-by-step plan
Profile boot first. Instrument
BaseCompiler.initialise()to log time spent ingetCmakeBaseEnv()andbuildenvsetup.initialise()across all compilers. Run on a real prod-shaped node with prediscovered JSON. Confirm this is actually a meaningful chunk of boot time before optimising it. (If it's not, close this and move on.)Extend
CompilerInfointypes/compiler.interfaces.tswith two optional fields:cachedCmakeBaseEnv?: Record<string, string>cachedBuildEnvSetup?: { compilerSupportsX86: boolean; compilerSupportsAMD64: boolean; compilerArch: string | false; defaultLibCxx: string }
Populate during discovery. In
BaseCompiler.initialise()non-prediscovered path:- After computing
this.cmakeBaseEnv = await this.getCmakeBaseEnv(), store it onthis.compiler.cachedCmakeBaseEnv. - After
buildenvsetup.initialise(), snapshot the resultingcompilerSupportsX86/AMD64/compilerArch/defaultLibCxxontothis.compiler.cachedBuildEnvSetup.
- After computing
Persist via
handleDiscoveryOnlyMode(compiler-discovery.ts:113) — these fields are on thecompilerobject so they'll serialise automatically. Verify with a manual run.Restore in prediscovered path.
- In
BaseCompiler.initialise(isPrediscovered=true): ifcachedCmakeBaseEnvexists, setthis.cmakeBaseEnvdirectly and skipgetCmakeBaseEnv(). - In
buildenvsetup.initialise()(or wrap the call site): ifcachedBuildEnvSetupexists, populate the four fields directly and skiphasSupportForArch(). Cleanest place is probably a new branch inBaseCompiler.initialise()that takes the cached values and seedsthis.buildenvsetupbefore calling itsinitialise(), withBuildEnvSetupBase.initialise()made a no-op when arch is already known (it already early-returns whenthis.compilerArchis set). - Strip both fields from the live
CompilerInfoafter restoration to match current behaviour (similar to howcachedPossibleArgumentsisdeleted).
- In
Backwards compatibility. Discovery JSON without the new fields must still boot — code branches on field presence. No format version needed.
Tests.
- Unit:
loadPrediscoveredwithcachedCmakeBaseEnvskipsfileExists; withcachedBuildEnvSetupskipshasSupportForArch. - Unit: discovery output now includes both fields for relevant compilers.
- Integration sanity: discovery → prediscovered roundtrip yields equivalent compiler state.
- Unit:
Measure. Re-run the profiling from step 1; confirm the saved time matches expectations.
Out of scope
- Possible later wins (sidecar-per-sqfs, format dedupe, eliminating the runner entirely) are tracked separately.
- This is purely "stop redoing work we already did during discovery."
Files likely to touch
types/compiler.interfaces.tslib/base-compiler.ts(initialise,getCmakeBaseEnvcallers)lib/buildenvsetup/base.ts(early-out path)lib/app/compiler-discovery.ts(no changes if step 3 puts fields on the right object)test/base-compiler-tests.tsand friends
Open questions
- Are there other quiet probes inside
initialise()worth caching at the same time?populatePossibleOverrides()andpopulatePossibleRuntimeTools()are already skipped for prediscovered, andargParser.parse()likewise — so the audit is mostly about whetherbuildenvsetup.initialise()has subclasses (ceconan, etc.) that also do per-boot work worth caching.
Source: compiler-explorer/compiler-explorer