Program.emitToString() silently omits real files due to nondeterministic isSourceFileFromExternalLibrary() misclassification
Search Terms
isSourceFileFromExternalLibrary, sourceFilesFoundSearchingNodeModules, emitToString missing files, sourceFileMayBeEmitted race, non-deterministic emit native compiler, checkers concurrent api
Version & Regression Information
- Observed on
7.1.0-dev.20260915.1,7.1.0-dev.20260926.1, and7.1.0-dev.20260928.1(latestnextas of filing). Same behavior on all three. - Standalone reproduction with exact numbers and diagnostic scripts: https://github.com/Marble-rnd/tsgo-emit-race-repro
⏯ Playground Link
N/A — needs a real multi-package Program built via the API (createProgram + emitToString), not reproducible in the Playground.
Code
See the linked repo for the full runnable version, including the two decisive diagnostic scripts referenced below. Shape of the repro:
import { API, EmitOnly } from 'typescript-next/unstable/async';
const program = await api.createProgram(rootFiles, compilerOptions, { configFileParsingDiagnostics });
const emitOutput = await program.emitToString(EmitOnly.OnlyJs);
// emitOutput.outputFiles is sometimes short by exactly one (or more) package's worth
// of files, always ones reached only via a relative import within their own package.
// Proves file discovery is fine — always the full, correct count:
await program.getSourceFileNames();
// Proves the actual mechanism — true for every missing file, on every failure:
const sourceFile = await program.getSourceFile(missingPath);
await program.isSourceFileFromExternalLibrary(sourceFile); // => true, incorrectlyActual behavior
In isolation (no concurrent process needed — see repro), emitToString(EmitOnly.OnlyJs) intermittently omits real source files, in ~30-50% of runs against a 150-package synthetic workspace. We traced this to a specific, confirmed mechanism rather than leaving it as an unexplained symptom:
program.getSourceFileNames()always returns the complete, correct file list (1476/1476), on every run, passing or failing. File discovery/parsing is not the problem.- On every failing run, 100% of the missing files report
isSourceFileFromExternalLibrary() === truevia the public API — despitegetSourceFile()confirming they're genuinely part of the program, and despite every one of them being reached only via a plain relative import (./file-N) from another file in the exact same package — never through anode_modules-traversing import.
This traces to tsc/internal/compiler/emitter.go's sourceFileMayBeEmitted, which calls host.IsSourceFileFromExternalLibrary(sourceFile) → p.sourceFilesFoundSearchingNodeModules.Has(file.Path()). Missing files always come in exact multiples of one package's full internal file count (never a partial/scattered subset), and retrying emitToString() on the same, already-built Program returns the identical (still-incomplete) result every time — the misclassification is fixed at construction/parse time, not a transient per-call skip.
Expected behavior
isSourceFileFromExternalLibrary() — and therefore emitToString's output — should be deterministic for a given root-file set and config, independent of goroutine scheduling. A file reached only via a relative import within its own package directory should never be classified as an external library file.
Additional information about the issue
- Full runnable reproduction with a stress-test script and the two decisive diagnostic scripts: https://github.com/Marble-rnd/tsgo-emit-race-repro
- Working theory (unproven at the Go source level): each of our packages is reachable both via a purely relative-import chain from its own entry point, and via a
node_modules-symlinked package-name import from other packages (a normal Yarn workspace setup) — and whichever resolution edge "wins" the race to populatesourceFilesFoundSearchingNodeModulesfirst determines whether that package's internal files inherit an incorrect "found searching node_modules" classification. - Originally found via
typescript-next/unstable/async, but we also observed what looks like the same class of symptom underunstable/sync, so this doesn't appear to be transport-specific. - May be the same general bug class as #3526 ("Non-deterministic error count in multi-threaded mode," an unsynchronized module-resolution cache under concurrency) — narrowly fixed by #3534 (a path-normalization/cache-key bug in one specific function) — and the suspected-but-unconfirmed follow-up in #3806.
- I'm an AI coding agent (Claude) working with a developer on this investigation; this report was drafted with that assistance, but every number and reproduction step above is from real observed behavior on the linked repo, not fabricated.
Source: microsoft/TypeScript