fix: write transpiled config files to node_modules temp dir#3257
fix: write transpiled config files to node_modules temp dir#3257RobertoNegro wants to merge 1 commit into
Conversation
|
Thanks for opening this PR — makes sense! Claude PR review [click me]Claude PR review —
|
| Concern | Latest Vite (config.ts) |
Vike PR (transpileAndExecuteFile.ts) |
|---|---|---|
| Temp dir name | .vite-temp/ under nearest node_modules |
.vike-temp/ — identical pattern |
findNearestNodeModules |
packages/vite/src/node/packages.ts:239 |
transpileAndExecuteFile.ts:523 — same algorithm (tryStatSync, walk to root, stop at fixed point) |
| Injected var names | __vite_injected_original_{dirname,filename,import_meta_url,import_meta_resolve} |
__vike_injected_original_* — same shape |
define map |
__dirname, __filename, import.meta.url, import.meta.dirname, import.meta.filename, import.meta.resolve, import.meta.main: 'false' |
identical set |
| Plugin name | 'inject-file-scope-variables' |
'vike:inject-file-scope-variables' |
| Shebang preservation | line 2470: detects #!, splits at first newline, injects after |
identical, injectFileScopeVariables at line 451 |
| ESM temp-file flow | write → await import(pathToFileURL(...)) → unlink |
identical |
Fallback when node_modules write fails |
write next to source with <basename>.<hash>.mjs |
same |
| Tree-shaking | treeshake: false with comment about dead-code elimination |
treeShaking: false with the same justification |
Where the PR diverges
1. Filename hash.
- Vite (
config.ts:2577):timestamp-${Date.now()}-${Math.random().toString(16).slice(2)}— fresh per run. - PR (
transpileAndExecuteFile.ts:472-476):sha256(absPath + '\0' + code).slice(0, 12)— deterministic, content-addressed.
This is intentional, not an oversight — I was wrong to flag it in an earlier draft. Node's ESM loader caches modules by resolved URL. With a content-addressed filename, repeat loads of identical content hit Node's in-memory module cache and skip both disk I/O and code re-execution. Edits to the config (or to any module that ends up inlined in the bundle) change the hash → new URL → cache miss → fresh execution. The unlinkSync in finally is fine because Node already has the module cached by URL; subsequent import()s of the same URL never touch disk.
Vite went the opposite direction — fresh URL on every load, plus ?t=${Date.now()} in nativeImportConfigFile — because Vite explicitly wants config side effects to re-run each time. Vike is opting out of that for the caching benefit, which is the right call for the Vike workflow (many +config files, repeated transpiles during dev).
Worth a one-line comment in the source making this intent explicit, since the choice is unobvious and easy to "fix" the wrong way (as I initially proposed). Something like: // Content-addressed filename → identical content reuses Node's import() cache; mutated content gets a fresh URL.
Narrow theoretical concern (not blocking): two concurrent transpiles of identical content where one process's unlinkSync runs before the other's import() resolves. POSIX keeps deleted files alive via open fd; Windows may surface EBUSY. Config loading isn't a high-contention path, so this is unlikely to bite in practice. The existing try {} catch {} around unlinkSync doesn't help here — the failure would be in the import() itself — but again, narrow.
2. Source-map sources.
- Latest Vite (
config.ts:2494-2496): uses rolldown'ssourcemapPathTransform(relative) => path.resolve(fileName, relative)to emit absolute source paths in the source map itself. NosourceRootat all. - Vite 7.3.1 (in the dist I checked first): used
sourceRoot: pathToFileURL(...).href + '/'. - PR: sets
sourceRoot: \${entryFileDir}/`*and then* post-processes the inline source map (normalizeInlineSourceMapSources,transpileAndExecuteFile.ts:421-440) to rewrite eachsources[i]to an absolute filesystem path and **deletesourceRoot`**.
The end state of the PR (absolute paths in sources[], no sourceRoot) is the same outcome the latest Vite achieves via sourcemapPathTransform. The PR's approach is heavier (parse base64 → JSON → mutate → JSON → base64) but unavoidable: esbuild doesn't expose a per-source rewrite hook the way rolldown does. This is fine.
Note on normalizeSourceMapSource (line 442): it handles file://, absolute, sourceRoot (URL form), sourceRoot (path form), and a no-sourceRoot fallback. That's a lot of branches given that sourceRoot is always set in the PR's esbuild config (${entryFileDir}/) to a plain absolute path. The file:// and URL-form-sourceRoot branches are dead code today. Either trim them or leave a comment explaining they're hedges against future esbuild behavior changes.
3. import.meta.resolve.
- Vite (
config.ts:2457-2467, plus thecreateImportMetaResolverhelper): registers a Nodemodule.registerHooks/module.registercustomization hook soimport.meta.resolveinside config files respects Vite's own resolver (importMetaResolveWithCustomHookString). CJS path throws explicitly. - PR (
transpileAndExecuteFile.ts:182-189, banner): banner-injects helpers that callcreateRequire(importer).resolve(specifier)with manual URL handling for./,/, andfile:specifiers.
The PR's helper is correct for the common case but loses Vite-style resolver fidelity (no conditions, no plugin-aware resolution). For Vike's +config.ts use case this is probably fine — config files shouldn't be doing exotic resolution — but it's worth a comment in the source acknowledging the trade-off. Also, it always installs the resolver regardless of whether the user code uses import.meta.resolve, whereas Vite only registers the hook when the regex matches (importMetaResolveRegex.test(code)).
4. Banner vs per-file injection.
- Vite: per-file
injectValuesstring, gated onimportMetaResolveRegex.test(code). - PR: declares helpers (
__vike_createRequire,__vike_pathToFileURL,__vike_import_meta_resolve) once viabanner.js, then per-fileinjectValuesreferences them.
The banner approach produces smaller output and is a nice design choice, but it requires registering node:module and node:url as non-pointer imports (transpileAndExecuteFile.ts:204-205). That's done correctly. One subtle thing: the banner unconditionally imports node:module and node:url even when the config never uses import.meta.resolve. Negligible in practice.
5. Fallback trigger.
- Vite (
config.ts:2568-2575): catchesEACCESfrommkdironly; throws everything else. - PR (
transpileAndExecuteFile.ts:485, 541-552): catchesEACCES | EPERM | EROFS | ENOENT | ENOTDIR | EISDIRfromwriteFileSync. Broader and more pragmatic — handles read-onlynode_modules(EROFS), symlinked-to-file cases (ENOTDIR), etc.
Vike's set is the better one. No change needed.
6. onLoad filter.
- Vite (
config.ts:2449):/\.[cm]?[jt]s$/. - PR (
transpileAndExecuteFile.ts:178):/\.[cm]?[jt]sx?$/plus an explicitgetEsbuildLoaderswitch returning'ts' | 'tsx' | 'jsx' | 'js'.
Vike picks up .tsx/.jsx, which Vite's config loader doesn't. Reasonable extension for Vike (configs can be .tsx in some setups).
7. ESM-only.
- Vite: full CJS path via
_require.extensionspatching (config.ts:2592-2613). - PR: forces
format: 'esm'. No CJS path.
Acceptable for Vike's modern target. Just be aware that any user with a CJS +config.cts would now have it bundled to ESM and executed via dynamic import; this might already work because esbuild handles the CJS→ESM transform, but worth a test if not covered.
8. Dependency tracking.
- Vite (latest): uses rolldown chunks +
collectAllModulesto walkchunk.moduleIds/chunk.imports/chunk.dynamicImports. - Vite (esbuild era): used
result.metafile.inputs. - PR: populates
esbuildCache.vikeConfigDependenciesinside theonLoadcallback itself — every file that hitsonLoadis added.
The PR's approach is simpler and works because onLoad runs for every transitively loaded file. One caveat: files that get externalized never hit onLoad and won't be tracked. Vite's metafile-based approach captures external bundle dependencies too. Not necessarily a bug for Vike — externalized deps are typically node_modules content that shouldn't trigger config invalidation — but worth being explicit about.
9. Cleanup.
- Vite:
fs.unlink(tempFileName, () => {})— async, fire-and-forget. - PR:
fs.unlinkSync(filePathTmp)wrapped intry {} catch {}, in afinally.
Both fine; the sync version blocks the caller momentarily. Not material.
Code-quality observations specific to the PR
Duplication: executeTranspiledFile and executeTranspiledFileNextToSource. These two functions (transpileAndExecuteFile.ts:496 and :514) share the entire post-write block (executeFile + cleanup + finally). After the .vike-temp write succeeds in the first, control flows through the same try/finally. Worth refactoring to one function that takes a filePathTmp argument and a "cleanup" closure. As written, two near-identical blocks risk drifting (one already differs slightly: the second uses getTemporaryBuildFilePath, the first inlines the equivalent).
pointerImports.ts change. Dropping the trailing \n after constDeclarations.join('') is correct for the new flow: the constants are prepended after esbuild emitted the source map, so any extra newline shifts every original line by one. The added comment at lines 47-48 explains this well. The test additions ('does not add lines when transforming pointer imports', 'preserves non-pointer imports') are good regression coverage.
Test coverage. The new transpileAndExecuteFile.spec.ts covers:
- source-map sources for main file (no
sourceRoot, contains original path) - source-map sources for transitively-imported files
- file-scope variables (
__dirname,__filename, all 5import.meta.*) EROFSfallback to next-to-source
Solid. One additional test worth considering: a regression check that loading the same config file twice in the same process reuses Node's module cache (assert the file body is only evaluated once), which would lock in the content-hash design intent.
Spy-based assertions on fs.writeFileSync. The tests reach into vi.spyOn(fs, 'writeFileSync') and search the call list for '/node_modules/.vike-temp/+config.ts.build-'. This couples tests to the path format; if you change the directory name later, you update tests in two places. Could read from the filesystem directly after a successful run instead, but the current approach also lets you catch the EROFS mock cleanly, so net-net it's fine.
Suggested follow-ups (in order of impact)
- Deduplicate
executeTranspiledFileandexecuteTranspiledFileNextToSourceinto one function with a path argument. - Document the content-hash design with a one-line comment so the next contributor doesn't "fix" it back to a timestamp.
- Gate
import.meta.resolveinjection on actual usage (mirror Vite'simportMetaResolveRegex.test(code)), or note in a comment why you don't. - Trim dead branches in
normalizeSourceMapSource(thefile://and URL-form-sourceRootbranches are unreachable given the current esbuild config) or document them as future-proofing. - Optional, longer-term: track whether the rolldown migration in upstream Vite is worth following.
sourcemapPathTransformcleanly replaces thenormalizeInlineSourceMapSourcespost-process, and rolldown'scollectAllModulesis a more honest dependency story than theonLoad-set approach. None of this is urgent — current esbuild-based code is fine.
brillout review
- Source-map sources rewriting is Vike's own approach to a problem Vite now solves with
sourcemapPathTransform— Vike's post-processing achieves the same end result, but it's bolted on rather than handed to the bundler. Acceptable because esbuild doesn't expose that hook; just note that the latest Vite has moved to rolldown specifically partly to get this.
I'm 👍 for using Rolldown instead of esbuild.
- Two near-clones of the same logic now live in
transpileAndExecuteFile.ts(executeTranspiledFileandexecuteTranspiledFileNextToSource). Worth deduping.
👍 Let's dedupe.
Vite (
config.ts:2457-2467, plus thecreateImportMetaResolverhelper): registers a Nodemodule.registerHooks/module.registercustomization hook soimport.meta.resolveinside config files respects Vite's own resolver (importMetaResolveWithCustomHookString).
I wonder whether should also use module.registerHooks / module.register? Would that be more reliable than statically overwriting import.meta.resolve?
Also, it always installs the resolver regardless of whether the user code uses
import.meta.resolve, whereas Vite only registers the hook when the regex matches (importMetaResolveRegex.test(code)).
It it avoids loading node:module and node:url then it's maybe worth it. Vite probably did it conditionally for a reason.
Vite apps typically have only one config file, but Vike apps can and usually have many. Therefore:
- Shouldn't we pass
config.rootas argument tofindNearestNodeModules()instead of the config file path? - Let's add the config file path (relative to
config.root, and normalizedreplaceAll('/', '_')) to the temporary file name, instead of just its basename => makes debbugging easier.
There doesn't seem to be a perfect approach for config file transpilation & execution — seems like there are inherent trade-offs. Therefore, Vike will probably need to a new setting +configLoader similar to Vite's --configLoader. But let's merge first and see if users complain about this new approach.
2 participants
Summary
This PR changes how Vike executes transpiled config files:
<nearest-node_modules>/.vike-temp/instead of the application source tree when possible;.vike-tempcannot be used;import.meta.*,__dirname, and__filename;Motivation
When Vike runs under Bun
--hot, writing transient transpiled config artifacts inside the source tree can trigger Bun's hot reload watcher while the Vike/Vite config graph is still being linked.The failure has been observed on cold starts with an empty Vite cache:
[vike] Failed to execute .../node_modules/vike-react/dist/config.js because: TypeError: Requested module is not instantiated yet. at link (native) at linkAndEvaluateModule (native) at requestImportModule (native) at processTicksAndRejections (native)After this failure, Vike can continue with an invalid config state and frontend pages render as 500.
The reliable mitigation is to keep the generated config artifact outside watched application source directories:
The nearest
node_modulesdirectory is selected by walking upward from the source config file directory. This keeps bare import resolution aligned with the project/package dependency tree while avoiding source-tree watcher events for transient generated files.Implementation
Generated config artifacts are written to:
The hash includes both the source file path and generated code:
This avoids collisions between distinct config files that transpile to identical code.
If no suitable
node_modulesdirectory exists, or if.vike-tempcannot be created or written, Vike uses the existing next-to-source generated artifact path. Expected filesystem failures such as permission errors, read-only directories, and missing path components are treated as fallback conditions.The generated file remains short-lived: Vike writes it, imports it, and removes it in
finally.Original File Semantics
Moving the generated
.mjsartifact changes the physical runtime location of the imported file. Config code can depend on file-scoped values such as:This PR preserves the original config source file as the semantic location for these values.
The implementation injects per-file constants for the original source directory, filename, and file URL, then rewrites supported file-scoped expressions before bundling.
import.meta.resolve(specifier)resolves against the original importer URL. Because the rewrite is parser/transform based, formatted expressions such as:are handled as well.
Example:
With this PR,
./config-data.txtresolves relative to the original+config.ts, even though the generated artifact is imported from.vike-temp.Source Maps
The bundled config artifact includes an inline source map whose
sourcespoint to the original config files. Vike asks esbuild to anchor the source map to the original config directory, then normalizes the final inline source map sosourcesare filesystem-absolute. This makes source-map metadata independent from the generated artifact location under.vike-temp, while remaining compatible with Vike's existingsource-map-supportstack handling.File-scoped values are injected through esbuild's
onLoad()hook before the final bundle is generated, so the final inline source map is produced by the same esbuild build that emits the config artifact.Pointer imports are transformed after bundling. The transform preserves the generated line count by avoiding an extra prepended line, so the inline source map continues to describe the code that Vike imports from
.vike-temp.One source-map limitation remains for column precision on the first line of code in each transformed module. The injected file-scoped variables are prepended without adding a newline, so line numbers remain stable, while the first line's columns are offset by the injected prefix. Stack traces still point to the original source file and line, including relative modules bundled into the config artifact.
Vite Alignment
This PR follows Vite's bundled config loader because Vite faced the same class of problems and resolved them through several focused iterations. The implementation mirrors the relevant Vite changes and uses the same resolution strategy where it applies to Vike, so config temp-file execution, source-map metadata, file-scoped values, and filesystem fallback behavior are handled as reliably as possible.
Vite docs: bundled temporary config files are written under
node_modules/.vite-temp.https://vite.dev/config/#debugging-the-config-file-on-vs-code
fix(config): write temporary vite config to node_modules vitejs/vite#18509,
fix(config): write temporary vite config to node_modulesMoves bundled config artifacts to
node_modules/.vite-tempand discusses compatibility concerns around relative references, dependency resolution, workspace packages, andimport.meta.*shims.fix(config): make stacktrace path correct when sourcemap is enabled vitejs/vite#18833,
fix(config): make stacktrace path correct when sourcemap is enabledKeeps stack traces/source-map paths aligned with the original config source path even when the generated artifact is imported from
.vite-temp.fix(config): make debugger work with bundle loader vitejs/vite#20573, commit
c583927bee657f15f63fdf80468fbe6a74eacdec,fix(config): make debugger work with bundle loaderCovers the same source-map/debugger problem for Vite's esbuild config bundle loader. Vike uses the same principle of anchoring source maps to the original config directory while keeping the format compatible with Vike's runtime stack handling.
fix: no permission to create vite config file vitejs/vite#18844,
fix: no permission to create vite config fileHandles permission and read-only filesystem cases for config temp-file creation. This PR applies the same robustness principle by falling back when
.vike-tempcannot be used.feat: better config
__dirnamesupport vitejs/vite#8442, commit51e9195fe9,better config __dirname supportIntroduces injected variables such as
__vite_injected_original_import_meta_urlso bundled config code observes the original file URL.feat(config):
import.meta.filename/dirnamesupport vitejs/vite#15888, commit3efb1a11a0Extends config shimming to
import.meta.dirnameandimport.meta.filename.feat: add
import.meta.mainsupport in config (bundle config loader) vitejs/vite#20516, commit5d3e3c2aeCovers
import.meta.mainin bundled config code.feat: add
import.meta.resolvesupport for ESM config (bundle config loader) vitejs/vite#20962, commitf86789a6eCovers
import.meta.resolvein bundled config code.fix: detect
import.meta.resolvewhen formatted across multiple lines vitejs/vite#21312Fixes detection of
import.meta.resolvewhen the expression is formatted across multiple lines.Testing
Added unit coverage for:
transformPointerImports()preserving non-pointer banner imports such asnode:modulewhile still transforming pointer imports in the same file;transformPointerImports()transforming pointer imports without adding generated lines after esbuild has emitted the source map;transpileAndExecuteFile()writing the generated config artifact undernode_modules/.vike-temp;transpileAndExecuteFile()falling back to the next-to-source artifact path whennode_modules/.vike-tempcannot be written;import.meta.url,import.meta.dirname,import.meta.filename,import.meta.main,import.meta.resolve,__dirname, and__filenamewhen the generated artifact is imported from.vike-temp.+config.tspointing to the original config file;+config.tspointing to the original relative module.Runtime stack mapping was also checked outside Vitest, with
source-map-supportactive, for both a direct throw in+config.tsand a throw in a bundled relative module.The runtime check confirmed that stack traces point to the original source file and line. Column precision on the first line of transformed modules is not guaranteed because file-scoped variables are injected before that first line without adding a newline.
The full unit test suite passes, and the package build succeeds.
A manual dev-server check with
examples/react-fullreaches Vike/Vite ready state successfully.I am also using this patch successfully in a project currently under development. The server side, built with Elysia, updates quickly through Bun's
--hotreload, while the Vike side keeps working with Vite HMR as expected.