From 36e1de6ecf2f0e23d36881385c147ab5c65719b7 Mon Sep 17 00:00:00 2001 From: Joyee Cheung Date: Thu, 29 Aug 2024 01:22:57 +0200 Subject: [PATCH] src: add JS APIs for compile cache and NODE_DISABLE_COMPILE_CACHE This patch adds the following API for tools to enable compile cache dynamically and query its status. - module.enableCompileCache(cacheDir) - module.getCompileCacheDir() In addition this adds a NODE_DISABLE_COMPILE_CACHE environment variable to disable the code cache enabled by the APIs as an escape hatch to avoid unexpected/undesired effects of the compile cache (e.g. less precise test coverage). When the module.enableCompileCache() method is invoked without a specified directory, Node.js will use the value of the NODE_COMPILE_CACHE environment variable if it's set, or defaults to `path.join(os.tmpdir(), 'node-compile-cache')` otherwise. Therefore it's recommended for tools to call this method without specifying the directory to allow overrides. PR-URL: https://github.com/nodejs/node/pull/54501 Fixes: https://github.com/nodejs/node/issues/53639 Reviewed-By: Benjamin Gruenbaum Reviewed-By: James M Snell Reviewed-By: Antoine du Hamel --- doc/api/cli.md | 34 ++-- doc/api/module.md | 157 +++++++++++++++++- lib/internal/modules/helpers.js | 54 +++++- lib/module.js | 8 + src/compile_cache.cc | 8 +- src/compile_cache.h | 7 +- src/env.cc | 15 +- src/node_modules.cc | 45 +++++ test/fixtures/compile-cache-wrapper.js | 21 +++ test/parallel/test-compile-cache-api-env.js | 81 +++++++++ .../test-compile-cache-api-permission.js | 56 +++++++ .../test-compile-cache-api-success.js | 79 +++++++++ .../parallel/test-compile-cache-api-tmpdir.js | 92 ++++++++++ test/parallel/test-compile-cache-disable.js | 39 +++++ 14 files changed, 665 insertions(+), 31 deletions(-) create mode 100644 test/fixtures/compile-cache-wrapper.js create mode 100644 test/parallel/test-compile-cache-api-env.js create mode 100644 test/parallel/test-compile-cache-api-permission.js create mode 100644 test/parallel/test-compile-cache-api-success.js create mode 100644 test/parallel/test-compile-cache-api-tmpdir.js create mode 100644 test/parallel/test-compile-cache-disable.js diff --git a/doc/api/cli.md b/doc/api/cli.md index 0b5e061482a210..e76499d914a664 100644 --- a/doc/api/cli.md +++ b/doc/api/cli.md @@ -2868,25 +2868,8 @@ added: v22.1.0 > Stability: 1.1 - Active Development -When set, whenever Node.js compiles a CommonJS or a ECMAScript Module, -it will use on-disk [V8 code cache][] persisted in the specified directory -to speed up the compilation. This may slow down the first load of a -module graph, but subsequent loads of the same module graph may get -a significant speedup if the contents of the modules do not change. - -To clean up the generated code cache, simply remove the directory. -It will be recreated the next time the same directory is used for -`NODE_COMPILE_CACHE`. - -Compilation cache generated by one version of Node.js may not be used -by a different version of Node.js. Cache generated by different versions -of Node.js will be stored separately if the same directory is used -to persist the cache, so they can co-exist. - -Caveat: currently when using this with [V8 JavaScript code coverage][], the -coverage being collected by V8 may be less precise in functions that are -deserialized from the code cache. It's recommended to turn this off when -running tests to generate precise coverage. +Enable the [module compile cache][] for the Node.js instance. See the documentation of +[module compile cache][] for details. ### `NODE_DEBUG=module[,…]` @@ -2908,6 +2891,17 @@ added: v0.3.0 When set, colors will not be used in the REPL. +### `NODE_DISABLE_COMPILE_CACHE=1` + + + +> Stability: 1.1 - Active Development + +Disable the [module compile cache][] for the Node.js instance. See the documentation of +[module compile cache][] for details. + ### `NODE_EXTRA_CA_CERTS=file` + +> Stability: 1.1 - Active Development + +The following constants are returned as the `status` field in the object returned by +[`module.enableCompileCache()`][] to indicate the result of the attempt to enable the +[module compile cache][]. + + + + + + + + + + + + + + + + + + + + + + +
ConstantDescription
ENABLED + Node.js has enabled the compile cache successfully. The directory used to store the + compile cache will be returned in the directory field in the + returned object. +
ALREADY_ENABLED + The compile cache has already been enabled before, either by a previous call to + module.enableCompileCache(), or by the NODE_COMPILE_CACHE=dir + environment variable. The directory used to store the + compile cache will be returned in the directory field in the + returned object. +
FAILED + Node.js fails to enable the compile cache. This can be caused by the lack of + permission to use the specified directory, or various kinds of file system errors. + The detail of the failure will be returned in the message field in the + returned object. +
DISABLED + Node.js cannot enable the compile cache because the environment variable + NODE_DISABLE_COMPILE_CACHE=1 has been set. +
+ +### `module.enableCompileCache([cacheDir])` + + + +> Stability: 1.1 - Active Development + +* `cacheDir` {string|undefined} Optional path to specify the directory where the compile cache + will be stored/retrieved. +* Returns: {Object} + * `status` {integer} One of the [`module.constants.compileCacheStatus`][] + * `message` {string|undefined} If Node.js cannot enable the compile cache, this contains + the error message. Only set if `status` is `module.constants.compileCacheStatus.FAILED`. + * `directory` {string|undefined} If the compile cache is enabled, this contains the directory + where the compile cache is stored. Only set if `status` is + `module.constants.compileCacheStatus.ENABLED` or + `module.constants.compileCacheStatus.ALREADY_ENABLED`. + +Enable [module compile cache][] in the current Node.js instance. + +If `cacheDir` is not specified, Node.js will either use the directory specified by the +[`NODE_COMPILE_CACHE=dir`][] environment variable if it's set, or use +`path.join(os.tmpdir(), 'node-compile-cache')` otherwise. For general use cases, it's +recommended to call `module.enableCompileCache()` without specifying the `cacheDir`, +so that the directory can be overriden by the `NODE_COMPILE_CACHE` environment +variable when necessary. + +Since compile cache is supposed to be a quiet optimization that is not required for the +application to be functional, this method is designed to not throw any exception when the +compile cache cannot be enabled. Instead, it will return an object containing an error +message in the `message` field to aid debugging. +If compile cache is enabled successefully, the `directory` field in the returned object +contains the path to the directory where the compile cache is stored. The `status` +field in the returned object would be one of the `module.constants.compileCacheStatus` +values to indicate the result of the attempt to enable the [module compile cache][]. + +This method only affects the current Node.js instance. To enable it in child worker threads, +either call this method in child worker threads too, or set the +`process.env.NODE_COMPILE_CACHE` value to compile cache directory so the behavior can +be inheritend into the child workers. The directory can be obtained either from the +`directory` field returned by this method, or with [`module.getCompileCacheDir()`][]. + +#### Module compile cache + + + +The module compile cache can be enabled either using the [`module.enableCompileCache()`][] +method or the [`NODE_COMPILE_CACHE=dir`][] environemnt variable. After it's enabled, +whenever Node.js compiles a CommonJS or a ECMAScript Module, it will use on-disk +[V8 code cache][] persisted in the specified directory to speed up the compilation. +This may slow down the first load of a module graph, but subsequent loads of the same module +graph may get a significant speedup if the contents of the modules do not change. + +To clean up the generated compile cache on disk, simply remove the cache directory. The cache +directory will be recreated the next time the same directory is used for for compile cache +storage. To avoid filling up the disk with stale cache, it is recommended to use a directory +under the [`os.tmpdir()`][]. If the compile cache is enabled by a call to +[`module.enableCompileCache()`][] without specifying the directory, Node.js will use +the [`NODE_DISABLE_COMPILE_CACHE=1`][] environment variable if it's set, or defaults +to `path.join(os.tmpdir(), 'node-compile-cache')` otherwise. To locate the compile cache +directory used by a running Node.js instance, use [`module.getCompileCacheDir()`][]. + +Currently when using the compile cache with [V8 JavaScript code coverage][], the +coverage being collected by V8 may be less precise in functions that are +deserialized from the code cache. It's recommended to turn this off when +running tests to generate precise coverage. + +The enabled module compile cache can be disabled by the [`NODE_DISABLE_COMPILE_CACHE=1`][] +environment variable. This can be useful when the compile cache leads to unexpected or +undesired behaviors (e.g. less precise test coverage). + +Compilation cache generated by one version of Node.js can not be reused by a different +version of Node.js. Cache generated by different versions of Node.js will be stored +separately if the same base directory is used to persist the cache, so they can co-exist. + +### `module.getCompileCacheDir()` + + + +> Stability: 1.1 - Active Development + +* Returns: {string|undefined} Path to the [module compile cache][] directory if it is enabled, + or `undefined` otherwise. + ### `module.isBuiltin(moduleName)`