Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

async_hooks: buffer bootstrap events #29848

Closed
wants to merge 10 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions doc/api/async_hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ specifics of all functions that can be passed to `callbacks` is in the
const async_hooks = require('async_hooks');

const asyncHook = async_hooks.createHook({
init(asyncId, type, triggerAsyncId, resource) { },
init(asyncId, type, triggerAsyncId, resource, bootstrap) { },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other way we could do this would be to use a special trigger ID for bootstrap

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By special ID do you mean something like making it -1 for the bootstrap?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, something along those lines. Rather than negative tho, since Node.js is the one assigning the numbers, we could just say that a triggerAsyncId of 1 === bootstrap.

destroy(asyncId) { }
});
```
Expand All @@ -116,7 +116,7 @@ The callbacks will be inherited via the prototype chain:

```js
class MyAsyncCallbacks {
init(asyncId, type, triggerAsyncId, resource) { }
init(asyncId, type, triggerAsyncId, resource, bootstrap) { }
destroy(asyncId) {}
}

Expand Down Expand Up @@ -203,14 +203,16 @@ Key events in the lifetime of asynchronous events have been categorized into
four areas: instantiation, before/after the callback is called, and when the
instance is destroyed.

##### init(asyncId, type, triggerAsyncId, resource)
##### init(asyncId, type, triggerAsyncId, resource, bootstrap)

* `asyncId` {number} A unique ID for the async resource.
* `type` {string} The type of the async resource.
* `triggerAsyncId` {number} The unique ID of the async resource in whose
execution context this async resource was created.
* `resource` {Object} Reference to the resource representing the async
operation, needs to be released during _destroy_.
* `bootstrap` {boolean} Indicates whether this event was created during Node.js
bootstrap.

Called when a class is constructed that has the _possibility_ to emit an
asynchronous event. This _does not_ mean the instance must call
Expand Down Expand Up @@ -319,8 +321,13 @@ elaborate to make calling context easier to see.

```js
let indent = 0;
const bootstrapIds = new Set();
async_hooks.createHook({
init(asyncId, type, triggerAsyncId) {
init(asyncId, type, triggerAsyncId, resource, bootstrap) {
if (bootstrap) {
bootstrapIds.add(asyncId);
return;
}
const eid = async_hooks.executionAsyncId();
const indentStr = ' '.repeat(indent);
fs.writeSync(
Expand All @@ -329,18 +336,21 @@ async_hooks.createHook({
` trigger: ${triggerAsyncId} execution: ${eid}\n`);
},
before(asyncId) {
if (bootstrapIds.has(asyncId)) return;
const indentStr = ' '.repeat(indent);
fs.writeFileSync('log.out',
`${indentStr}before: ${asyncId}\n`, { flag: 'a' });
indent += 2;
},
after(asyncId) {
if (bootstrapIds.has(asyncId)) return;
indent -= 2;
const indentStr = ' '.repeat(indent);
fs.writeFileSync('log.out',
`${indentStr}after: ${asyncId}\n`, { flag: 'a' });
},
destroy(asyncId) {
if (bootstrapIds.has(asyncId)) return;
const indentStr = ' '.repeat(indent);
fs.writeFileSync('log.out',
`${indentStr}destroy: ${asyncId}\n`, { flag: 'a' });
Expand Down Expand Up @@ -685,7 +695,7 @@ never be called.
[`after` callback]: #async_hooks_after_asyncid
[`before` callback]: #async_hooks_before_asyncid
[`destroy` callback]: #async_hooks_destroy_asyncid
[`init` callback]: #async_hooks_init_asyncid_type_triggerasyncid_resource
[`init` callback]: #async_hooks_init_asyncid_type_triggerasyncid_resource_bootstrap
[`promiseResolve` callback]: #async_hooks_promiseresolve_asyncid
[Hook Callbacks]: #async_hooks_hook_callbacks
[PromiseHooks]: https://docs.google.com/document/d/1rda3yKGHimKIhg5YeoAmCOtyURgsbTH_qaYR79FELlk/edit
Expand Down
2 changes: 1 addition & 1 deletion doc/api/n-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -5152,7 +5152,7 @@ This API may only be called from the main thread.
[Working with JavaScript Properties]: #n_api_working_with_javascript_properties
[Working with JavaScript Values - Abstract Operations]: #n_api_working_with_javascript_values_abstract_operations
[Working with JavaScript Values]: #n_api_working_with_javascript_values
[`init` hooks]: async_hooks.html#async_hooks_init_asyncid_type_triggerasyncid_resource
[`init` hooks]: async_hooks.html#async_hooks_init_asyncid_type_triggerasyncid_resource_bootstrap
[`napi_add_finalizer`]: #n_api_napi_add_finalizer
[`napi_async_init`]: #n_api_napi_async_init
[`napi_cancel_async_work`]: #n_api_napi_cancel_async_work
Expand Down
5 changes: 5 additions & 0 deletions lib/async_hooks.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ const {
emitAfter,
emitDestroy,
initHooksExist,
emitBootstrapHooksBuffer
} = internal_async_hooks;

// Get symbols
Expand Down Expand Up @@ -92,6 +93,10 @@ class AsyncHook {
enableHooks();
}

// If there are unemitted bootstrap hooks, emit them now.
// This only applies during sync execution of user code after bootsrap.
emitBootstrapHooksBuffer();

return this;
}

Expand Down
124 changes: 119 additions & 5 deletions lib/internal/async_hooks.js
Original file line number Diff line number Diff line change
Expand Up @@ -125,15 +125,16 @@ function validateAsyncId(asyncId, type) {
// Used by C++ to call all init() callbacks. Because some state can be setup
// from C++ there's no need to perform all the same operations as in
// emitInitScript.
function emitInitNative(asyncId, type, triggerAsyncId, resource) {
function emitInitNative(asyncId, type, triggerAsyncId, resource,
bootstrap = false) {
active_hooks.call_depth += 1;
// Use a single try/catch for all hooks to avoid setting up one per iteration.
try {
for (var i = 0; i < active_hooks.array.length; i++) {
if (typeof active_hooks.array[i][init_symbol] === 'function') {
active_hooks.array[i][init_symbol](
asyncId, type, triggerAsyncId,
resource
resource, bootstrap
);
}
}
Expand Down Expand Up @@ -231,6 +232,114 @@ function restoreActiveHooks() {
active_hooks.tmp_fields = null;
}

// Bootstrap hooks are buffered to emit to userland listeners
let bootstrapHooks, bootstrapBuffer;
function bufferBootstrapHooks() {
const { getOptionValue } = require('internal/options');
if (getOptionValue('--no-force-async-hooks-checks')) {
return;
}
async_hook_fields[kInit]++;
async_hook_fields[kBefore]++;
async_hook_fields[kAfter]++;
async_hook_fields[kDestroy]++;
async_hook_fields[kPromiseResolve]++;
async_hook_fields[kTotals] += 5;
bootstrapHooks = {
[init_symbol]: (asyncId, type, triggerAsyncId, resource) => {
bootstrapBuffer.push({
type: kInit,
asyncId,
args: [type, triggerAsyncId, resource, true]
});
},
[before_symbol]: (asyncId) => {
bootstrapBuffer.push({
type: kBefore,
asyncId,
args: null
});
},
[after_symbol]: (asyncId) => {
bootstrapBuffer.push({
type: kAfter,
asyncId,
args: null
});
},
[destroy_symbol]: (asyncId) => {
bootstrapBuffer.push({
type: kDestroy,
asyncId,
args: null
});
},
[promise_resolve_symbol]: (asyncId) => {
bootstrapBuffer.push({
type: kPromiseResolve,
asyncId,
args: null
});
}
};
bootstrapBuffer = [];
active_hooks.array.push(bootstrapHooks);
if (async_hook_fields[kTotals] === 5) {
enableHooks();
}
}

function clearBootstrapHooksBuffer() {
if (!bootstrapBuffer)
return;
if (bootstrapHooks) {
stopBootstrapHooksBuffer();
}
const _bootstrapBuffer = bootstrapBuffer;
bootstrapBuffer = null;
return _bootstrapBuffer;
}

function stopBootstrapHooksBuffer() {
if (!bootstrapHooks)
return;
async_hook_fields[kInit]--;
async_hook_fields[kBefore]--;
async_hook_fields[kAfter]--;
async_hook_fields[kDestroy]--;
async_hook_fields[kPromiseResolve]--;
async_hook_fields[kTotals] -= 5;
active_hooks.array.splice(active_hooks.array.indexOf(bootstrapHooks), 1);
bootstrapHooks = null;
if (async_hook_fields[kTotals] === 0) {
disableHooks();
// Ensure disable happens immediately and synchronously.
process._tickCallback();
}
}

function emitBootstrapHooksBuffer() {
const bootstrapBuffer = clearBootstrapHooksBuffer();
if (!bootstrapBuffer || async_hook_fields[kTotals] === 0) {
return;
}
for (const { type, asyncId, args } of bootstrapBuffer) {
switch (type) {
case kInit:
emitInitNative(asyncId, ...args);
break;
case kBefore:
emitBeforeNative(asyncId);
break;
case kAfter:
emitAfterNative(asyncId);
break;
case kDestroy:
emitDestroyNative(asyncId);
break;
}
}
}

let wantPromiseHook = false;
function enableHooks() {
Expand Down Expand Up @@ -318,7 +427,8 @@ function destroyHooksExist() {
}


function emitInitScript(asyncId, type, triggerAsyncId, resource) {
function emitInitScript(asyncId, type, triggerAsyncId, resource,
bootstrap = false) {
validateAsyncId(asyncId, 'asyncId');
if (triggerAsyncId !== null)
validateAsyncId(triggerAsyncId, 'triggerAsyncId');
Expand All @@ -338,7 +448,7 @@ function emitInitScript(asyncId, type, triggerAsyncId, resource) {
triggerAsyncId = getDefaultTriggerAsyncId();
}

emitInitNative(asyncId, type, triggerAsyncId, resource);
emitInitNative(asyncId, type, triggerAsyncId, resource, bootstrap);
}


Expand Down Expand Up @@ -468,5 +578,9 @@ module.exports = {
after: emitAfterNative,
destroy: emitDestroyNative,
promise_resolve: emitPromiseResolveNative
}
},
bufferBootstrapHooks,
clearBootstrapHooksBuffer,
emitBootstrapHooksBuffer,
stopBootstrapHooksBuffer
};
5 changes: 5 additions & 0 deletions lib/internal/bootstrap/pre_execution.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ const { Object, SafeWeakMap } = primordials;
const { getOptionValue } = require('internal/options');
const { Buffer } = require('buffer');
const { ERR_MANIFEST_ASSERT_INTEGRITY } = require('internal/errors').codes;
const { bufferBootstrapHooks } = require('internal/async_hooks');

function prepareMainThreadExecution(expandArgv1 = false) {
// Patch the process object with legacy properties and normalizations
Expand Down Expand Up @@ -32,6 +33,10 @@ function prepareMainThreadExecution(expandArgv1 = false) {

setupDebugEnv();

// Buffer all async hooks emitted by core bootstrap
// to allow user listeners to attach to these
bufferBootstrapHooks();

// Only main thread receives signals.
setupSignalHandlers();

Expand Down
2 changes: 1 addition & 1 deletion lib/internal/main/run_main_module.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ markBootstrapComplete();
// --experimental-modules is on.
// TODO(joyeecheung): can we move that logic to here? Note that this
// is an undocumented method available via `require('module').runMain`
CJSModule.runMain();
CJSModule.runMain(process.argv[1]);
4 changes: 2 additions & 2 deletions lib/internal/main/worker_thread.js
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,8 @@ port.on('message', (message) => {
const { evalScript } = require('internal/process/execution');
evalScript('[worker eval]', filename);
} else {
process.argv[1] = filename; // script filename
require('module').runMain();
// script filename
require('module').runMain(process.argv[1] = filename);
}
} else if (message.type === STDIO_PAYLOAD) {
const { stream, chunk, encoding } = message;
Expand Down
18 changes: 10 additions & 8 deletions lib/internal/modules/cjs/loader.js
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ const manifest = getOptionValue('--experimental-policy') ?
require('internal/process/policy').manifest :
null;
const { compileFunction } = internalBinding('contextify');

const { clearBootstrapHooksBuffer } = require('internal/async_hooks');
const {
ERR_INVALID_ARG_VALUE,
ERR_INVALID_OPT_VALUE,
Expand Down Expand Up @@ -821,10 +821,12 @@ Module.prototype.load = function(filename) {
if (module !== undefined && module.module !== undefined) {
if (module.module.getStatus() >= kInstantiated)
module.module.setExport('default', exports);
} else { // preemptively cache
} else {
// Preemptively cache
// We use a function to defer promise creation for async hooks.
ESMLoader.moduleMap.set(
url,
new ModuleJob(ESMLoader, url, () =>
() => new ModuleJob(ESMLoader, url, () =>
new ModuleWrap(function() {
this.setExport('default', exports);
}, ['default'], url)
Expand Down Expand Up @@ -1007,22 +1009,22 @@ Module._extensions['.mjs'] = function(module, filename) {
throw new ERR_REQUIRE_ESM(filename);
};

// Bootstrap main module.
Module.runMain = function() {
Module.runMain = function(mainPath) {
// Load the main module--the command line argument.
if (experimentalModules) {
asyncESM.loaderPromise.then((loader) => {
return loader.import(pathToFileURL(process.argv[1]).href);
return loader.import(pathToFileURL(mainPath).href);
})
.catch((e) => {
internalBinding('errors').triggerUncaughtException(
e,
true /* fromPromise */
);
});
return;
} else {
clearBootstrapHooksBuffer();
Module._load(mainPath, null, true);
}
Module._load(process.argv[1], null, true);
};

function createRequireFromPath(filename) {
Expand Down
3 changes: 3 additions & 0 deletions lib/internal/modules/esm/loader.js
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,9 @@ class Loader {
async getModuleJob(specifier, parentURL) {
const { url, format } = await this.resolve(specifier, parentURL);
let job = this.moduleMap.get(url);
// CJS injects jobs as functions to defer promise creation for async hooks.
if (typeof job === 'function')
this.moduleMap.set(url, job = job());
if (job !== undefined)
return job;

Expand Down
12 changes: 11 additions & 1 deletion lib/internal/modules/esm/module_job.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ const { ModuleWrap } = internalBinding('module_wrap');
const { decorateErrorStack } = require('internal/util');
const { getOptionValue } = require('internal/options');
const assert = require('internal/assert');
const { clearBootstrapHooksBuffer, stopBootstrapHooksBuffer } =
require('internal/async_hooks');
const resolvedPromise = SafePromise.resolve();

function noop() {}
Expand Down Expand Up @@ -106,7 +108,15 @@ class ModuleJob {
const module = await this.instantiate();
const timeout = -1;
const breakOnSigint = false;
return { module, result: module.evaluate(timeout, breakOnSigint) };
if (this.isMain)
stopBootstrapHooksBuffer();
const output = {
module,
result: module.evaluate(timeout, breakOnSigint)
};
if (this.isMain)
clearBootstrapHooksBuffer();
return output;
}
}
Object.setPrototypeOf(ModuleJob.prototype, null);
Expand Down
Loading