inspector: implement --heap-prof #27596
Conversation
nodejs-github-bot
added
c++
|
cc @nodejs/diagnostics |
|
It would be quite useful to expose a JS API for this as well. Would you consider doing that as part of this PR? BTW, that might be one of the stronger reasons to avoid implementing this on top of the Inspector API. The |
Because the lack of serialization v8-profiler.h. I think it's best to implement this in V8 and make sure the serialization code is tested there, instead of maintaining the serialization by ourselves, since it could get broken by upstream changes and it's best that the upstream fixes it within the CL that breaks it (theoretically v8 would notice as they also run Node.js tests but that would be a much longer roundtrip). I opened https://bugs.chromium.org/p/v8/issues/detail?id=9182 and I am working on it, but before that happens and gets backported, I think it's fine to use the inspector protocol as a start. The flags are all experimental at the moment.
I would prefer to do that in a future PR, as we may need a more thought-out API design for this. See #27421 (comment). What I had in mind was something like
Thanks for the explanation, I think with the current CLI interface, we could just rewrite the internals after https://bugs.chromium.org/p/v8/issues/detail?id=9182 gets fixed? At the moment if when the process exits, the profiler has already been stopped, it would just fail silently with a message printed to the stderr. |
|
@joyeecheung can you elaborate on the JSON.parse/serialize issue you mention in https://bugs.chromium.org/p/v8/issues/detail?id=9182. I am not sure I am following. (Apologies in advance for asking for explanation, it is quite possible that I am missing something obvious in the light of the following). FYI, we have a user-space implementation of the heap profiler here: https://github.com/google/pprof-nodejs/blob/master/ts/src/heap-profiler.ts. This is usable for production workloads with ongoing monitoring. The API doesn't do any JSON encoding, and provides direct access to the profile tree to user-space. Converting the result to DevTools format is quite simple, see here. |
| return env()->heap_prof_name(); | ||
| } | ||
|
|
||
| MaybeLocal<Object> V8HeapProfilerConnection::GetProfile(Local<Object> result) { |
There was a problem hiding this comment.
@ofrobots See here and V8ProfilerConnection:ParseProfile(). Because the response returned from the protocol is:
{
"result": {
"profile": {
// ... what we need to write to the file
}
}
}
We need to do JSON::Parse and then JSON::Stringify to get to the profile. It would be better if the profile is formatted directly from the C++ data and since we already know about the schema of the returned object, we can serialize it more efficiently (similar to how heap snapshots are serialized in the API). That may be useful for a particularly big profile.
Thanks for the information! Although I believe in the end if the user wants to serialize the object, the would still need to do a I think the question here is whether the user wants to consume the profile as a JS object in-memory, or whether they just want to serialize the profile and write it / send it to somewhere else. I believe in most cases, the user would want to do the analysis offline, so simply writing the serialized profile would be more common. |
|
I moved my comment about using |
In addition implements --heap-prof-name, --heap-prof-dir and --heap-prof-interval. These flags are similar to --cpu-prof flags but they are meant for the V8 sampling heap profiler instead of the CPU profiler.
Co-Authored-By: Ali Ijaz Sheikh <[email protected]>
|
Updated the docs and bumped the default sampling interval to 512KiB. @ofrobots PTAL. |
|
Ping @ofrobots If there are no more reviews, I plan to land this later this week. |
|
Landed in 4b74dae |
In addition implements --heap-prof-name, --heap-prof-dir and --heap-prof-interval. These flags are similar to --cpu-prof flags but they are meant for the V8 sampling heap profiler instead of the CPU profiler. PR-URL: #27596 Fixes: #27421 Reviewed-By: Jan Krems <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>
In addition implements --heap-prof-name, --heap-prof-dir and --heap-prof-interval. These flags are similar to --cpu-prof flags but they are meant for the V8 sampling heap profiler instead of the CPU profiler. PR-URL: #27596 Fixes: #27421 Reviewed-By: Jan Krems <[email protected]> Reviewed-By: Anna Henningsen <[email protected]>
targos
added
the
semver-minor
Notable changes:
* doc:
* The JSON variant of the API documentation is no longer experimental
(Rich Trott) #27842.
* esm:
* JSON module support is always enabled under
`--experimental-modules`. The `--experimental-json-modules` flag
has been removed (Myles Borins)
#27752.
* http,http2:
* A new flag has been added for overriding the default HTTP server
socket timeout (which is two minutes). Pass
`--http-server-default-timeout=milliseconds`
or `--http-server-default-timeout=0` to respectively change or
disable the timeout. Starting with Node.js 13.0.0, the timeout will
be disabled by default
(Ali Ijaz Sheikh) #27704.
* inspector:
* Added an experimental `--heap-prof` flag to start the V8 heap
profiler on startup and write the heap profile to disk before exit
(Joyee Cheung) #27596.
* stream:
* The `readable.unshift()` method now correctly converts strings to
buffers. Additionally, a new optional argument is accepted to
specify the string's encoding, such as `'utf8'` or `'ascii'`
(Marcos Casagrande) #27194.
* v8:
* The object returned by `v8.getHeapStatistics()` has two new
properties: `number_of_native_contexts` and
`number_of_detached_contexts` (Yuriy Vasiyarov)
#27933.
PR-URL: #28040
Notable changes:
* doc:
* The JSON variant of the API documentation is no longer experimental
(Rich Trott) nodejs#27842.
* esm:
* JSON module support is always enabled under
`--experimental-modules`. The `--experimental-json-modules` flag
has been removed (Myles Borins)
nodejs#27752.
* http,http2:
* A new flag has been added for overriding the default HTTP server
socket timeout (which is two minutes). Pass
`--http-server-default-timeout=milliseconds`
or `--http-server-default-timeout=0` to respectively change or
disable the timeout. Starting with Node.js 13.0.0, the timeout will
be disabled by default
(Ali Ijaz Sheikh) nodejs#27704.
* inspector:
* Added an experimental `--heap-prof` flag to start the V8 heap
profiler on startup and write the heap profile to disk before exit
(Joyee Cheung) nodejs#27596.
* stream:
* The `readable.unshift()` method now correctly converts strings to
buffers. Additionally, a new optional argument is accepted to
specify the string's encoding, such as `'utf8'` or `'ascii'`
(Marcos Casagrande) nodejs#27194.
* v8:
* The object returned by `v8.getHeapStatistics()` has two new
properties: `number_of_native_contexts` and
`number_of_detached_contexts` (Yuriy Vasiyarov)
nodejs#27933.
PR-URL: nodejs#28040
|
I've marked this as |
In addition implements --heap-prof-name, --heap-prof-dir and
--heap-prof-interval.
These flags are similar to --cpu-prof flags but they are meant
for the V8 sampling heap profiler instead of the CPU profiler.
Fixes: #27421
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passes