@kbn/profiler-cli: collect and display CPU profiles#216356
Merged
dgieselaar merged 18 commits intoelastic:mainfrom Apr 2, 2025
Merged
@kbn/profiler-cli: collect and display CPU profiles#216356dgieselaar merged 18 commits intoelastic:mainfrom
dgieselaar merged 18 commits intoelastic:mainfrom
Conversation
dgieselaar
added
release_note:skip
dgieselaar
commented
Mar 29, 2025
| require('trace'); | ||
| // `clarify` tries to remove Node.js' internal libraries from the stack trace to make it a bit more human-friendly. | ||
| require('clarify'); | ||
| const isDev = process.argv.includes('--dev'); |
Contributor
Author
There was a problem hiding this comment.
@afharo I've taken the liberty of only enabling these libs when --dev is explicitly set. They're pretty costly in terms of CPU and create a memory leak. If you have a better way to do this, I'm all ears.
Contributor
Author
There was a problem hiding this comment.
@afharo you good with this? I'm not sure if this is the right way to do this
Ikuni17
approved these changes
Mar 31, 2025
afharo
approved these changes
Apr 1, 2025
Member
afharo
left a comment
afharo
left a comment
There was a problem hiding this comment.
Approving to unblock. Just left a question to make sure that we don't confuse users.
| )}ms out of ${Math.round(duration)}ms) and ${eluThreshold * 100}% (${Math.round( | ||
| utilization * 100 | ||
| )}%) `, | ||
| )}%). Run \`node scripts/profile.js\` to find out why.`, |
Member
There was a problem hiding this comment.
Q: is this script valid in production? If not, can we log this part only when running in dev mode?
Contributor
Author
There was a problem hiding this comment.
they're only enabled in dev mode (by default). someone would have to explicitly turn them on in non-dev mode. That being said, as long as someone has access to the machine that Kibana is running on, then yes, they can use node scripts/profile.js.
Contributor
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
History
|
Contributor
|
Starting backport for target branches: 8.x https://github.com/elastic/kibana/actions/runs/14213298647 |
Contributor
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
Contributor
Author
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
dgieselaar
added a commit
to dgieselaar/kibana
that referenced
this pull request
Apr 2, 2025
# @kbn/profiler-cli Profile Kibana while it's running, and open the CPU profile in Speedscope. ## Usage Run a command by either preceding it with the profiler script: `node scripts/profile.js -- $command` Or by piping it in: `$command | node scripts/profile.js` You can also just run it until SIGINT: `node scripts/profile.js` Or with a timeout: `node scripts/profile.js --timeout=10000` ## Examples ### Commands You can copy a curl request from the browser, and place it after the command: `node scripts/profile.js --connections=10 --amount=50 -- curl ...` You can also use stdin for this, for example: `pbpaste | node scripts/profile.js` When using stdin, take into consideration that there is some lag between starting the script and connecting the profiler, so the profiler might miss the first second or so of the running process. You can also use any other command, like `autocannon`, `sleep` or `xargs`. ### SigInt By default, the profiler will run until the process exits:`node scripts/profile.js`. This is useful when you have a long running process running separately and you want to collect the profile over a longer time period. Be aware that this might cause memory issues because the profile will get huge. When you press Cmd+C, the profiler will gracefully exit and first write the profile to disk and open Speedscope. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit d1493b9) # Conflicts: # .github/CODEOWNERS # src/cli/dev.js
This was referenced Apr 2, 2025
| const exit = process.exit.bind(process); | ||
|
|
||
| // @ts-expect-error | ||
| process.exit = () => {}; |
Member
There was a problem hiding this comment.
QQ: Cant we use the process.on('exit') event and stop?
Contributor
Author
There was a problem hiding this comment.
docs say:
There is no way to prevent the exiting of the event loop at this point, and once all 'exit' listeners have finished running the Node.js process will terminate.
so can't do any async stuff there. this wouldn't be necessary if we could upgrade to exit-hook 4.x which supports async callbacks, but we can't use ESM (yet)
dgieselaar
added a commit
that referenced
this pull request
Apr 3, 2025
…216774) # Backport This will backport the following commits from `main` to `8.x`: - [@kbn/profiler-cli: collect and display CPU profiles (#216356)](#216356) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Dario Gieselaar","email":"dario.gieselaar@elastic.co"},"sourceCommit":{"committedDate":"2025-04-02T06:47:33Z","message":"@kbn/profiler-cli: collect and display CPU profiles (#216356)\n\n# @kbn/profiler-cli\n\nProfile Kibana while it's running, and open the CPU profile in\nSpeedscope.\n\n## Usage\n\nRun a command by either preceding it with the profiler script:\n`node scripts/profile.js -- $command`\n\nOr by piping it in:\n`$command | node scripts/profile.js`\n\nYou can also just run it until SIGINT:\n\n`node scripts/profile.js`\n\nOr with a timeout:\n\n`node scripts/profile.js --timeout=10000`\n\n## Examples\n\n### Commands\n\nYou can copy a curl request from the browser, and place it after the\ncommand:\n\n`node scripts/profile.js --connections=10 --amount=50 -- curl ...`\n\nYou can also use stdin for this, for example:\n\n`pbpaste | node scripts/profile.js`\n\nWhen using stdin, take into consideration that there is some lag between\nstarting the script and connecting the profiler, so the profiler might\nmiss the first second or so of the running process.\n\nYou can also use any other command, like `autocannon`, `sleep` or\n`xargs`.\n\n### SigInt\n\nBy default, the profiler will run until the process exits:`node\nscripts/profile.js`. This is useful when you have a long running process\nrunning separately and you want to collect the profile over a longer\ntime period. Be aware that this might cause memory issues because the\nprofile will get huge. When you press Cmd+C, the profiler will\ngracefully exit and first write the profile to disk and open Speedscope.\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"d1493b98e71db3a66014e29ba3264fcf21155f2e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:version","v9.1.0","v8.19.0"],"title":"@kbn/profiler-cli: collect and display CPU profiles","number":216356,"url":"https://github.com/elastic/kibana/pull/216356","mergeCommit":{"message":"@kbn/profiler-cli: collect and display CPU profiles (#216356)\n\n# @kbn/profiler-cli\n\nProfile Kibana while it's running, and open the CPU profile in\nSpeedscope.\n\n## Usage\n\nRun a command by either preceding it with the profiler script:\n`node scripts/profile.js -- $command`\n\nOr by piping it in:\n`$command | node scripts/profile.js`\n\nYou can also just run it until SIGINT:\n\n`node scripts/profile.js`\n\nOr with a timeout:\n\n`node scripts/profile.js --timeout=10000`\n\n## Examples\n\n### Commands\n\nYou can copy a curl request from the browser, and place it after the\ncommand:\n\n`node scripts/profile.js --connections=10 --amount=50 -- curl ...`\n\nYou can also use stdin for this, for example:\n\n`pbpaste | node scripts/profile.js`\n\nWhen using stdin, take into consideration that there is some lag between\nstarting the script and connecting the profiler, so the profiler might\nmiss the first second or so of the running process.\n\nYou can also use any other command, like `autocannon`, `sleep` or\n`xargs`.\n\n### SigInt\n\nBy default, the profiler will run until the process exits:`node\nscripts/profile.js`. This is useful when you have a long running process\nrunning separately and you want to collect the profile over a longer\ntime period. Be aware that this might cause memory issues because the\nprofile will get huge. When you press Cmd+C, the profiler will\ngracefully exit and first write the profile to disk and open Speedscope.\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"d1493b98e71db3a66014e29ba3264fcf21155f2e"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/216356","number":216356,"mergeCommit":{"message":"@kbn/profiler-cli: collect and display CPU profiles (#216356)\n\n# @kbn/profiler-cli\n\nProfile Kibana while it's running, and open the CPU profile in\nSpeedscope.\n\n## Usage\n\nRun a command by either preceding it with the profiler script:\n`node scripts/profile.js -- $command`\n\nOr by piping it in:\n`$command | node scripts/profile.js`\n\nYou can also just run it until SIGINT:\n\n`node scripts/profile.js`\n\nOr with a timeout:\n\n`node scripts/profile.js --timeout=10000`\n\n## Examples\n\n### Commands\n\nYou can copy a curl request from the browser, and place it after the\ncommand:\n\n`node scripts/profile.js --connections=10 --amount=50 -- curl ...`\n\nYou can also use stdin for this, for example:\n\n`pbpaste | node scripts/profile.js`\n\nWhen using stdin, take into consideration that there is some lag between\nstarting the script and connecting the profiler, so the profiler might\nmiss the first second or so of the running process.\n\nYou can also use any other command, like `autocannon`, `sleep` or\n`xargs`.\n\n### SigInt\n\nBy default, the profiler will run until the process exits:`node\nscripts/profile.js`. This is useful when you have a long running process\nrunning separately and you want to collect the profile over a longer\ntime period. Be aware that this might cause memory issues because the\nprofile will get huge. When you press Cmd+C, the profiler will\ngracefully exit and first write the profile to disk and open Speedscope.\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"d1493b98e71db3a66014e29ba3264fcf21155f2e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@kbn/profiler-cli
Profile Kibana while it's running, and open the CPU profile in Speedscope.
Usage
Run a command by either preceding it with the profiler script:
node scripts/profile.js -- $commandOr by piping it in:
$command | node scripts/profile.jsYou can also just run it until SIGINT:
node scripts/profile.jsOr with a timeout:
node scripts/profile.js --timeout=10000Examples
Commands
You can copy a curl request from the browser, and place it after the command:
node scripts/profile.js --connections=10 --amount=50 -- curl ...You can also use stdin for this, for example:
pbpaste | node scripts/profile.jsWhen using stdin, take into consideration that there is some lag between starting the script and connecting the profiler, so the profiler might miss the first second or so of the running process.
You can also use any other command, like
autocannon,sleeporxargs.SigInt
By default, the profiler will run until the process exits:
node scripts/profile.js. This is useful when you have a long running process running separately and you want to collect the profile over a longer time period. Be aware that this might cause memory issues because the profile will get huge. When you press Cmd+C, the profiler will gracefully exit and first write the profile to disk and open Speedscope.