@kbn/profiler-cli: collect and display CPU profiles #216356
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the "Elastic License | ||
| * 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side | ||
| * Public License v 1"; you may not use this file except in compliance with, at | ||
| * your election, the "Elastic License 2.0", the "GNU Affero General Public | ||
| * License v3.0 only", or the "Server Side Public License, v 1". | ||
| */ | ||
|
|
||
| require('@babel/register')({ | ||
| extensions: ['.ts', '.js'], | ||
| presets: [['@babel/preset-env', { targets: { node: 'current' } }], '@babel/preset-typescript'], | ||
| }); | ||
|
|
||
| require('@kbn/profiler-cli/cli'); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| # @kbn/profiler-cli | ||
|
|
||
| Profile Kibana while it's running, and open the CPU profile in Speedscope. | ||
|
|
||
| ## Usage | ||
|
|
||
| Run a command by either preceding it with the profiler script: | ||
| `node scripts/profile.js -- $command` | ||
|
|
||
| Or by piping it in: | ||
| `$command | node scripts/profile.js` | ||
|
|
||
| You can also just run it until SIGINT: | ||
|
|
||
| `node scripts/profile.js` | ||
|
|
||
| Or with a timeout: | ||
|
|
||
| `node scripts/profile.js --timeout=10000` | ||
|
|
||
| ## Examples | ||
|
|
||
| ### Commands | ||
|
|
||
| You can copy a curl request from the browser, and place it after the command: | ||
|
|
||
| `node scripts/profile.js --connections=10 --amount=50 -- curl ...` | ||
|
|
||
| You can also use stdin for this, for example: | ||
|
|
||
| `pbpaste | node scripts/profile.js` | ||
|
|
||
| When using stdin, take into consideration that there is some lag between starting the script and connecting the profiler, so the profiler might miss the first second or so of the running process. | ||
|
|
||
| You can also use any other command, like `autocannon`, `sleep` or `xargs`. | ||
|
|
||
| ### SigInt | ||
|
|
||
| By default, the profiler will run until the process exits:`node scripts/profile.js`. This is useful when you have a long running process running separately and you want to collect the profile over a longer time period. Be aware that this might cause memory issues because the profile will get huge. When you press Cmd+C, the profiler will gracefully exit and first write the profile to disk and open Speedscope. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
|
|
||
| require('.').cli(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
| import { run } from '@kbn/dev-cli-runner'; | ||
| import { compact, once, uniq } from 'lodash'; | ||
| import { getKibanaProcessId } from './src/get_kibana_process_id'; | ||
| import { runCommand } from './src/run_command'; | ||
| import { runUntilSigInt } from './src/run_until_sigint'; | ||
| import { getProfiler } from './src/get_profiler'; | ||
| import { untilStdinCompletes } from './src/until_stdin_completes'; | ||
|
|
||
| export function cli() { | ||
| run( | ||
| async ({ flags, log, addCleanupTask }) => { | ||
| const pid = flags.pid | ||
| ? Number(flags.pid) | ||
| : await getKibanaProcessId({ | ||
| ports: uniq(compact([Number(flags.port), 5603, 5601])), | ||
| }); | ||
|
|
||
| const controller = new AbortController(); | ||
| if (flags.timeout) { | ||
| setTimeout(() => { | ||
| controller.abort(); | ||
| }, Number(flags.timeout)); | ||
| } | ||
|
|
||
| process.kill(pid, 'SIGUSR1'); | ||
|
|
||
| const stop = once(await getProfiler({ log, type: flags.heap ? 'heap' : 'cpu' })); | ||
|
|
||
| addCleanupTask(() => { | ||
| // exit-hook, which is used by addCleanupTask, | ||
| // only allows for synchronous exits, and 3.x | ||
| // are on ESM which we currently can't use. so | ||
| // we do a really gross thing where we make | ||
| // process.exit a noop for a bit until the | ||
| // profile has been collected and opened | ||
| const exit = process.exit.bind(process); | ||
|
|
||
| // @ts-expect-error | ||
| process.exit = () => {}; | ||
|
Member
There was a problem hiding this comment. QQ: Cant we use the process.on('exit') event and stop?
Contributor
Author
There was a problem hiding this comment. docs say:
so can't do any async stuff there. this wouldn't be necessary if we could upgrade to exit-hook 4.x which supports async callbacks, but we can't use ESM (yet) |
||
|
|
||
| stop() | ||
| .then(() => { | ||
| exit(0); | ||
| }) | ||
| .catch((error) => { | ||
| log.error(error); | ||
| exit(1); | ||
| }); | ||
| }); | ||
|
|
||
| if (!process.stdin.isTTY) { | ||
| await untilStdinCompletes(); | ||
| } else if (flags._.length) { | ||
| const connections = Number(flags.c || flags.connections || 1); | ||
| const amount = Number(flags.a || flags.amount || 1); | ||
| const command = flags._; | ||
|
|
||
| log.info(`Executing "${command}" ${amount} times, ${connections} at a time`); | ||
|
|
||
| await runCommand({ | ||
| command, | ||
| connections, | ||
| amount, | ||
| signal: controller.signal, | ||
| }); | ||
| } else { | ||
| if (flags.timeout) { | ||
| log.info(`Awaiting timeout of ${flags.timeout}ms`); | ||
| } else { | ||
| log.info(`Awaiting SIGINT (Cmd+C)...`); | ||
| } | ||
|
|
||
| await runUntilSigInt({ | ||
| log, | ||
| signal: controller.signal, | ||
| }); | ||
| } | ||
|
|
||
| await stop(); | ||
| }, | ||
| { | ||
| flags: { | ||
| string: ['port', 'pid', 't', 'timeout', 'c', 'connections', 'a', 'amount'], | ||
| boolean: ['heap'], | ||
| help: ` | ||
| Usage: node scripts/profiler.js <args> <command> | ||
|
|
||
| --port Port on which Kibana is running. Falls back to 5603 & 5601. | ||
| --pid Process ID to hook into it. Takes precedence over \`port\`. | ||
| --timeout Run commands until timeout (in milliseconds) | ||
| --c, --connections Number of commands that can be run in parallel. | ||
| --a, --amount Amount of times the command should be run | ||
| --heap Collect a heap snapshot | ||
| `, | ||
| allowUnexpected: false, | ||
| }, | ||
| } | ||
| ); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
|
|
||
| module.exports = { | ||
| preset: '@kbn/test/jest_node', | ||
| rootDir: '../../../../..', | ||
| roots: ['<rootDir>/x-pack/platform/packages/shared/kbn-profiler-cli'], | ||
| }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| { | ||
| "type": "shared-server", | ||
| "id": "@kbn/profiler-cli", | ||
| "owner": "@elastic/obs-knowledge-team", | ||
| "group": "platform", | ||
| "visibility": "shared" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| { | ||
| "name": "@kbn/profiler-cli", | ||
| "private": true, | ||
| "version": "1.0.0", | ||
| "license": "Elastic License 2.0" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| /* | ||
| * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
| * or more contributor license agreements. Licensed under the Elastic License | ||
| * 2.0; you may not use this file except in compliance with the Elastic License | ||
| * 2.0. | ||
| */ | ||
| import execa from 'execa'; | ||
|
|
||
| async function getProcessIdAtPort(port: number) { | ||
| return await execa | ||
| .command(`lsof -ti :${port}`) | ||
| .then(({ stdout }) => { | ||
| return parseInt(stdout.trim().split('\n')[0], 10); | ||
| }) | ||
| .catch((error) => { | ||
| return undefined; | ||
| }); | ||
| } | ||
|
|
||
| export async function getKibanaProcessId({ ports }: { ports: number[] }): Promise<number> { | ||
| for (const port of ports) { | ||
| const pid = await getProcessIdAtPort(port); | ||
| if (pid) { | ||
| return pid; | ||
| } | ||
| } | ||
|
|
||
| throw new Error(`Kibana process id not found at ports ${ports.join(', ')}`); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@afharo I've taken the liberty of only enabling these libs when
--devis explicitly set. They're pretty costly in terms of CPU and create a memory leak. If you have a better way to do this, I'm all ears.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@afharo you good with this? I'm not sure if this is the right way to do this