Stack sampling flamegraph profiler#13220
Merged
Mic92 merged 3 commits intoNixOS:masterfrom May 23, 2025
Merged
Conversation
Member
|
Needs a rebase. |
xokdvium
force-pushed
the
flamegraph
branch
3 times, most recently
from
113be6c to
7c25744
Compare
github-actions
bot
added
the
with-tests
xokdvium
changed the title
xokdvium
force-pushed
the
flamegraph
branch
3 times, most recently
from
5ef4e67 to
a80e026
Compare
Contributor
Author
|
Rebased, pushed a more polished version with tests and added an example profile for |
Mic92
reviewed
May 19, 2025
Mic92
reviewed
May 19, 2025
xokdvium
force-pushed
the
flamegraph
branch
2 times, most recently
from
7acd364 to
0275cb3
Compare
xokdvium
commented
May 19, 2025
Ericson2314
reviewed
May 21, 2025
xokdvium
commented
May 21, 2025
This patch adds support for a native stack sampling profiler to the evaluator, which saves a collapsed stack profile information to a configurable location. Introduced options (in `EvalSettings`): - `eval-profile-file` - path to the collected profile file. - `eval-profiler-frequency` - sampling frequency. - `eval-profiler` - enumeration option for enabling the profiler. Currently only `flamegraph` is supported, but having this an enumeration rather than a boolean switch leaves the door open for other profiler variants (e.g. tracy). Profile includes the following information on best-effort basis (e.g. some lambdas might have an undefined name). Callstack information contains: - Call site location (where the function gets called). - Primop/lambda name of the function being called. - Functors/partial applications don't have a name attached to them unlike special-cased primops and lambads. For cases where callsite location isn't available we have to resort to providing the location where the lambda itself is defined. This removes some of the confusing `«none»:0` locations in the profile from previous attempts. Example usage with piping directly into zstd for compression: ``` nix eval --no-eval-cache nixpkgs#nixosTests.gnome \ --eval-profiler flamegraph \ --eval-profile-file >(zstd -of nix.profile.zstd) ``` Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
The tests are mostly based on existing `function-trace.sh` with some tests for corner cases.
Ericson2314
approved these changes
May 21, 2025
Contributor
Author
Member
|
Nice looks good to me! |
This was referenced May 23, 2025
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/a-look-at-nixos-nixpkgs-evaluation-times-over-the-years/65114/3 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/2025-05-21-nix-team-meeting-minutes-228/65204/1 |
eljamm
added a commit
to eljamm/perfanno.nvim
that referenced
this pull request
Jun 15, 2025
This is useful for stack traces that have more information than the `file:line count` format. For example, the [Nix language's evaluator profiler](NixOS/nix#13220) also includes information about the function being called, which sometimes has a non-canonical call site location and therefore cannot be annotated. Here is what this might look like: ```log <nix/derivation-internal.nix>:37:12:primop derivationStrict ``` As such, with this change, we effectively skip annotating frames that don't conform to the `file:line count` format.
eljamm
added a commit
to eljamm/perfanno.nvim
that referenced
this pull request
Jun 15, 2025
This is useful for stack traces that have more information than the `file:line count` format. For example, the [Nix language's evaluator profiler](NixOS/nix#13220) also includes information about the function being called, which sometimes has a non-canonical call site location and therefore cannot be annotated. Here is how this might look like: ```log <nix/derivation-internal.nix>:37:12:primop derivationStrict ``` As such, with this change, we effectively skip annotating frames that don't conform to the `file:line count` format.
eljamm
added a commit
to eljamm/perfanno.nvim
that referenced
this pull request
Jun 15, 2025
This is useful for stack traces that have more information than the `file:line count` format. For example, the [Nix language's evaluator profiler](NixOS/nix#13220) also includes information about the function being called, which sometimes has a non-canonical call site location and therefore cannot be annotated. Here is how this might look like: ```log <nix/derivation-internal.nix>:37:12:primop derivationStrict ``` As such, with this change, we effectively skip annotating frames that don't conform to the `file:line count` format.
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
This patch adds support for a native stack sampling
profiler to the evaluator, which saves a collapsed stack
profile information to a configurable location.
Introduced options (in
EvalSettings):eval-profile-file- path to the collected profile file.eval-profiler-sample-periodeval-profiler-frequency- sampling frequency.eval-profiler- enumeration option for enabling the profiler.Currently only
flamegraphis supported, but having this anenumeration rather than a boolean switch leaves the door open
for other profiler variants (e.g. tracy).
Profile includes the following information on best-effort basis (e.g. some lambdas might
have an undefined name). Callstack information contains:
For cases where callsite location isn't available we have to resort to providing
the location where the lambda itself is defined. This removes some of the confusing
«none»:0locations in the profile from #11373.Example usage with piping directly into zstd for compression:
Sample profile for `nixosTests.gnome` with 99Hz sampling frequency
(Updated to 99Hz default)
nix.profile.gz
Context
This builds upon #11373 and hopefully leaves room for other profiler implementations like #9967.
Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.