Skip to content

NarIndexer: Implement skip#15163

Merged
Ericson2314 merged 1 commit intoNixOS:masterfrom
xokdvium:faster-nar-listing
Feb 6, 2026
Merged

NarIndexer: Implement skip#15163
Ericson2314 merged 1 commit intoNixOS:masterfrom
xokdvium:faster-nar-listing

Conversation

@xokdvium
Copy link
Contributor

@xokdvium xokdvium commented Feb 6, 2026

Motivation

This improves the performance of parseNarListing, which is used by commands like nix nar ls when the underlying source allows cheap seeks (like StringSource or FdSource that does lseek).

For nix nar ls of a NAR for linux source tarball this cuts down the runtime almost in half (from 300ms -> 175ms).

Context


Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

This improves the performance of parseNarListing, which is used by commands
like `nix nar ls` when the underlying source allows cheap seeks (like StringSource
or FdSource that does lseek).

For `nix nar ls` of a NAR for linux source tarball this cuts down the runtime almost
in half (from 300ms -> 175ms).
@xokdvium xokdvium requested a review from edolstra as a code owner February 6, 2026 18:02
Copy link
Contributor

@Radvendii Radvendii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change makes sense to me, and it indeed seems to speed things up a lot

poop -d 30000 '../master/result/bin/nix nar ls ../foo.nar /' '../feature/result/bin/nix nar ls ../foo.nar /'
Benchmark 1 (147 runs): ../master/result/bin/nix nar ls ../foo.nar /
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           205ms ± 4.92ms     202ms …  252ms          9 ( 6%)        0%
  peak_rss           37.4MB ±  200KB    36.9MB … 38.0MB          0 ( 0%)        0%
  cpu_cycles          192M  ± 4.01M      186M  …  212M          11 ( 7%)        0%
  instructions        341M  ±  136K      340M  …  341M          18 (12%)        0%
  cache_references   57.3M  ± 1.32M     56.4M  … 67.5M           9 ( 6%)        0%
  cache_misses        399K  ± 7.53K      380K  …  426K           1 ( 1%)        0%
  branch_misses       755K  ± 6.49K      746K  …  801K           9 ( 6%)        0%
Benchmark 2 (683 runs): ../feature/result/bin/nix nar ls ../foo.nar /
  measurement          mean ± σ            min … max           outliers         delta
  wall_time          43.9ms ± 1.78ms    40.4ms … 48.6ms          0 ( 0%)        ⚡- 78.6% ±  0.2%
  peak_rss           37.1MB ±  204KB    36.5MB … 37.7MB          2 ( 0%)          -  0.9% ±  0.1%
  cpu_cycles         98.8M  ± 1.66M     95.6M  …  108M           5 ( 1%)        ⚡- 48.6% ±  0.2%
  instructions        239M  ± 17.1K      239M  …  239M          15 ( 2%)        ⚡- 30.0% ±  0.0%
  cache_references   4.63M  ± 79.8K     4.50M  … 5.61M          32 ( 5%)        ⚡- 91.9% ±  0.2%
  cache_misses        265K  ± 6.33K      250K  …  290K           4 ( 1%)        ⚡- 33.7% ±  0.3%
  branch_misses       496K  ± 6.86K      487K  …  566K           8 ( 1%)        ⚡- 34.3% ±  0.2%

Wall time is -78.6%

@Ericson2314 Ericson2314 enabled auto-merge February 6, 2026 18:44
@Ericson2314 Ericson2314 added this pull request to the merge queue Feb 6, 2026
Merged via the queue into NixOS:master with commit 103f912 Feb 6, 2026
15 checks passed
@xokdvium xokdvium deleted the faster-nar-listing branch February 6, 2026 19:43
@xokdvium
Copy link
Contributor Author

xokdvium commented Feb 6, 2026

@myclevorname cc. Noticed this while looking at your updated benchmarks and was wondering if we could improve anything. Turns out this content skipping wasn't working properly due to this silly mistake.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants