Skip to content

libstore: Use boost::regex for GC root discovery (backport #13142)#13227

Merged
Mic92 merged 3 commits into2.29-maintenancefrom
mergify/bp/2.29-maintenance/pr-13142
May 18, 2025
Merged

libstore: Use boost::regex for GC root discovery (backport #13142)#13227
Mic92 merged 3 commits into2.29-maintenancefrom
mergify/bp/2.29-maintenance/pr-13142

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented May 18, 2025

Motivation

As it turns out using std::regex is actually the bottleneck for root discovery. Just substituting std:: -> boost:: makes root discovery twice as fast (3x if counting only userspace time).

Some rather ad-hoc measurements to motivate the switch:

(On master)

nix build github:nixos/nix/1e822bd4149a8bce1da81ee2ad9404986b07914c#nix-cli --out-link result-1e822bd4149a8bce1da81ee2ad9404986b07914c
taskset -c 2,3 hyperfine "result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run --max 0"
Benchmark 1: result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run --max 0
  Time (mean ± σ):     481.6 ms ±   3.9 ms    [User: 336.2 ms, System: 142.0 ms]
  Range (min … max):   474.6 ms … 487.7 ms    10 runs

(After this patch)

taskset -c 2,3 hyperfine "result/bin/nix store gc --dry-run --max 0"
Benchmark 1: result/bin/nix store gc --dry-run --max 0
  Time (mean ± σ):     254.7 ms ±   9.7 ms    [User: 111.1 ms, System: 141.3 ms]
  Range (min … max):   246.5 ms … 281.3 ms    10 runs

boost::regex is a drop-in replacement for std::regex, but much faster. Doing a simple before/after comparison doesn't surface any change in behavior:

result/bin/nix store gc --dry-run -vvvvv --max 0 |& grep "got additional" | wc -l
result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run -vvvvv --max 0 |& grep "got additional" | wc -l

Note

This really doesn't make a dent in the actual runtime of GC itself.

Context


Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.


This is an automatic backport of pull request #13142 done by [Mergify](https://mergify.com).

xokdvium added 3 commits May 18, 2025 19:46
As it turns out using `std::regex` is actually the bottleneck
for root discovery. Just substituting `std::` -> `boost::`
makes root discovery twice as fast (3x if counting only userspace time).

Some rather ad-hoc measurements to motivate the switch:

(On master)

```
nix build github:nixos/nix/1e822bd4149a8bce1da81ee2ad9404986b07914c#nix-cli --out-link result-1e822bd4149a8bce1da81ee2ad9404986b07914c
taskset -c 2,3 hyperfine "result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run --max 0"
Benchmark 1: result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run --max 0
  Time (mean ± σ):     481.6 ms ±   3.9 ms    [User: 336.2 ms, System: 142.0 ms]
  Range (min … max):   474.6 ms … 487.7 ms    10 runs
```

(After this patch)

```
taskset -c 2,3 hyperfine "result/bin/nix store gc --dry-run --max 0"
Benchmark 1: result/bin/nix store gc --dry-run --max 0
  Time (mean ± σ):     254.7 ms ±   9.7 ms    [User: 111.1 ms, System: 141.3 ms]
  Range (min … max):   246.5 ms … 281.3 ms    10 runs
```

`boost::regex` is a drop-in replacement for `std::regex`, but much faster.
Doing a simple before/after comparison doesn't surface any change in behavior:

```
result/bin/nix store gc --dry-run -vvvvv --max 0 |& grep "got additional" | wc -l
result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run -vvvvv --max 0 |& grep "got additional" | wc -l
```

(cherry picked from commit 3a1301c)
This reduces the closure size on master by 40MiB.

```
$ nix build github:nixos/nix/1e822bd4149a8bce1da81ee2ad9404986b07914c#nix-store --out-link closure-on-master
$ nix build .#nix-store -L --out-link closure-without-icu
$ nix path-info --closure-size -h ./closure-on-master
/nix/store/8gwr38m5h6p7245ji9jv28a2a11w1isx-nix-store-2.29.0pre  124.4 MiB
$ nix path-info --closure-size -h ./closure-without-icu
/nix/store/k0gwfykjqpnmaqbwh23nk55lhanc9g24-nix-store-2.29.0pre   86.6 MiB
```

(cherry picked from commit f3090ef)
@mergify mergify bot added automatic backport This PR is a backport produced by automation (does not trigger backporting) merge-queue labels May 18, 2025
@mergify mergify bot requested review from Ericson2314 and edolstra as code owners May 18, 2025 19:46
@mergify mergify bot added merge-queue automatic backport This PR is a backport produced by automation (does not trigger backporting) labels May 18, 2025
@Mic92 Mic92 enabled auto-merge May 18, 2025 19:47
@Mic92 Mic92 merged commit d1e4be6 into 2.29-maintenance May 18, 2025
25 checks passed
@Mic92 Mic92 deleted the mergify/bp/2.29-maintenance/pr-13142 branch May 18, 2025 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automatic backport This PR is a backport produced by automation (does not trigger backporting) merge-queue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants